From julianokyap at gmail.com  Fri Aug  1 01:50:22 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 13:50:22 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
Message-ID: <d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>

Yes, this would be correct.

In my example, a particular message string would be the same.

Example:
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad

So then is could be $msg == 'This is really bad' or perhaps $msg
contains 'really bad'.  But for me, matching the exact $msg would be
fine. :P


On Thu, Jul 31, 2008 at 10:38 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence:
>
> 1. a - a - a - b
> 2. a - a - b - a
>
> Result: in case 1 an alert is triggered, in case 2 not.
>
> Is this understanding correct?
>
> rainer
>
> ----- Urspr?ngliche Nachricht -----
> Von: "Julian Yap" <julianokyap at gmail.com>
> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> Cc: "rgerhards at hq.adiscon.com" <rgerhards at hq.adiscon.com>; "hks.private at gmail.com" <hks.private at gmail.com>
> Gesendet: 31.07.08 21:59
> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
>
> That's pretty much it for now.  I've written Alerts for single line
> events.  But for one particular event, it's only really a factor if it
> happens tree times in a row.
>
>
> On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> What exactly do you need to do except the "three in a row" alert?
>>
>> ----- Urspr?ngliche Nachricht -----
>> Von: "Julian Yap" <julianokyap at gmail.com>
>> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
>> Gesendet: 31.07.08 20:27
>> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> Hmm, Nagios is a pain to set up.  Looking for something more light
>> weight...  Was hoping that I could have consolidated lots of Alerts
>> under Rsyslog.
>>
>> Any other suggestions besides Swatch?
>>
>>
>>
>> On 7/31/08, (private) HKS <hks.private at gmail.com> wrote:
>>> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or
>>> some other monitoring-type software.
>>>
>>> -HKS
>>>
>>> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap <julianokyap at gmail.com> wrote:
>>>> Is there a way to set an Alert when multiple repeated lines are found in a
>>>> log?
>>>>
>>>> I want to spawn an email Alert if a message is received 3 times.
>>>>
>>>> Example log lines:
>>>> Jul 30 04:19:29 localhost program: Error detected
>>>> Jul 30 05:19:29 localhost program: Error detected
>>>> Jul 30 06:19:29 localhost program: Error detected
>>>>
>>>> Thanks,
>>>> Julian
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>
>


From rgerhards at hq.adiscon.com  Fri Aug  1 07:45:15 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 07:45:15 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>

Just to make sure:

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:04 server program: This is really bad

[Note the last timestamp!] would still make up for "three in a row"?

Now, syslog contains not only timestamps, but also hostnames. So how about

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server2 program: This is really bad
Jul 31 13:45:03 server program: This is really bad

And what about this:

Jul 31 13:45:03 server program2: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad

And would that trigger any alert at all:

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server2 program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program2: This is really bad
Jul 31 13:45:03 server program: This is really bad

Finally, does "three in a row" time out? So what would happen in the case below. Watch the timestamps and let's assume there are no other records inside the log:

Jul 29 13:45:03 server program: This is really bad
Jul 30 13:45:03 server program: This is really bad
Jul 31 13:45:04 server program: This is really bad

And a related question. You write:

> So then is could be $msg == 'This is really bad' or perhaps $msg
> contains 'really bad'.  But for me, matching the exact $msg would be
> fine. :P

Does this imply you would like to do the "if $msg == 'this is really bad'" check? So far, I assume you do NOT do this but expect an alert to be raised whenever ANY messages fulfills the identity criterion n times in a row. Please explain.

Thanks,
Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 1:50 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Yes, this would be correct.
> 
> In my example, a particular message string would be the same.
> 
> Example:
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> 
> So then is could be $msg == 'This is really bad' or perhaps $msg
> contains 'really bad'.  But for me, matching the exact $msg would be
> fine. :P
> 
> 
> On Thu, Jul 31, 2008 at 10:38 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > To clarify: be "a" the event in question and "b" any other event. Two
> samples of an event sequence:
> >
> > 1. a - a - a - b
> > 2. a - a - b - a
> >
> > Result: in case 1 an alert is triggered, in case 2 not.
> >
> > Is this understanding correct?
> >
> > rainer
> >
> > ----- Urspr?ngliche Nachricht -----
> > Von: "Julian Yap" <julianokyap at gmail.com>
> > An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> > Cc: "rgerhards at hq.adiscon.com" <rgerhards at hq.adiscon.com>;
> "hks.private at gmail.com" <hks.private at gmail.com>
> > Gesendet: 31.07.08 21:59
> > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
> >
> > That's pretty much it for now.  I've written Alerts for single line
> > events.  But for one particular event, it's only really a factor if
> it
> > happens tree times in a row.
> >
> >
> > On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> What exactly do you need to do except the "three in a row" alert?
> >>
> >> ----- Urspr?ngliche Nachricht -----
> >> Von: "Julian Yap" <julianokyap at gmail.com>
> >> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> >> Gesendet: 31.07.08 20:27
> >> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
> >>
> >> Hmm, Nagios is a pain to set up.  Looking for something more light
> >> weight...  Was hoping that I could have consolidated lots of Alerts
> >> under Rsyslog.
> >>
> >> Any other suggestions besides Swatch?
> >>
> >>
> >>
> >> On 7/31/08, (private) HKS <hks.private at gmail.com> wrote:
> >>> Not in rsyslogd itself, but you could do this with Swatch, Nagios,
> or
> >>> some other monitoring-type software.
> >>>
> >>> -HKS
> >>>
> >>> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap <julianokyap at gmail.com>
> wrote:
> >>>> Is there a way to set an Alert when multiple repeated lines are
> found in a
> >>>> log?
> >>>>
> >>>> I want to spawn an email Alert if a message is received 3 times.
> >>>>
> >>>> Example log lines:
> >>>> Jul 30 04:19:29 localhost program: Error detected
> >>>> Jul 30 05:19:29 localhost program: Error detected
> >>>> Jul 30 06:19:29 localhost program: Error detected
> >>>>
> >>>> Thanks,
> >>>> Julian
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>
> >


From rgerhards at hq.adiscon.com  Fri Aug  1 07:58:10 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 07:58:10 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>

Sorry for being brief, I was on the road and needed to write from a cell phone :(

It also looks like I was wrong ;) I have reviewed the case. If I understand correctly, you get you data from the different unix sockets and then forward via a single rule. So you can not use different templates (which most often are the real solution to any such problem, but are not obvious to most people not directly involved in rsyslog ;)).

Is my understanding correct now?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, July 31, 2008 5:04 PM
> To: David Darville; rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> Use a template with fixed name.
> 
> --- Urspr?ngliche Nachricht ---
> Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> Betreff: Re: [rsyslog] Changing hostname field
> Datum: 31. Juli 2008
> Uhrzeit: 16:46:59
> 
> The jails all have their own unique hostname (and IP), but all share an
> rsyslogd instance running on the main host, and the %hostname% and
> %fromhost% in all the log messages from the jails are set to the
> hostname of
> the main host. And that is what I want to change.
> 
> On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > Do the jails all share the same hostname and IP? If not, you should
> be
> > able to use the %hostname% or %fromhost% properties.
> >
> > If so, are they each running their own instance of (r)syslogd?
> >
> > -HKS
> >
> > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > <ml at darville.vm.bytemark.co.uk> wrote:
> > > Hello everyone
> > >
> > > I am trying to configure rsyslog to service a number of chroot
> jails in
> > > addition to the host itself.
> > >
> > > But I need to change the hostname field of the syslog messages from
> the
> > > different jails, so that I place them in the right log file on the
> central
> > > logging host.
> > >
> > > My current rsyslog.conf is as follows:
> > >
> > > $ModLoad imuxsock
> > > $ModLoad imklog
> > > $ModLoad immark
> > > $ModLoad omrelp
> > >
> > > $AddUnixListenSocket /jail/1/dev/log
> > > $AddUnixListenSocket /jail/2/dev/log
> > >
> > > *.* :omrelp:10.0.0.4:2514
> > >
> > >
> > > Can anyone please advice me on how to do that?
> > >
> > >
> > > ---
> > >
> > > David Darville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From julianokyap at gmail.com  Fri Aug  1 09:47:02 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 21:47:02 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>

Hi Rainer,

Thanks for taking the time to work on my issue.  I'll approach this
from my situation.

On Thu, Jul 31, 2008 at 7:45 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just to make sure:
>
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:04 server program: This is really bad
>
> [Note the last timestamp!] would still make up for "three in a row"?

Yes.  Something that would be nice would be 'three in a row in the
last x minutes'...  But let's keep things simple for now :)

> Now, syslog contains not only timestamps, but also hostnames. So how about
>
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server2 program: This is really bad
> Jul 31 13:45:03 server program: This is really bad

In my particular case, it would only come from the one server.  But if
I had 2 servers logging to the same log file, it should have the
option to filter based on server and/or message.

> And what about this:
>
> Jul 31 13:45:03 server program2: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
>
> And would that trigger any alert at all:

In my particular case, it would only come from the one program.  But
if I had 2 programs logging to the same log file, it should have the
option to filter based on program and/or message.

> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server2 program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program2: This is really bad
> Jul 31 13:45:03 server program: This is really bad
>
> Finally, does "three in a row" time out? So what would happen in the case below. Watch the timestamps and let's assume there are no other records inside the log:

In my particular case, 'three in a row' wouldn't time out.  I would
use ActionMail so I would use a corresponding
$ActionExecOnlyOnceEveryInterval value.

For my particular case, it happens when a process totally locks up.  A
very rare instance which requires manually restarting a process...  So
the 'This is really bad' messages would eventually stop.

> Jul 29 13:45:03 server program: This is really bad
> Jul 30 13:45:03 server program: This is really bad
> Jul 31 13:45:04 server program: This is really bad
>
> And a related question. You write:
>
>> So then is could be $msg == 'This is really bad' or perhaps $msg
>> contains 'really bad'.  But for me, matching the exact $msg would be
>> fine. :P
>
> Does this imply you would like to do the "if $msg == 'this is really bad'" check? So far, I assume you do NOT do this but expect an alert to be raised whenever ANY messages fulfills the identity criterion n times in a row. Please explain.

Yes, you assume correctly.

What I'm looking for:
if $msg == 'This is really bad' happens 3 times in a row then :ommail:;mailBody

This would be nice but is not required since the 'This is really bad'
message in my case is very unique:
if ($msg == 'This is really bad' and $server == 'server' and $program
== 'program') happens 3 times in a row then :ommail:;mailBody

- Julian

> Thanks,
> Rainer


From rgerhards at hq.adiscon.com  Fri Aug  1 10:18:59 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 10:18:59 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>

Just one more re-confirmation:

> What I'm looking for:
> if $msg == 'This is really bad' happens 3 times in a row then
> :ommail:;mailBody
> 
> This would be nice but is not required since the 'This is really bad'
> message in my case is very unique:
> if ($msg == 'This is really bad' and $server == 'server' and $program
> == 'program') happens 3 times in a row then :ommail:;mailBody

So you would actually use such a rule. If "this other thing is really
bad" happened three times, the rule shall not trigger. Is this right?

Rainer


From julianokyap at gmail.com  Fri Aug  1 11:02:32 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 23:02:32 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>

On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just one more re-confirmation:
>
>> What I'm looking for:
>> if $msg == 'This is really bad' happens 3 times in a row then
>> :ommail:;mailBody
>>
>> This would be nice but is not required since the 'This is really bad'
>> message in my case is very unique:
>> if ($msg == 'This is really bad' and $server == 'server' and $program
>> == 'program') happens 3 times in a row then :ommail:;mailBody
>
> So you would actually use such a rule. If "this other thing is really
> bad" happened three times, the rule shall not trigger. Is this right?

Yes, I would use such a rule.  It would make what is already an
awesome application even more awesome. :P  I am also willing to test
it out and run the latest development version...  Which I'm doing
anyway.

And yes, what you just wrote is correct.

- Julian


From rgerhards at hq.adiscon.com  Fri Aug  1 11:58:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 11:58:19 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>

OK, that greatly simplifies things. Actually, it now boils down to
"execute an action only on the n-the time the filter evaluates to true".
I think this is quite easy to implement, but I must verify that...

Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 11:03 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Just one more re-confirmation:
> >
> >> What I'm looking for:
> >> if $msg == 'This is really bad' happens 3 times in a row then
> >> :ommail:;mailBody
> >>
> >> This would be nice but is not required since the 'This is really
> bad'
> >> message in my case is very unique:
> >> if ($msg == 'This is really bad' and $server == 'server' and
> $program
> >> == 'program') happens 3 times in a row then :ommail:;mailBody
> >
> > So you would actually use such a rule. If "this other thing is
really
> > bad" happened three times, the rule shall not trigger. Is this
right?
> 
> Yes, I would use such a rule.  It would make what is already an
> awesome application even more awesome. :P  I am also willing to test
> it out and run the latest development version...  Which I'm doing
> anyway.
> 
> And yes, what you just wrote is correct.
> 
> - Julian


From julianokyap at gmail.com  Fri Aug  1 12:13:50 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Fri, 1 Aug 2008 00:13:50 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>

Roger that Rainer.

Thanks,
Julian

On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> OK, that greatly simplifies things. Actually, it now boils down to
> "execute an action only on the n-the time the filter evaluates to true".
> I think this is quite easy to implement, but I must verify that...
>
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 01, 2008 11:03 AM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>> > Just one more re-confirmation:
>> >
>> >> What I'm looking for:
>> >> if $msg == 'This is really bad' happens 3 times in a row then
>> >> :ommail:;mailBody
>> >>
>> >> This would be nice but is not required since the 'This is really
>> bad'
>> >> message in my case is very unique:
>> >> if ($msg == 'This is really bad' and $server == 'server' and
>> $program
>> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>> >
>> > So you would actually use such a rule. If "this other thing is
> really
>> > bad" happened three times, the rule shall not trigger. Is this
> right?
>>
>> Yes, I would use such a rule.  It would make what is already an
>> awesome application even more awesome. :P  I am also willing to test
>> it out and run the latest development version...  Which I'm doing
>> anyway.
>>
>> And yes, what you just wrote is correct.
>>
>> - Julian
>


From ml at darville.vm.bytemark.co.uk  Fri Aug  1 12:26:50 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Fri, 1 Aug 2008 11:26:50 +0100
Subject: [rsyslog] Changing hostname field
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
Message-ID: <20080801102649.GA27628@darville.vm.bytemark.co.uk>

On Fri, Aug 01, 2008 at 07:58:10AM +0200, Rainer Gerhards wrote:
> Sorry for being brief, I was on the road and needed to write from
> a cell phone :(

Supporting rsyslog on your cell phone - you are very dedicated to you
product ;-)

> It also looks like I was wrong ;) I have reviewed the case. If I
> understand correctly, you get you data from the different unix sockets
> and then forward via a single rule. So you can not use different
> templates (which most often are the real solution to any such problem,
> but are not obvious to most people not directly involved in rsyslog ;)).
>
> Is my understanding correct now?

Yes, and then I'm splitting log messages on the dedicated log host, based on
the hostname field. 

We are keeping the rsyslogd process outside the jails, to to keep the syslog
traffic on an isolated network segment.

-- David

> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, July 31, 2008 5:04 PM
> > To: David Darville; rsyslog-users
> > Subject: Re: [rsyslog] Changing hostname field
> > 
> > Use a template with fixed name.
> > 
> > --- Urspr?ngliche Nachricht ---
> > Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> > Betreff: Re: [rsyslog] Changing hostname field
> > Datum: 31. Juli 2008
> > Uhrzeit: 16:46:59
> > 
> > The jails all have their own unique hostname (and IP), but all share an
> > rsyslogd instance running on the main host, and the %hostname% and
> > %fromhost% in all the log messages from the jails are set to the
> > hostname of
> > the main host. And that is what I want to change.
> > 
> > On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > > Do the jails all share the same hostname and IP? If not, you should
> > be
> > > able to use the %hostname% or %fromhost% properties.
> > >
> > > If so, are they each running their own instance of (r)syslogd?
> > >
> > > -HKS
> > >
> > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > > <ml at darville.vm.bytemark.co.uk> wrote:
> > > > Hello everyone
> > > >
> > > > I am trying to configure rsyslog to service a number of chroot
> > jails in
> > > > addition to the host itself.
> > > >
> > > > But I need to change the hostname field of the syslog messages from
> > the
> > > > different jails, so that I place them in the right log file on the
> > central
> > > > logging host.
> > > >
> > > > My current rsyslog.conf is as follows:
> > > >
> > > > $ModLoad imuxsock
> > > > $ModLoad imklog
> > > > $ModLoad immark
> > > > $ModLoad omrelp
> > > >
> > > > $AddUnixListenSocket /jail/1/dev/log
> > > > $AddUnixListenSocket /jail/2/dev/log
> > > >
> > > > *.* :omrelp:10.0.0.4:2514
> > > >
> > > >
> > > > Can anyone please advice me on how to do that?
> > > >
> > > >
> > > > ---
> > > >
> > > > David Darville
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Fri Aug  1 15:06:43 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 15:06:43 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <20080801102649.GA27628@darville.vm.bytemark.co.uk>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
	<20080801102649.GA27628@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>

Inline...

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Friday, August 01, 2008 12:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> On Fri, Aug 01, 2008 at 07:58:10AM +0200, Rainer Gerhards wrote:
> > Sorry for being brief, I was on the road and needed to write from
> > a cell phone :(
> 
> Supporting rsyslog on your cell phone - you are very dedicated to you
> product ;-)

Thanks - I am trying my best ;) And, if you have not already done so, I'd appreciate if you could rate rsyslog at freshmeat.net, it's just a simple click (even if you need to create an account it doesn't take long). It helps us get momentum and this is as important as code quality ;)

You can do it here (middle of the page):
http://freshmeat.net/projects/rsyslog/

> 
> > It also looks like I was wrong ;) I have reviewed the case. If I
> > understand correctly, you get you data from the different unix
> sockets
> > and then forward via a single rule. So you can not use different
> > templates (which most often are the real solution to any such
> problem,
> > but are not obvious to most people not directly involved in rsyslog
> ;)).
> >
> > Is my understanding correct now?
> 
> Yes, and then I'm splitting log messages on the dedicated log host,
> based on
> the hostname field.
> 
> We are keeping the rsyslogd process outside the jails, to to keep the
> syslog
> traffic on an isolated network segment.

OK, that was fairly easy to do. I have created a special version for you:

http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz

It contains a new config directive that allows you to specify the hostname, see last sample on

http://www.rsyslog.com/doc-imuxsock.html

In my lab it works, but I would appreciate if you could test and verify.

Thanks,
Rainer

> 
> -- David
> 
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > Sent: Thursday, July 31, 2008 5:04 PM
> > > To: David Darville; rsyslog-users
> > > Subject: Re: [rsyslog] Changing hostname field
> > >
> > > Use a template with fixed name.
> > >
> > > --- Urspr?ngliche Nachricht ---
> > > Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> > > Betreff: Re: [rsyslog] Changing hostname field
> > > Datum: 31. Juli 2008
> > > Uhrzeit: 16:46:59
> > >
> > > The jails all have their own unique hostname (and IP), but all
> share an
> > > rsyslogd instance running on the main host, and the %hostname% and
> > > %fromhost% in all the log messages from the jails are set to the
> > > hostname of
> > > the main host. And that is what I want to change.
> > >
> > > On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > > > Do the jails all share the same hostname and IP? If not, you
> should
> > > be
> > > > able to use the %hostname% or %fromhost% properties.
> > > >
> > > > If so, are they each running their own instance of (r)syslogd?
> > > >
> > > > -HKS
> > > >
> > > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > > > <ml at darville.vm.bytemark.co.uk> wrote:
> > > > > Hello everyone
> > > > >
> > > > > I am trying to configure rsyslog to service a number of chroot
> > > jails in
> > > > > addition to the host itself.
> > > > >
> > > > > But I need to change the hostname field of the syslog messages
> from
> > > the
> > > > > different jails, so that I place them in the right log file on
> the
> > > central
> > > > > logging host.
> > > > >
> > > > > My current rsyslog.conf is as follows:
> > > > >
> > > > > $ModLoad imuxsock
> > > > > $ModLoad imklog
> > > > > $ModLoad immark
> > > > > $ModLoad omrelp
> > > > >
> > > > > $AddUnixListenSocket /jail/1/dev/log
> > > > > $AddUnixListenSocket /jail/2/dev/log
> > > > >
> > > > > *.* :omrelp:10.0.0.4:2514
> > > > >
> > > > >
> > > > > Can anyone please advice me on how to do that?
> > > > >
> > > > >
> > > > > ---
> > > > >
> > > > > David Darville
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Fri Aug  1 16:25:02 2008
From: hks.private at gmail.com ((private) HKS)
Date: Fri, 1 Aug 2008 10:25:02 -0400
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
Message-ID: <ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>

This may be a silly question, but is the syslog daemon the proper
place for something like this?

-HKS

On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com> wrote:
> Roger that Rainer.
>
> Thanks,
> Julian
>
> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> OK, that greatly simplifies things. Actually, it now boils down to
>> "execute an action only on the n-the time the filter evaluates to true".
>> I think this is quite easy to implement, but I must verify that...
>>
>> Rainer


From rgerhards at hq.adiscon.com  Fri Aug  1 17:02:32 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 17:02:32 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>

That's not a silly one ;) And it is coming up every now and then. The
last time it came up, I was smart enough to write a blogpost:
http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-plugins.ht
ml

In short, and to this questions: there are different schools of thought.
If you think about a plain ole syslogd shuffling data to disk files, you
do not need that. My vision of the syslogd (actually the "event logging
and alerting") subsystem is much broader. IMHO, it should support
anything that is needed to gather, process and persistently store
events. Also note that I say "events" for a reason - syslog messages are
just a subset of the potential set of events.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Friday, August 01, 2008 4:25 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> This may be a silly question, but is the syslog daemon the proper
> place for something like this?
> 
> -HKS
> 
> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
> wrote:
> > Roger that Rainer.
> >
> > Thanks,
> > Julian
> >
> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> OK, that greatly simplifies things. Actually, it now boils down to
> >> "execute an action only on the n-the time the filter evaluates to
> true".
> >> I think this is quite easy to implement, but I must verify that...
> >>
> >> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Fri Aug  1 17:17:03 2008
From: hks.private at gmail.com ((private) HKS)
Date: Fri, 1 Aug 2008 11:17:03 -0400
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>

Thanks for the link, that post makes a lot of sense. I interpreted
this discussion as moving towards adding alert functionality to the
rsyslogd core - but your stance of keeping the rsyslogd core lean and
efficient while plugins provide a full suite of event handling
processes is reassuring.

I'm looking forward to seeing where you take this plugin architecture.

-HKS

On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> That's not a silly one ;) And it is coming up every now and then. The
> last time it came up, I was smart enough to write a blogpost:
> http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-plugins.ht
> ml
>
> In short, and to this questions: there are different schools of thought.
> If you think about a plain ole syslogd shuffling data to disk files, you
> do not need that. My vision of the syslogd (actually the "event logging
> and alerting") subsystem is much broader. IMHO, it should support
> anything that is needed to gather, process and persistently store
> events. Also note that I say "events" for a reason - syslog messages are
> just a subset of the potential set of events.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of (private) HKS
>> Sent: Friday, August 01, 2008 4:25 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> This may be a silly question, but is the syslog daemon the proper
>> place for something like this?
>>
>> -HKS
>>
>> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
>> wrote:
>> > Roger that Rainer.
>> >
>> > Thanks,
>> > Julian
>> >
>> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> >> OK, that greatly simplifies things. Actually, it now boils down to
>> >> "execute an action only on the n-the time the filter evaluates to
>> true".
>> >> I think this is quite easy to implement, but I must verify that...
>> >>
>> >> Rainer
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From rgerhards at hq.adiscon.com  Fri Aug  1 17:23:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 17:23:35 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
	<ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>

Actually, in this case the (limited) functionality will go into the
core. But that's not because we need it for alerting but it is something
that was scheduled for the core engine at all (because there are other
use cases besides alerting where you need it, e.g. start some corrective
action only after the n-th error indication). Plus, it will be very
limited code. There are some other things, namely the enhanced rate
limiter, which will go to the core. I have some ways to do that via
plug-ins too, but given the expected complexity of this functionality vs
the expected complexity of pluginizing them, the choice to go to core is
really obvious. Another point where one could debate is enhanced
queuing. This, in the long term, is a candidate for being moved to a
plugin because many installations do never use disk-based (or assisted)
queues.

Now comes the important difference: if a generic, any message let me
know when it happens n times in the row filter would have been needed,
that would NOT go into the core. Because it does not belong there. It is
quite complex and even performance intense. For that, future version
will have customer (RainerScript) functions which can be provided by
library plugins. I have plans to implement such a beast (much later),
but it will come as a function that you provide the message to and that
is only loaded on an as-needed basis.

I hope this clarifies.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Friday, August 01, 2008 5:17 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Thanks for the link, that post makes a lot of sense. I interpreted
> this discussion as moving towards adding alert functionality to the
> rsyslogd core - but your stance of keeping the rsyslogd core lean and
> efficient while plugins provide a full suite of event handling
> processes is reassuring.
> 
> I'm looking forward to seeing where you take this plugin architecture.
> 
> -HKS
> 
> On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > That's not a silly one ;) And it is coming up every now and then.
The
> > last time it came up, I was smart enough to write a blogpost:
> > http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-
> plugins.ht
> > ml
> >
> > In short, and to this questions: there are different schools of
> thought.
> > If you think about a plain ole syslogd shuffling data to disk files,
> you
> > do not need that. My vision of the syslogd (actually the "event
> logging
> > and alerting") subsystem is much broader. IMHO, it should support
> > anything that is needed to gather, process and persistently store
> > events. Also note that I say "events" for a reason - syslog messages
> are
> > just a subset of the potential set of events.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> >> Sent: Friday, August 01, 2008 4:25 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> >>
> >> This may be a silly question, but is the syslog daemon the proper
> >> place for something like this?
> >>
> >> -HKS
> >>
> >> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
> >> wrote:
> >> > Roger that Rainer.
> >> >
> >> > Thanks,
> >> > Julian
> >> >
> >> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >> > <rgerhards at hq.adiscon.com> wrote:
> >> >> OK, that greatly simplifies things. Actually, it now boils down
> to
> >> >> "execute an action only on the n-the time the filter evaluates
to
> >> true".
> >> >> I think this is quite easy to implement, but I must verify
> that...
> >> >>
> >> >> Rainer
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From lovely at crm114.net  Mon Aug  4 03:25:28 2008
From: lovely at crm114.net (Julian Tyler)
Date: Mon, 4 Aug 2008 11:25:28 +1000
Subject: [rsyslog] Denial of Service
Message-ID: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>


I have a client/server setup of rsyslog, however if the server becomes  
unreachable
and the client's memory buffers fill up, everything that needs to log  
just hangs/blocks,
such as ssh.

I have enabled disk queueing however it never seems to queue anything  
to disk.

What am i doing wrong ?

---
----> Rsyslog version
---
[app-00-0001 ~]# rsyslogd -v
rsyslogd 3.16.1, compiled with:
	FEATURE_REGEXP:				Yes
	FEATURE_LARGEFILE:			Yes
	FEATURE_NETZIP (message compression):	Yes
	GSSAPI Kerberos 5 support:		No
	FEATURE_DEBUG (debug build, slow code):	No
	Runtime Instrumentation (slow code):	No

---
-----> Here is my rsyslog.conf from the offending box
---
[app-00-0001 ~]# cat /etc/rsyslog.conf
# rsyslog v3: load input modules
# If you do not load inputs, nothing happens!
# You may need to set the module load path if modules are not found.

# Debug Template
$template MyDebug,"Debug line with all properties:\nFROMHOST:  
'%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%',  
PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%',  
APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%', 
\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%', 
\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n"

$template Sys23Forward, "<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME 
% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%"


$ModLoad immark.so # provides --MARK-- message capability
$ModLoad imuxsock.so # provides support for local system logging (e.g.  
via logger command)
$ModLoad imklog.so # kernel logging (formerly provided by rklogd)
$ModLoad lmregexp.so

*.*							/var/log/all

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                -/var/log/ 
messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/ 
maillog


# Log cron stuff
cron.*                                                  -/var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          -/var/log/ 
spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/ 
boot.log

# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/spool/rsyslog/ # where to place spool files
$ActionQueueType LinkedList   # run asynchronously
$ActionQueueFileName remotelog # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as  
possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@(z9)10.238.0.1;RSYSLOG_SyslogProtocol23Format
*.* @@10.238.0.1;Sys23Forward


# ######### Receiving Messages from Remote Hosts ##########
# TCP Syslog Server:
# provides TCP syslog reception and GSS-API (if compiled to support it)
#$ModLoad imtcp.so  # load module
# Note: as of now, you need to use the -t command line option to
# enable TCP reception (e.g. -t514 to run a server at port 514/tcp)
# This will change in later v3 releases.

# UDP Syslog Server:
$ModLoad imudp.so  # provides UDP syslog reception
#$UDPServerRun 514 # start a UDP syslog server at standard port 514
$UDPServerRun 513 # start a UDP syslog server at standard port 514
#$UDPServerAddress



From rgerhards at hq.adiscon.com  Mon Aug  4 11:07:17 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 4 Aug 2008 11:07:17 +0200
Subject: [rsyslog] Denial of Service
In-Reply-To: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>
References: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF16@grfint2.intern.adiscon.com>

I think you are affected by this bug:

http://bugzilla.adiscon.com/show_bug.cgi?id=86

You need to upgrade to the recent v3-stable version.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Julian Tyler
> Sent: Monday, August 04, 2008 3:25 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Denial of Service
> 
> 
> I have a client/server setup of rsyslog, however if the server becomes
> unreachable
> and the client's memory buffers fill up, everything that needs to log
> just hangs/blocks,
> such as ssh.
> 
> I have enabled disk queueing however it never seems to queue anything
> to disk.
> 
> What am i doing wrong ?
> 
> ---
> ----> Rsyslog version
> ---
> [app-00-0001 ~]# rsyslogd -v
> rsyslogd 3.16.1, compiled with:
> 	FEATURE_REGEXP:				Yes
> 	FEATURE_LARGEFILE:			Yes
> 	FEATURE_NETZIP (message compression):	Yes
> 	GSSAPI Kerberos 5 support:		No
> 	FEATURE_DEBUG (debug build, slow code):	No
> 	Runtime Instrumentation (slow code):	No
> 
> ---
> -----> Here is my rsyslog.conf from the offending box
> ---
> [app-00-0001 ~]# cat /etc/rsyslog.conf
> # rsyslog v3: load input modules
> # If you do not load inputs, nothing happens!
> # You may need to set the module load path if modules are not found.
> 
> # Debug Template
> $template MyDebug,"Debug line with all properties:\nFROMHOST:
> '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%',
> PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%',
> APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',
> \nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',
> \nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg:
'%rawmsg%'\n\n"
> 
> $template Sys23Forward, "<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME
> % %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%"
> 
> 
> $ModLoad immark.so # provides --MARK-- message capability
> $ModLoad imuxsock.so # provides support for local system logging (e.g.
> via logger command)
> $ModLoad imklog.so # kernel logging (formerly provided by rklogd)
> $ModLoad lmregexp.so
> 
> *.*							/var/log/all
> 
> # Log all kernel messages to the console.
> # Logging much else clutters up the screen.
> #kern.*                                                 /dev/console
> 
> # Log anything (except mail) of level info or higher.
> # Don't log private authentication messages!
> *.info;mail.none;authpriv.none;cron.none                -/var/log/
> messages
> 
> # The authpriv file has restricted access.
> authpriv.*
/var/log/secure
> 
> # Log all the mail messages in one place.
> mail.*                                                  -/var/log/
> maillog
> 
> 
> # Log cron stuff
> cron.*                                                  -/var/log/cron
> 
> # Everybody gets emergency messages
> *.emerg                                                 *
> 
> # Save news errors of level crit and higher in a special file.
> uucp,news.crit                                          -/var/log/
> spooler
> 
> # Save boot messages also to boot.log
> local7.*                                                /var/log/
> boot.log
> 
> # Remote Logging (we use TCP for reliable delivery)
> # An on-disk queue is created for this action. If the remote host is
> # down, messages are spooled to disk and sent when it is up again.
> $WorkDirectory /var/spool/rsyslog/ # where to place spool files
> $ActionQueueType LinkedList   # run asynchronously
> $ActionQueueFileName remotelog # unique name prefix for spool files
> $ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as
> possible)
> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
> $ActionResumeRetryCount -1    # infinite retries if host is down
> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
> #*.* @@(z9)10.238.0.1;RSYSLOG_SyslogProtocol23Format
> *.* @@10.238.0.1;Sys23Forward
> 
> 
> # ######### Receiving Messages from Remote Hosts ##########
> # TCP Syslog Server:
> # provides TCP syslog reception and GSS-API (if compiled to support
it)
> #$ModLoad imtcp.so  # load module
> # Note: as of now, you need to use the -t command line option to
> # enable TCP reception (e.g. -t514 to run a server at port 514/tcp)
> # This will change in later v3 releases.
> 
> # UDP Syslog Server:
> $ModLoad imudp.so  # provides UDP syslog reception
> #$UDPServerRun 514 # start a UDP syslog server at standard port 514
> $UDPServerRun 513 # start a UDP syslog server at standard port 514
> #$UDPServerAddress
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From friedl at hq.adiscon.com  Mon Aug  4 17:40:28 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 4 Aug 2008 17:40:28 +0200
Subject: [rsyslog] rsyslog 3.21.2 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF2D@grfint2.intern.adiscon.com>

Hi all,

rsyslog 3.21.2, a member of the devel branch, has been released today.
It offers two improvements: Imuxsock now permits to specify an
alternative hostname to be used with messages originating from that
socket. This is useful for logging jail processes. Ommail now offers the
ability to specify multiple email recipients. Also, the troubleshooting
documentation has been enhanced and now contains information on debug
mode. There are no bug fixes in this release. This is a recommended
update for all devel branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-124.phtml

Changelog:

http://www.rsyslog.com/Article264.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .




From alorbach at ro1.adiscon.com  Tue Aug  5 16:25:04 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 5 Aug 2008 16:25:04 +0200
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>

Hi again, 

I want to inform you all that the search within phpLogCon has been
extended to support full and partial searches in Version 2.5.3,
available for download here: http://www.phplogcon.org/downloads

I first tests have shown, that filtering by hostname and other string
based fields performs much better on database sources now. 
You can use the submenu buttons on each field within the messages view
to perform such a search. If you want to do manual searches, it is very
simple. Here is a sample: 

Search for a full hostname: source:=SERVERNAME (Will only find if the
match is 100% = SERVERNAME)
Search for a partial hostname source:SERVER (Willmatch if the hostname
is SERVER or SERVERNAME).  

I hope this helps, 
Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Thursday, July 31, 2008 10:15 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> Hi,
> 
> the like query can indeed have quiet an impact on performance when
doing
> queries on large databases.
> But I think we can expand the syntax, so you can either search by part
> of a string (LIKE '%search%') or the whole string (= 'search'). This
> should be rather easy to implement. I will put this on my todolist, if
> it is as easy as I think, the next minor update of the devel branch
will
> contain this new feature.
> 
> Best regards,
> Andre Lorbach
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > Sent: Thursday, July 31, 2008 4:10 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] tips for managing data
> >
> > OK, so it seems that doing a query from the query line does a LIKE,
> > which can take significantly longer (sample query 8 seconds vs. 50
> msecs...)
> >
> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
> things
> > up quite a but, but I lose some flexibility. Is there some kind of
> > search syntax where I can differentiate between LIKE and =?
> >
> > If not, I'm thinking something like:
> >
> > source:foo.bar.com   # would be using =
> >
> > ~source:foo   # would be using LIKE
> >
> >
> >
> > Rory Toma wrote:
> > > So, my current mysql rsyslog drops about 20 million rows of data
per
> day.
> > >
> > > Over time, this gets slow as tables grow.
> > >
> > > I'm not a dba, so I was wondering if anyone had some suggestions
for
> > > keeping performance still on the order of seconds, and not minutes
> or hours.
> > >
> > > thx
> > >
> > > I did add a key for EventSource, as that is commonly searched.
> However,
> > > using PhpLogCon, it seems that if I search using the web interface
> (i.e.
> > > I click on a host entry and hit the available searches) it is
> relatively
> > > quick. However, changing the text field that is generated and
> hitting
> > > the "search" button is slow. Do these two methods use the same
> query, or
> > > is something else going on?
> > >
> > > thx
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Tue Aug  5 17:09:22 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 5 Aug 2008 11:09:22 -0400
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
Message-ID: <ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>

Will searching for NAME match SERVERNAME?

-HKS

On Tue, Aug 5, 2008 at 10:25 AM, Andre Lorbach <alorbach at ro1.adiscon.com> wrote:
> Hi again,
>
> I want to inform you all that the search within phpLogCon has been
> extended to support full and partial searches in Version 2.5.3,
> available for download here: http://www.phplogcon.org/downloads
>
> I first tests have shown, that filtering by hostname and other string
> based fields performs much better on database sources now.
> You can use the submenu buttons on each field within the messages view
> to perform such a search. If you want to do manual searches, it is very
> simple. Here is a sample:
>
> Search for a full hostname: source:=SERVERNAME (Will only find if the
> match is 100% = SERVERNAME)
> Search for a partial hostname source:SERVER (Willmatch if the hostname
> is SERVER or SERVERNAME).
>
> I hope this helps,
> Best regards,
> Andre Lorbach
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>> Sent: Thursday, July 31, 2008 10:15 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] tips for managing data
>>
>> Hi,
>>
>> the like query can indeed have quiet an impact on performance when
> doing
>> queries on large databases.
>> But I think we can expand the syntax, so you can either search by part
>> of a string (LIKE '%search%') or the whole string (= 'search'). This
>> should be rather easy to implement. I will put this on my todolist, if
>> it is as easy as I think, the next minor update of the devel branch
> will
>> contain this new feature.
>>
>> Best regards,
>> Andre Lorbach
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> > Sent: Thursday, July 31, 2008 4:10 AM
>> > To: rsyslog-users
>> > Subject: Re: [rsyslog] tips for managing data
>> >
>> > OK, so it seems that doing a query from the query line does a LIKE,
>> > which can take significantly longer (sample query 8 seconds vs. 50
>> msecs...)
>> >
>> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
>> things
>> > up quite a but, but I lose some flexibility. Is there some kind of
>> > search syntax where I can differentiate between LIKE and =?
>> >
>> > If not, I'm thinking something like:
>> >
>> > source:foo.bar.com   # would be using =
>> >
>> > ~source:foo   # would be using LIKE
>> >
>> >
>> >
>> > Rory Toma wrote:
>> > > So, my current mysql rsyslog drops about 20 million rows of data
> per
>> day.
>> > >
>> > > Over time, this gets slow as tables grow.
>> > >
>> > > I'm not a dba, so I was wondering if anyone had some suggestions
> for
>> > > keeping performance still on the order of seconds, and not minutes
>> or hours.
>> > >
>> > > thx
>> > >
>> > > I did add a key for EventSource, as that is commonly searched.
>> However,
>> > > using PhpLogCon, it seems that if I search using the web interface
>> (i.e.
>> > > I click on a host entry and hit the available searches) it is
>> relatively
>> > > quick. However, changing the text field that is generated and
>> hitting
>> > > the "search" button is slow. Do these two methods use the same
>> query, or
>> > > is something else going on?
>> > >
>> > > thx
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > >
>> >
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From alorbach at ro1.adiscon.com  Tue Aug  5 17:31:24 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 5 Aug 2008 17:31:24 +0200
Subject: [rsyslog] tips for managing data
In-Reply-To: <ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>
References: <4891042D.9040805@ooma.com>
	<48911EF0.1010004@ooma.com><F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
	<ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D71@rmail.adiscon.com>

Yes if you use a partial search, it will match as well. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Tuesday, August 05, 2008 5:09 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> Will searching for NAME match SERVERNAME?
> 
> -HKS
> 
> On Tue, Aug 5, 2008 at 10:25 AM, Andre Lorbach
<alorbach at ro1.adiscon.com>
> wrote:
> > Hi again,
> >
> > I want to inform you all that the search within phpLogCon has been
> > extended to support full and partial searches in Version 2.5.3,
> > available for download here: http://www.phplogcon.org/downloads
> >
> > I first tests have shown, that filtering by hostname and other
string
> > based fields performs much better on database sources now.
> > You can use the submenu buttons on each field within the messages
view
> > to perform such a search. If you want to do manual searches, it is
very
> > simple. Here is a sample:
> >
> > Search for a full hostname: source:=SERVERNAME (Will only find if
the
> > match is 100% = SERVERNAME)
> > Search for a partial hostname source:SERVER (Willmatch if the
hostname
> > is SERVER or SERVERNAME).
> >
> > I hope this helps,
> > Best regards,
> > Andre Lorbach
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> >> Sent: Thursday, July 31, 2008 10:15 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] tips for managing data
> >>
> >> Hi,
> >>
> >> the like query can indeed have quiet an impact on performance when
> > doing
> >> queries on large databases.
> >> But I think we can expand the syntax, so you can either search by
part
> >> of a string (LIKE '%search%') or the whole string (= 'search').
This
> >> should be rather easy to implement. I will put this on my todolist,
if
> >> it is as easy as I think, the next minor update of the devel branch
> > will
> >> contain this new feature.
> >>
> >> Best regards,
> >> Andre Lorbach
> >>
> >> > -----Original Message-----
> >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> >> > Sent: Thursday, July 31, 2008 4:10 AM
> >> > To: rsyslog-users
> >> > Subject: Re: [rsyslog] tips for managing data
> >> >
> >> > OK, so it seems that doing a query from the query line does a
LIKE,
> >> > which can take significantly longer (sample query 8 seconds vs.
50
> >> msecs...)
> >> >
> >> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
> >> things
> >> > up quite a but, but I lose some flexibility. Is there some kind
of
> >> > search syntax where I can differentiate between LIKE and =?
> >> >
> >> > If not, I'm thinking something like:
> >> >
> >> > source:foo.bar.com   # would be using =
> >> >
> >> > ~source:foo   # would be using LIKE
> >> >
> >> >
> >> >
> >> > Rory Toma wrote:
> >> > > So, my current mysql rsyslog drops about 20 million rows of
data
> > per
> >> day.
> >> > >
> >> > > Over time, this gets slow as tables grow.
> >> > >
> >> > > I'm not a dba, so I was wondering if anyone had some
suggestions
> > for
> >> > > keeping performance still on the order of seconds, and not
minutes
> >> or hours.
> >> > >
> >> > > thx
> >> > >
> >> > > I did add a key for EventSource, as that is commonly searched.
> >> However,
> >> > > using PhpLogCon, it seems that if I search using the web
interface
> >> (i.e.
> >> > > I click on a host entry and hit the available searches) it is
> >> relatively
> >> > > quick. However, changing the text field that is generated and
> >> hitting
> >> > > the "search" button is slow. Do these two methods use the same
> >> query, or
> >> > > is something else going on?
> >> > >
> >> > > thx
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > >
> >> >
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Tue Aug  5 20:45:31 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 11:45:31 -0700
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com>
	<48911EF0.1010004@ooma.com>	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
Message-ID: <48989FCB.9030609@ooma.com>

Andre Lorbach wrote:
> Hi again, 
>
> I want to inform you all that the search within phpLogCon has been
> extended to support full and partial searches in Version 2.5.3,
> available for download here: http://www.phplogcon.org/downloads
>
> I first tests have shown, that filtering by hostname and other string
> based fields performs much better on database sources now. 
> You can use the submenu buttons on each field within the messages view
> to perform such a search. If you want to do manual searches, it is very
> simple. Here is a sample: 
>
> Search for a full hostname: source:=SERVERNAME (Will only find if the
> match is 100% = SERVERNAME)
> Search for a partial hostname source:SERVER (Willmatch if the hostname
> is SERVER or SERVERNAME).  
>   
Thanks for the quick change.

Now I have another question, which is more sql but I thought I'd ask it 
anyway.

What I'd like to do is keep 3 months worth of data, but rotate and merge 
my tables after a set amount of time to keep search times low. Tables 
more than 3 months old would be dropped.

If anyone has a nice, canned script for this, it would be appreciated. 
Otherwise, I'll read the mysql manuals. 8-)

thx


From rory at ooma.com  Tue Aug  5 23:29:40 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:29:40 -0700
Subject: [rsyslog] phplogcon UI question
Message-ID: <4898C644.5060906@ooma.com>

Is there a way to turn off the hover-exploding info windows when you 
move the mouse over the "message" field?

On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
away.

thx


From rgerhards at hq.adiscon.com  Tue Aug  5 23:31:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 5 Aug 2008 23:31:56 +0200
Subject: [rsyslog] phplogcon UI question
Message-ID: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>

There is a config setting. But I thought it would be off by default?

rainer

----- Urspr?ngliche Nachricht -----
Von: "Rory Toma" <rory at ooma.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 05.08.08 23:29
Betreff: [rsyslog] phplogcon UI question

Is there a way to turn off the hover-exploding info windows when you 
move the mouse over the "message" field?

On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
away.

thx
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Tue Aug  5 23:35:42 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:35:42 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <4898C7AE.7030800@ooma.com>

Rory Toma wrote:
> Is there a way to turn off the hover-exploding info windows when you 
> move the mouse over the "message" field?
>
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
> away.
>
> thx
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   
Firefox 3 does not have this issue.

Neither does IE7.


From rory at ooma.com  Tue Aug  5 23:38:24 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:38:24 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
References: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
Message-ID: <4898C850.1000508@ooma.com>

Rainer Gerhards wrote:
> There is a config setting. But I thought it would be off by default?
>
> rainer
>
>   


ViewEnableDetailPopups

thx


From rory at ooma.com  Wed Aug  6 03:07:46 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 18:07:46 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if query returns no
	results
Message-ID: <4898F962.1030200@ooma.com>

Minor issue, but thought I'd report it.

If i do a query that returns no results, I get a bunch of empty rows 
displayed in phpLogCon.




From rgerhards at hq.adiscon.com  Wed Aug  6 07:43:06 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 07:43:06 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if query
	returns noresults
In-Reply-To: <4898F962.1030200@ooma.com>
References: <4898F962.1030200@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>

Which version are you using? There was an issue with postgres, but that
has been fixed in recent (devel) builds.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 3:08 AM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon displays lots of empty rows if query
> returns noresults
> 
> Minor issue, but thought I'd report it.
> 
> If i do a query that returns no results, I get a bunch of empty rows
> displayed in phpLogCon.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Wed Aug  6 07:47:14 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 22:47:14 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if
 query	returns noresults
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
References: <4898F962.1030200@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
Message-ID: <48993AE2.7050607@ooma.com>

I'm running 2.5.3.


Rainer Gerhards wrote:
> Which version are you using? There was an issue with postgres, but that
> has been fixed in recent (devel) builds.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> Sent: Wednesday, August 06, 2008 3:08 AM
>> To: rsyslog-users
>> Subject: [rsyslog] phplogcon displays lots of empty rows if query
>> returns noresults
>>
>> Minor issue, but thought I'd report it.
>>
>> If i do a query that returns no results, I get a bunch of empty rows
>> displayed in phpLogCon.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   



From rgerhards at hq.adiscon.com  Wed Aug  6 08:01:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 08:01:19 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if
	query	returns noresults
In-Reply-To: <48993AE2.7050607@ooma.com>
References: <4898F962.1030200@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
	<48993AE2.7050607@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>

Can you provide the config of the data source so that Andre can create a
testbed?

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 7:47 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon displays lots of empty rows if query
> returns noresults
> 
> I'm running 2.5.3.
> 
> 
> Rainer Gerhards wrote:
> > Which version are you using? There was an issue with postgres, but
> that
> > has been fixed in recent (devel) builds.
> >
> > Rainer
> >
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> >> Sent: Wednesday, August 06, 2008 3:08 AM
> >> To: rsyslog-users
> >> Subject: [rsyslog] phplogcon displays lots of empty rows if query
> >> returns noresults
> >>
> >> Minor issue, but thought I'd report it.
> >>
> >> If i do a query that returns no results, I get a bunch of empty
rows
> >> displayed in phpLogCon.
> >>
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Wed Aug  6 10:20:34 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 6 Aug 2008 10:20:34 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C7AE.7030800@ooma.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:36 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> Rory Toma wrote:
> > Is there a way to turn off the hover-exploding info windows when you
> > move the mouse over the "message" field?
> >
> > On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> > away.
> >
> > thx
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> Firefox 3 does not have this issue.
> 
> Neither does IE7.

I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
unfortanetly don't have a MacOS machine to test Safari Browser. 

Best regards,
Andre Lorbach


From alorbach at ro1.adiscon.com  Wed Aug  6 10:21:32 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 6 Aug 2008 10:21:32 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C850.1000508@ooma.com>
References: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
	<4898C850.1000508@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D76@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:38 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> Rainer Gerhards wrote:
> > There is a config setting. But I thought it would be off by default?
> >
> > rainer
> >
> >
> 
> ViewEnableDetailPopups

If you install phpLogCon with UserDB System, you can turn this feature
off and on within the Admin Center ;).

Best regards,
Andre Lorbach


From rory at ooma.com  Wed Aug  6 22:58:41 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 13:58:41 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
Message-ID: <489A1081.1090609@ooma.com>


> I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
> unfortanetly don't have a MacOS machine to test Safari Browser. 
>
>   
What about the Windows version of Safari?

http://www.apple.com/downloads/macosx/apple/windows/


From rory at ooma.com  Wed Aug  6 22:59:23 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 13:59:23 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows
 if	query	returns noresults
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>
References: <4898F962.1030200@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>	<48993AE2.7050607@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>
Message-ID: <489A10AB.30101@ooma.com>

Rainer Gerhards wrote:
> Can you provide the config of the data source so that Andre can create a
> testbed?
>
> Thanks,
> Rainer
>
>   
>
Exactly what info would you like? I am using the default syslog schema 
in SystemEvents.




From rgerhards at hq.adiscon.com  Wed Aug  6 23:04:59 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 23:04:59 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if	queryreturns
	noresults
Message-ID: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>

The definition of the data source in question - should be in xour config file...

----- Urspr?ngliche Nachricht -----
Von: "Rory Toma" <rory at ooma.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 06.08.08 22:59
Betreff: Re: [rsyslog] phplogcon displays lots of empty rows if	query	returns noresults

Rainer Gerhards wrote:
> Can you provide the config of the data source so that Andre can create a
> testbed?
>
> Thanks,
> Rainer
>
>   
>
Exactly what info would you like? I am using the default syslog schema 
in SystemEvents.


_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Wed Aug  6 23:09:40 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 14:09:40 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if	queryreturns
 noresults
In-Reply-To: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>
References: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>
Message-ID: <489A1314.3040909@ooma.com>

Rainer Gerhards wrote:
> The definition of the data source in question - should be in xour config file...
>
>
>   

$CFG['DefaultSourceID'] = 'Source1';

$CFG['Sources']['Source1']['ID'] = 'Source1';
$CFG['Sources']['Source1']['Name'] = 'rsyslog';
$CFG['Sources']['Source1']['ViewID'] = 'SYSLOG';
$CFG['Sources']['Source1']['SourceType'] = SOURCE_DB;
$CFG['Sources']['Source1']['DBTableType'] = 'monitorware';
$CFG['Sources']['Source1']['DBServer'] = 'rsyslog.corp.ooma.com';
$CFG['Sources']['Source1']['DBName'] = 'Syslog';
$CFG['Sources']['Source1']['DBUser'] = 'xxx';
$CFG['Sources']['Source1']['DBPassword'] = 'xxx';
$CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents';
$CFG['Sources']['Source1']['DBEnableRowCounting'] = false;




From rory at ooma.com  Thu Aug  7 01:53:49 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 16:53:49 -0700
Subject: [rsyslog] ok, here's my sql rotation...
Message-ID: <489A398D.6010402@ooma.com>

OK, so here's what I ended up doing...

I created SystemEvents1 thru SystemEvents90, and create SystemEvents as 
a union of 90 thru 1. (1 being last on the union line so it's the 
"active" table)

In cron.daily, I run

mysql <blah blah blah> < script.mysql

where script.mysql is

use Syslog;

CREATE TABLE `SystemEvents0` ( <table def>) Engine=MyISAM;

RENAME TABLE SystemEvents90 to SystemEvents91,
SystemEvents89 to SystemEvents 90,
SystemEvents88 to SystemEvents 89,
........
SystemEvents1 to SystemEvents2,
SystemEvents0 to SystemEvents1;

DROP TABLE SystemEvents91;

This gives me 90 days of logs at a reasonable performance level.

I'm kinda new to MySQL, so I took the route of ease of implementation.


From rory at ooma.com  Thu Aug  7 02:12:13 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 17:12:13 -0700
Subject: [rsyslog] see sql queries on phplogcon
Message-ID: <489A3DDD.4000205@ooma.com>

How do I see what queries are being done in phplogcon? I don't have sql 
logging turned on in my db.

the := syntax seems to be no longer working for me. I get an error 
telling me there were no syslog records found when I try it.


I've set up my SystemEvents table as a union and merged all the tables 
in with data. This shouldn't affect it, I would think. My sql queries by 
hand seem to work fine.




From mic at npgx.com.au  Thu Aug  7 04:23:29 2008
From: mic at npgx.com.au (Michael Mansour)
Date: Thu, 7 Aug 2008 13:23:29 +1100
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <489A1081.1090609@ooma.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
	<489A1081.1090609@ooma.com>
Message-ID: <20080807022212.M77981@npgx.com.au>

Hi guys,

> > I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
> > unfortanetly don't have a MacOS machine to test Safari Browser. 
> >
> >   
> What about the Windows version of Safari?
> 
> http://www.apple.com/downloads/macosx/apple/windows/

There are websites out there which will freely test your (public) web page on
a number of browsers of your choice, sending you the results after a period of
time.

A web search will be able to find these sites.

Michael.


From rgerhards at hq.adiscon.com  Thu Aug  7 08:06:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 08:06:35 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <489A3DDD.4000205@ooma.com>
References: <489A3DDD.4000205@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>

Andre,

I think this triggers a new feature request: I know you have this in
debugging (at least in times), but could you (easily enough) add an
option that displays the SQL statement, e.g. at the top of the page?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 07, 2008 2:12 AM
> To: rsyslog-users
> Subject: [rsyslog] see sql queries on phplogcon
> 
> How do I see what queries are being done in phplogcon? I don't have
sql
> logging turned on in my db.
> 
> the := syntax seems to be no longer working for me. I get an error
> telling me there were no syslog records found when I try it.
> 
> 
> I've set up my SystemEvents table as a union and merged all the tables
> in with data. This shouldn't affect it, I would think. My sql queries
> by
> hand seem to work fine.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:08:49 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:08:49 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <489A1081.1090609@ooma.com>
References: <4898C644.5060906@ooma.com>
	<4898C7AE.7030800@ooma.com><F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
	<489A1081.1090609@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7B@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 10:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> 
> > I usually test phpLogCon with the Browsers Opera, Firefox and IE7.
But I
> > unfortanetly don't have a MacOS machine to test Safari Browser.
> >
> >
> What about the Windows version of Safari?
> 
> http://www.apple.com/downloads/macosx/apple/windows/

Oh thanks I wasn't aware Safari was available for Windows ;). I will
download and install this browser and see what I can do. 

Best regards,
Andre Lorbach


From alorbach at ro1.adiscon.com  Thu Aug  7 10:11:07 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:11:07 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
References: <489A3DDD.4000205@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>

I agree, I will add some debug output so you can see the SQL queries
which are performed in background.
Currently I could only tell where to uncomment some debug code in the
logstream classes, but I recommend to wait until I added this feature. 

Best regards,
Andre Lorbach


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 8:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] see sql queries on phplogcon
> 
> Andre,
> 
> I think this triggers a new feature request: I know you have this in
> debugging (at least in times), but could you (easily enough) add an
> option that displays the SQL statement, e.g. at the top of the page?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > Sent: Thursday, August 07, 2008 2:12 AM
> > To: rsyslog-users
> > Subject: [rsyslog] see sql queries on phplogcon
> >
> > How do I see what queries are being done in phplogcon? I don't have
> sql
> > logging turned on in my db.
> >
> > the := syntax seems to be no longer working for me. I get an error
> > telling me there were no syslog records found when I try it.
> >
> >
> > I've set up my SystemEvents table as a union and merged all the
tables
> > in with data. This shouldn't affect it, I would think. My sql
queries
> > by
> > hand seem to work fine.
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:14:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:14:25 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <489A3DDD.4000205@ooma.com>
References: <489A3DDD.4000205@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>

One question, are you searching for a single word or a phrase? 
If you are searching for a phrase, please use + instead of spaces. 
For example, you want to find a syslogtag like "mysql safe", then you
use this search: 
source:=mysql+safe

If you do not use the + instead of the space, the filter will be
splitted at the word safe, which will be considered as message filter
then. 

Maybe this helps - best regards,
Andre Lorbach



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 07, 2008 2:12 AM
> To: rsyslog-users
> Subject: [rsyslog] see sql queries on phplogcon
> 
> How do I see what queries are being done in phplogcon? I don't have
sql
> logging turned on in my db.
> 
> the := syntax seems to be no longer working for me. I get an error
> telling me there were no syslog records found when I try it.
> 
> 
> I've set up my SystemEvents table as a union and merged all the tables
> in with data. This shouldn't affect it, I would think. My sql queries
by
> hand seem to work fine.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 10:26:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 10:26:44 +0200
Subject: [rsyslog] debug code in phpLogCon - was: see sql queries on
	phplogcon
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
References: <489A3DDD.4000205@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>

In general, let me recommend something that has been proven quite
valuable in rsyslog: Many debug facilities in rsyslog are not commented
out or removed, but can be activated during runtime by switch(es). This
is quite useful when it comes down to troubleshooting issues with a
user's installation. All that is needed is to activate a debug setting
and you see, in all its glory ;), what is going on in that installation.
It may even be worth a thought if phpLogCon should support creating
debug files, which can be sent to you for hard to find cases. Not sure
if that makes sense for phpLogCon, though...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Thursday, August 07, 2008 10:11 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] see sql queries on phplogcon
> 
> I agree, I will add some debug output so you can see the SQL queries
> which are performed in background.
> Currently I could only tell where to uncomment some debug code in the
> logstream classes, but I recommend to wait until I added this feature.
> 
> Best regards,
> Andre Lorbach
> 
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, August 07, 2008 8:07 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] see sql queries on phplogcon
> >
> > Andre,
> >
> > I think this triggers a new feature request: I know you have this in
> > debugging (at least in times), but could you (easily enough) add an
> > option that displays the SQL statement, e.g. at the top of the page?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > > Sent: Thursday, August 07, 2008 2:12 AM
> > > To: rsyslog-users
> > > Subject: [rsyslog] see sql queries on phplogcon
> > >
> > > How do I see what queries are being done in phplogcon? I don't
have
> > sql
> > > logging turned on in my db.
> > >
> > > the := syntax seems to be no longer working for me. I get an error
> > > telling me there were no syslog records found when I try it.
> > >
> > >
> > > I've set up my SystemEvents table as a union and merged all the
> tables
> > > in with data. This shouldn't affect it, I would think. My sql
> queries
> > > by
> > > hand seem to work fine.
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:32:39 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:32:39 +0200
Subject: [rsyslog] debug code in phpLogCon - was: see sql queries
	onphplogcon
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>
References: <489A3DDD.4000205@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7E@rmail.adiscon.com>

We have some kind of a debug facility already in phpLogCon, it just
needs to be enabled and extended a little bit.
I think this will be one of the next minor new features to do then ;)!

--
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 10:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] debug code in phpLogCon - was: see sql queries
> onphplogcon
> 
> In general, let me recommend something that has been proven quite
> valuable in rsyslog: Many debug facilities in rsyslog are not
commented
> out or removed, but can be activated during runtime by switch(es).
This
> is quite useful when it comes down to troubleshooting issues with a
> user's installation. All that is needed is to activate a debug setting
> and you see, in all its glory ;), what is going on in that
installation.
> It may even be worth a thought if phpLogCon should support creating
> debug files, which can be sent to you for hard to find cases. Not sure
> if that makes sense for phpLogCon, though...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > Sent: Thursday, August 07, 2008 10:11 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] see sql queries on phplogcon
> >
> > I agree, I will add some debug output so you can see the SQL queries
> > which are performed in background.
> > Currently I could only tell where to uncomment some debug code in
the
> > logstream classes, but I recommend to wait until I added this
feature.
> >
> > Best regards,
> > Andre Lorbach
> >
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > Sent: Thursday, August 07, 2008 8:07 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] see sql queries on phplogcon
> > >
> > > Andre,
> > >
> > > I think this triggers a new feature request: I know you have this
in
> > > debugging (at least in times), but could you (easily enough) add
an
> > > option that displays the SQL statement, e.g. at the top of the
page?
> > >
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > > > Sent: Thursday, August 07, 2008 2:12 AM
> > > > To: rsyslog-users
> > > > Subject: [rsyslog] see sql queries on phplogcon
> > > >
> > > > How do I see what queries are being done in phplogcon? I don't
> have
> > > sql
> > > > logging turned on in my db.
> > > >
> > > > the := syntax seems to be no longer working for me. I get an
error
> > > > telling me there were no syslog records found when I try it.
> > > >
> > > >
> > > > I've set up my SystemEvents table as a union and merged all the
> > tables
> > > > in with data. This shouldn't affect it, I would think. My sql
> > queries
> > > > by
> > > > hand seem to work fine.
> > > >
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:36:12 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:36:12 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:30 PM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon UI question
> 
> Is there a way to turn off the hover-exploding info windows when you
> move the mouse over the "message" field?
> 
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> away.

Ok I have Safari installed now, and I see what you mean. 
Actually this is some kind of a redraw issue within Safari, which does
not redraw the content which was behind the popup window. If you resize
the window after some popup messages have been viewed, you will see that
the content is redrawn. 

However I think we are going to work on this popup window anyway, as it
is not perfect yet. 

Best regards,
Andre Lorbach



From ml at darville.vm.bytemark.co.uk  Thu Aug  7 11:12:35 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Thu, 7 Aug 2008 10:12:35 +0100
Subject: [rsyslog] Changing hostname field
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
References: <20080801102649.GA27628@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
Message-ID: <20080807091235.GA16356@darville.vm.bytemark.co.uk>

On Fri, Aug 01, 2008 at 03:06:43PM +0200, Rainer Gerhards wrote:

> > > It also looks like I was wrong ;) I have reviewed the case. If I
> > > understand correctly, you get you data from the different unix
> > sockets
> > > and then forward via a single rule. So you can not use different
> > > templates (which most often are the real solution to any such
> > problem,
> > > but are not obvious to most people not directly involved in rsyslog
> > ;)).
> > >
> > > Is my understanding correct now?
> > 
> > Yes, and then I'm splitting log messages on the dedicated log host,
> > based on
> > the hostname field.
> > 
> > We are keeping the rsyslogd process outside the jails, to to keep the
> > syslog
> > traffic on an isolated network segment.
> 
> OK, that was fairly easy to do. I have created a special version for you:
> 
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz
> 
> It contains a new config directive that allows you to specify the hostname, see last sample on
> 
> http://www.rsyslog.com/doc-imuxsock.html
> 
> In my lab it works, but I would appreciate if you could test and verify.

Thank you very much, that is extremely good service - it works perfectly.

I owe you a beer for that.......


-- David Darville


From rgerhards at hq.adiscon.com  Thu Aug  7 10:42:14 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 10:42:14 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
References: <4898C644.5060906@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>

> Ok I have Safari installed now, and I see what you mean.
> Actually this is some kind of a redraw issue within Safari, which does
> not redraw the content which was behind the popup window. If you
resize
> the window after some popup messages have been viewed, you will see
> that
> the content is redrawn.
> 
> However I think we are going to work on this popup window anyway, as
it
> is not perfect yet.

Let me express my personal opinion in the hope that others will also
provide some input: I do *not* think it is well spent time to enhance
this Window. The nice thing is that it can be turned off and that should
solve most problems ;) Actually, I (again personally!) find the window
in itself quite problematic: if it is turned on, it blocks me from doing
some things in the natural way, because it always pops up and blocks
view of parts of the web page *just* because I happen to have moved the
mouse over some part of the page. For me, this leads to such funny
things that I need to concentrate on moving the mouse from bottom to top
(and not in the opposite direction ;)) when I want to activate some
context menu items - because otherwise the popup would block me from
reaching that point.

Again, this is just my personal view, but I would like to bring that
topic up to a broader audience.

Rainer


From rgerhards at hq.adiscon.com  Thu Aug  7 12:22:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 12:22:19 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <20080807091235.GA16356@darville.vm.bytemark.co.uk>
References: <20080801102649.GA27628@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
	<20080807091235.GA16356@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF67@grfint2.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Thursday, August 07, 2008 11:13 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> On Fri, Aug 01, 2008 at 03:06:43PM +0200, Rainer Gerhards wrote:
> 
> > > > It also looks like I was wrong ;) I have reviewed the case. If I
> > > > understand correctly, you get you data from the different unix
> > > sockets
> > > > and then forward via a single rule. So you can not use different
> > > > templates (which most often are the real solution to any such
> > > problem,
> > > > but are not obvious to most people not directly involved in
> rsyslog
> > > ;)).
> > > >
> > > > Is my understanding correct now?
> > >
> > > Yes, and then I'm splitting log messages on the dedicated log
host,
> > > based on
> > > the hostname field.
> > >
> > > We are keeping the rsyslogd process outside the jails, to to keep
> the
> > > syslog
> > > traffic on an isolated network segment.
> >
> > OK, that was fairly easy to do. I have created a special version for
> you:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz
> >
> > It contains a new config directive that allows you to specify the
> hostname, see last sample on
> >
> > http://www.rsyslog.com/doc-imuxsock.html
> >
> > In my lab it works, but I would appreciate if you could test and
> verify.
> 
> Thank you very much, that is extremely good service - it works
> perfectly.
> 

Excellent!

> I owe you a beer for that.......

Be careful: I extremely well remember some things ;)

Rainer
> 
> 
> -- David Darville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From ml at darville.vm.bytemark.co.uk  Thu Aug  7 14:05:52 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Thu, 7 Aug 2008 13:05:52 +0100
Subject: [rsyslog] Problems with spooling to disk
Message-ID: <20080807120552.GA16724@darville.vm.bytemark.co.uk>

Hello everybody

Now I'm trying to configure rsyslog to spool log messages to disk, while the
connection to the central log collector is down. But it does not work......

I am using rsyslog v 3.21.2

I have configured the client rsyslog like this:

$ModLoad imuxsock
$ModLoad omrelp
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName rsyslog-buffer
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
*.* :omrelp:172.16.0.43:2514

Which is similar to the reliable forwarding configuration from the
documentation, except that I prefer to use RELP.

But When I test it, by stopping rsyslog on the central log collector, and
running this small shell script on the client:

#! /bin/sh
for ((i=1;i<=10000;i+=1)); do
  logger Log line $i
done

The shell script does not complete until I restart rsyslog, on the central
log collector, and when I look at the timestamps in the logfile on the log
collector, I see a large time difference between log line 3303 and 3304.

My plan is to get rsyslog to spool all log entries locally, while the
connection the the central log collector is interrupted, and then flush all
the spooled log messages to the central log collector when the communication
is restored - while the rest of the client system continues with business as
usual.

I can send you a debug log is needed.


-- David Darville


From rgerhards at hq.adiscon.com  Thu Aug  7 14:19:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 14:19:04 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>

>From what I see, this should work. So please send the debug log, client
(more important) and server please :)

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Thursday, August 07, 2008 2:06 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Problems with spooling to disk
> 
> Hello everybody
> 
> Now I'm trying to configure rsyslog to spool log messages to disk,
> while the
> connection to the central log collector is down. But it does not
> work......
> 
> I am using rsyslog v 3.21.2
> 
> I have configured the client rsyslog like this:
> 
> $ModLoad imuxsock
> $ModLoad omrelp
> $WorkDirectory /var/spool/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueFileName rsyslog-buffer
> $ActionResumeRetryCount -1
> $ActionQueueSaveOnShutdown on
> *.* :omrelp:172.16.0.43:2514
> 
> Which is similar to the reliable forwarding configuration from the
> documentation, except that I prefer to use RELP.
> 
> But When I test it, by stopping rsyslog on the central log collector,
> and
> running this small shell script on the client:
> 
> #! /bin/sh
> for ((i=1;i<=10000;i+=1)); do
>   logger Log line $i
> done
> 
> The shell script does not complete until I restart rsyslog, on the
> central
> log collector, and when I look at the timestamps in the logfile on the
> log
> collector, I see a large time difference between log line 3303 and
> 3304.
> 
> My plan is to get rsyslog to spool all log entries locally, while the
> connection the the central log collector is interrupted, and then
flush
> all
> the spooled log messages to the central log collector when the
> communication
> is restored - while the rest of the client system continues with
> business as
> usual.
> 
> I can send you a debug log is needed.
> 
> 
> -- David Darville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 16:41:14 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 16:41:14 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D86@rmail.adiscon.com>

A new Version of the beta branch has been released a few moments ago
(mailing list should be informed very soon).
I have rewritten and added some Javascript code in order to enhance the
detail popup window. It does work now equal and very well in Firefox,
Opera and Internet Explorer. It looks a little bit better in Safari, but
it still has redraw problems. But I would say these are browser problems
with css. 

You can download the new Version from here: 
http://www.phplogcon.org/downloads

If you are working with the devel branch, you will have to wait until
tomorrow. 

Best regards,
Andre Lorbach


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:30 PM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon UI question
> 
> Is there a way to turn off the hover-exploding info windows when you
> move the mouse over the "message" field?
> 
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> away.
> 
> thx
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 17:00:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 17:00:25 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com><ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF6F@grfint2.intern.adiscon.com>

Just to prove the point: I have now implemented it. As expected, it is a
very small code change (once you know where to apply it ;)). Have a look
at the primary patch:

http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=action.c;h=3a2584de0f
24fdf3f1af0c35748d29fe5e3a3845;hp=f72194059d41f011ae6daf6b6aa1a61d147255
3e;hb=1480263b026984a3d48a7f750a78911777464797;hpb=0a7f964436af73f2e7fbd
403b563f8d5a743f4a5

and note that most is comment. The amount of code actually executed each
time is rater limited and does neither bear a large memory footprint,
nor execution footprint. If the feature is not used, it is one simple
branch. Even if it is used, the performance effects are very limited.
Most importantly, a costly call to time() could be avoided by using the
value that was already present (but needed a bit of reordering).

I am going to this detail just to explain the fine difference between
what belongs into the core (even though it is not "real core
functionality") compared to what must not get into it.

For example, if I had implemented that via a plugin, I would have needed
at least one (indirectly addressed) procedure call branch, still an if,
plus a return branch. Not good for speculative execution. Also, the
plumbing would probably have required more code than the full patch
shown (what means bad from a maintenance point of view as well a bad
from a CPU memory cache point of view ;)).

It remains the argument that if the code would not have been introduced,
the core would be a little more slim. That's right ;) An easy solution
would have been conditional compilation, but I have not applied it as I
think the few extra bytes and CPU cycles really don't matter (that
much).

Just for you info...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, August 01, 2008 5:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Actually, in this case the (limited) functionality will go into the
> core. But that's not because we need it for alerting but it is
> something
> that was scheduled for the core engine at all (because there are other
> use cases besides alerting where you need it, e.g. start some
> corrective
> action only after the n-th error indication). Plus, it will be very
> limited code. There are some other things, namely the enhanced rate
> limiter, which will go to the core. I have some ways to do that via
> plug-ins too, but given the expected complexity of this functionality
> vs
> the expected complexity of pluginizing them, the choice to go to core
> is
> really obvious. Another point where one could debate is enhanced
> queuing. This, in the long term, is a candidate for being moved to a
> plugin because many installations do never use disk-based (or
assisted)
> queues.
> 
> Now comes the important difference: if a generic, any message let me
> know when it happens n times in the row filter would have been needed,
> that would NOT go into the core. Because it does not belong there. It
> is
> quite complex and even performance intense. For that, future version
> will have customer (RainerScript) functions which can be provided by
> library plugins. I have plans to implement such a beast (much later),
> but it will come as a function that you provide the message to and
that
> is only loaded on an as-needed basis.
> 
> I hope this clarifies.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of (private) HKS
> > Sent: Friday, August 01, 2008 5:17 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> >
> > Thanks for the link, that post makes a lot of sense. I interpreted
> > this discussion as moving towards adding alert functionality to the
> > rsyslogd core - but your stance of keeping the rsyslogd core lean
and
> > efficient while plugins provide a full suite of event handling
> > processes is reassuring.
> >
> > I'm looking forward to seeing where you take this plugin
> architecture.
> >
> > -HKS
> >
> > On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> > > That's not a silly one ;) And it is coming up every now and then.
> The
> > > last time it came up, I was smart enough to write a blogpost:
> > > http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-
> > plugins.ht
> > > ml
> > >
> > > In short, and to this questions: there are different schools of
> > thought.
> > > If you think about a plain ole syslogd shuffling data to disk
> files,
> > you
> > > do not need that. My vision of the syslogd (actually the "event
> > logging
> > > and alerting") subsystem is much broader. IMHO, it should support
> > > anything that is needed to gather, process and persistently store
> > > events. Also note that I say "events" for a reason - syslog
> messages
> > are
> > > just a subset of the potential set of events.
> > >
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> > >> Sent: Friday, August 01, 2008 4:25 PM
> > >> To: rsyslog-users
> > >> Subject: Re: [rsyslog] Alert when multiple repeated lines are
> found
> > >>
> > >> This may be a silly question, but is the syslog daemon the proper
> > >> place for something like this?
> > >>
> > >> -HKS
> > >>
> > >> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap
<julianokyap at gmail.com>
> > >> wrote:
> > >> > Roger that Rainer.
> > >> >
> > >> > Thanks,
> > >> > Julian
> > >> >
> > >> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > >> > <rgerhards at hq.adiscon.com> wrote:
> > >> >> OK, that greatly simplifies things. Actually, it now boils
down
> > to
> > >> >> "execute an action only on the n-the time the filter evaluates
> to
> > >> true".
> > >> >> I think this is quite easy to implement, but I must verify
> > that...
> > >> >>
> > >> >> Rainer
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 17:08:20 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 17:08:20 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>

Julian,

as you have probably seen in my other post, I have implemented the
functionality. I have now also created a test tarball. I'd appreciate if
you could obtain it from

http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz

and give it a try. Read ./doc/rsyslog_conf.html in regard to
$ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
what you intend to do, this should work:

$ActionExecOnlyEveryNthTime 3
*.* ..your action..

You don't need the timeout, but I have included it for completeness.
Well, actually if I were you I'd think if you really don't need it. Is
it really OK that "three in a row" means one each day?

Please provide feedback on this feature.

Thanks,
Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 12:14 PM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> Roger that Rainer.
> 
> Thanks,
> Julian
> 
> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > OK, that greatly simplifies things. Actually, it now boils down to
> > "execute an action only on the n-the time the filter evaluates to
> true".
> > I think this is quite easy to implement, but I must verify that...
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> Sent: Friday, August 01, 2008 11:03 AM
> >> To: Rainer Gerhards
> >> Cc: rsyslog at lists.adiscon.com
> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > found
> >>
> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >> <rgerhards at hq.adiscon.com> wrote:
> >> > Just one more re-confirmation:
> >> >
> >> >> What I'm looking for:
> >> >> if $msg == 'This is really bad' happens 3 times in a row then
> >> >> :ommail:;mailBody
> >> >>
> >> >> This would be nice but is not required since the 'This is really
> >> bad'
> >> >> message in my case is very unique:
> >> >> if ($msg == 'This is really bad' and $server == 'server' and
> >> $program
> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
> >> >
> >> > So you would actually use such a rule. If "this other thing is
> > really
> >> > bad" happened three times, the rule shall not trigger. Is this
> > right?
> >>
> >> Yes, I would use such a rule.  It would make what is already an
> >> awesome application even more awesome. :P  I am also willing to
test
> >> it out and run the latest development version...  Which I'm doing
> >> anyway.
> >>
> >> And yes, what you just wrote is correct.
> >>
> >> - Julian
> >


From friedl at hq.adiscon.com  Thu Aug  7 17:28:30 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Thu, 7 Aug 2008 17:28:30 +0200
Subject: [rsyslog] rsyslog 2.0.6 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF73@grfint2.intern.adiscon.com>

Hi all,

we have just released rsyslog 2.0.6, a version of the ultra-conservative
v2-stable branch. There is a single bugfix in this release: IPv6
addresses could not be specified in forwarding actions, because they
contain colons and the colon character was already used for some other
purpose. IPv6 addresses can now be specified inside of square brackets.
This is a recommended update for all v2-stable branch users.


Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-125.phtml

Changelog:

http://www.rsyslog.com/Article266.phtml


As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From julianokyap at gmail.com  Fri Aug  8 00:38:07 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 7 Aug 2008 12:38:07 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
Message-ID: <d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>

Rainer,

Initial testing looks fine.  I'll try some more to see if I can break it.

- Julian

On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Julian,
>
> as you have probably seen in my other post, I have implemented the
> functionality. I have now also created a test tarball. I'd appreciate if
> you could obtain it from
>
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>
> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
> what you intend to do, this should work:
>
> $ActionExecOnlyEveryNthTime 3
> *.* ..your action..
>
> You don't need the timeout, but I have included it for completeness.
> Well, actually if I were you I'd think if you really don't need it. Is
> it really OK that "three in a row" means one each day?
>
> Please provide feedback on this feature.
>
> Thanks,
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 01, 2008 12:14 PM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> Roger that Rainer.
>>
>> Thanks,
>> Julian
>>
>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>> > OK, that greatly simplifies things. Actually, it now boils down to
>> > "execute an action only on the n-the time the filter evaluates to
>> true".
>> > I think this is quite easy to implement, but I must verify that...
>> >
>> > Rainer
>> >
>> >> -----Original Message-----
>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >> Sent: Friday, August 01, 2008 11:03 AM
>> >> To: Rainer Gerhards
>> >> Cc: rsyslog at lists.adiscon.com
>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> > found
>> >>
>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> >> <rgerhards at hq.adiscon.com> wrote:
>> >> > Just one more re-confirmation:
>> >> >
>> >> >> What I'm looking for:
>> >> >> if $msg == 'This is really bad' happens 3 times in a row then
>> >> >> :ommail:;mailBody
>> >> >>
>> >> >> This would be nice but is not required since the 'This is really
>> >> bad'
>> >> >> message in my case is very unique:
>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>> >> $program
>> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>> >> >
>> >> > So you would actually use such a rule. If "this other thing is
>> > really
>> >> > bad" happened three times, the rule shall not trigger. Is this
>> > right?
>> >>
>> >> Yes, I would use such a rule.  It would make what is already an
>> >> awesome application even more awesome. :P  I am also willing to
> test
>> >> it out and run the latest development version...  Which I'm doing
>> >> anyway.
>> >>
>> >> And yes, what you just wrote is correct.
>> >>
>> >> - Julian
>> >
>


From julianokyap at gmail.com  Fri Aug  8 01:55:03 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 7 Aug 2008 13:55:03 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
Message-ID: <d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>

Yep, after further testing this works great!  Thanks Rainer.

On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com> wrote:
> Rainer,
>
> Initial testing looks fine.  I'll try some more to see if I can break it.
>
> - Julian
>
> On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> Julian,
>>
>> as you have probably seen in my other post, I have implemented the
>> functionality. I have now also created a test tarball. I'd appreciate if
>> you could obtain it from
>>
>> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>>
>> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
>> what you intend to do, this should work:
>>
>> $ActionExecOnlyEveryNthTime 3
>> *.* ..your action..
>>
>> You don't need the timeout, but I have included it for completeness.
>> Well, actually if I were you I'd think if you really don't need it. Is
>> it really OK that "three in a row" means one each day?
>>
>> Please provide feedback on this feature.
>>
>> Thanks,
>> Rainer
>>
>>> -----Original Message-----
>>> From: Julian Yap [mailto:julianokyap at gmail.com]
>>> Sent: Friday, August 01, 2008 12:14 PM
>>> To: Rainer Gerhards
>>> Cc: rsyslog at lists.adiscon.com
>>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> found
>>>
>>> Roger that Rainer.
>>>
>>> Thanks,
>>> Julian
>>>
>>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>>> <rgerhards at hq.adiscon.com> wrote:
>>> > OK, that greatly simplifies things. Actually, it now boils down to
>>> > "execute an action only on the n-the time the filter evaluates to
>>> true".
>>> > I think this is quite easy to implement, but I must verify that...
>>> >
>>> > Rainer
>>> >
>>> >> -----Original Message-----
>>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>>> >> Sent: Friday, August 01, 2008 11:03 AM
>>> >> To: Rainer Gerhards
>>> >> Cc: rsyslog at lists.adiscon.com
>>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>>> > found
>>> >>
>>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>>> >> <rgerhards at hq.adiscon.com> wrote:
>>> >> > Just one more re-confirmation:
>>> >> >
>>> >> >> What I'm looking for:
>>> >> >> if $msg == 'This is really bad' happens 3 times in a row then
>>> >> >> :ommail:;mailBody
>>> >> >>
>>> >> >> This would be nice but is not required since the 'This is really
>>> >> bad'
>>> >> >> message in my case is very unique:
>>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>>> >> $program
>>> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>>> >> >
>>> >> > So you would actually use such a rule. If "this other thing is
>>> > really
>>> >> > bad" happened three times, the rule shall not trigger. Is this
>>> > right?
>>> >>
>>> >> Yes, I would use such a rule.  It would make what is already an
>>> >> awesome application even more awesome. :P  I am also willing to
>> test
>>> >> it out and run the latest development version...  Which I'm doing
>>> >> anyway.
>>> >>
>>> >> And yes, what you just wrote is correct.
>>> >>
>>> >> - Julian
>>> >
>>
>


From mic at npgx.com.au  Fri Aug  8 03:58:18 2008
From: mic at npgx.com.au (Michael Mansour)
Date: Fri, 8 Aug 2008 12:58:18 +1100
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>
References: <4898C644.5060906@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>
Message-ID: <20080808014639.M81869@npgx.com.au>

Hi,

> > Ok I have Safari installed now, and I see what you mean.
> > Actually this is some kind of a redraw issue within Safari, which does
> > not redraw the content which was behind the popup window. If you
> resize
> > the window after some popup messages have been viewed, you will see
> > that
> > the content is redrawn.
> > 
> > However I think we are going to work on this popup window anyway, as
> it
> > is not perfect yet.
> 
> Let me express my personal opinion in the hope that others will also
> provide some input: I do *not* think it is well spent time to enhance
> this Window. The nice thing is that it can be turned off and that should
> solve most problems ;) Actually, I (again personally!) find the 
> window in itself quite problematic: if it is turned on, it blocks me 
> from doing some things in the natural way, because it always pops up 
> and blocks view of parts of the web page *just* because I happen to 
> have moved the mouse over some part of the page. For me, this leads 
> to such funny things that I need to concentrate on moving the mouse 
> from bottom to top
> (and not in the opposite direction ;)) when I want to activate some 
> context menu items - because otherwise the popup would block me from 
> reaching that point.
> 
> Again, this is just my personal view, but I would like to bring that
> topic up to a broader audience.

I can only describe my opinion by giving an example of a firefox extension I
like to use. 

I use an extension (or Add-on in FF3 terminology) called Cooliris Previews.
This extension is nifty in that it allows you to open a sub window (pop up
window) for URL's on web pages. From there you can "lock" the sub window down
and mouse over other URL's and it updates the sub windows.

I find this very handy when going through helpdesk cases in our helpdesk
system or emails in my webmial system.

The problem is, before bringing up the sub window, each URL you mouse over
brings up the cooliris previews icon to the right of the link (location
configurable). By default, mousing over the icon brings up the sub window.
People that use cooliris would quickly find this very annoying because sub
windows will constantly open up by accident. So after changing the default to
only allow opening of a sub window by clicking the cooliris icon, it's more
pleasant to use.

After a while though even this becomes annoying, because 90% of my browsing
time I don't use/need cooliris previews, and many times I accidentally click
the icon and bring up sub windows by accident again (you'll realise how easy
it is to make this mistake when using it).

So now I have the bottom right toolbar which enables/disables the extension,
so I no longer see the cooliris icon to the right of every URL I mouse over.

Sometimes though (when I have multiple FF Windows open) that toolbar icon
turns itself on automatically, and does so in every tab that's open in every
window. It's not perfect (and maybe the developers concentrate more on having
this feature enabled in a FF browser than not) but I have come to the point of
considering removing the extension because the pop ups can be annoying when
browsing.

Regards,

Michael.

> Rainer



From rgerhards at hq.adiscon.com  Fri Aug  8 10:39:14 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 10:39:14 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>

Thanks for the feedback, it will now be part of the next devel release,
I think some time next week :)

Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 08, 2008 1:55 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> Yep, after further testing this works great!  Thanks Rainer.
> 
> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
> wrote:
> > Rainer,
> >
> > Initial testing looks fine.  I'll try some more to see if I can
break
> it.
> >
> > - Julian
> >
> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> Julian,
> >>
> >> as you have probably seen in my other post, I have implemented the
> >> functionality. I have now also created a test tarball. I'd
> appreciate if
> >> you could obtain it from
> >>
> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> >>
> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> >> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout.
> For
> >> what you intend to do, this should work:
> >>
> >> $ActionExecOnlyEveryNthTime 3
> >> *.* ..your action..
> >>
> >> You don't need the timeout, but I have included it for
completeness.
> >> Well, actually if I were you I'd think if you really don't need it.
> Is
> >> it really OK that "three in a row" means one each day?
> >>
> >> Please provide feedback on this feature.
> >>
> >> Thanks,
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> >>> Sent: Friday, August 01, 2008 12:14 PM
> >>> To: Rainer Gerhards
> >>> Cc: rsyslog at lists.adiscon.com
> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> >> found
> >>>
> >>> Roger that Rainer.
> >>>
> >>> Thanks,
> >>> Julian
> >>>
> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >>> <rgerhards at hq.adiscon.com> wrote:
> >>> > OK, that greatly simplifies things. Actually, it now boils down
> to
> >>> > "execute an action only on the n-the time the filter evaluates
to
> >>> true".
> >>> > I think this is quite easy to implement, but I must verify
> that...
> >>> >
> >>> > Rainer
> >>> >
> >>> >> -----Original Message-----
> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> >>> >> To: Rainer Gerhards
> >>> >> Cc: rsyslog at lists.adiscon.com
> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> are
> >>> > found
> >>> >>
> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >>> >> <rgerhards at hq.adiscon.com> wrote:
> >>> >> > Just one more re-confirmation:
> >>> >> >
> >>> >> >> What I'm looking for:
> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
then
> >>> >> >> :ommail:;mailBody
> >>> >> >>
> >>> >> >> This would be nice but is not required since the 'This is
> really
> >>> >> bad'
> >>> >> >> message in my case is very unique:
> >>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
> >>> >> $program
> >>> >> >> == 'program') happens 3 times in a row then
:ommail:;mailBody
> >>> >> >
> >>> >> > So you would actually use such a rule. If "this other thing
is
> >>> > really
> >>> >> > bad" happened three times, the rule shall not trigger. Is
this
> >>> > right?
> >>> >>
> >>> >> Yes, I would use such a rule.  It would make what is already an
> >>> >> awesome application even more awesome. :P  I am also willing to
> >> test
> >>> >> it out and run the latest development version...  Which I'm
> doing
> >>> >> anyway.
> >>> >>
> >>> >> And yes, what you just wrote is correct.
> >>> >>
> >>> >> - Julian
> >>> >
> >>
> >


From rgerhards at hq.adiscon.com  Fri Aug  8 12:28:11 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 12:28:11 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>

Thanks, I now received the log files. And, of course, they made
immediately clear what is going on ;) What you experience is flow
control kicking in. Let me first elaborate a bit:

Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
the local socket) and those who can not (like UDP). The idea is that we
can slow down senders where this does not result in loss of messages,
leaving room inside the queue on busy systems for those where we can not
do that. We had cases where not delaying caused a lot of grief.

There are also two watermarks, the low and high delay mark. If the low
delay mark is reached, a delayable source is delayed for one second. It
is at 70% of the (in-memory) queue capacity. Again, this is all done for
some very good reasons.

However, when I introduced these features, I did not make them
configurable. Also, I did classify the local unix socket as one who can
be delayed. In your scenario, this does not stop the system, but once
the local log socket is filled up, it needs to wait one second for each
message before it is pulled.

Again, the rate limiting logic is there for good reasons. So what I now
see is that I need to add a few config statements to allow to control it
according to your needs. I am also unsure if I should re-classify
imuxsock as an input which cannot be delayed - but that also can have
some drawbacks. Maybe the best thing to do is make its state
configurable and use "not to be delayed" as default.

Anyone with a thought on that?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 2:19 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> >From what I see, this should work. So please send the debug log,
> client
> (more important) and server please :)
> 
> Thanks,
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of David Darville
> > Sent: Thursday, August 07, 2008 2:06 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Problems with spooling to disk
> >
> > Hello everybody
> >
> > Now I'm trying to configure rsyslog to spool log messages to disk,
> > while the
> > connection to the central log collector is down. But it does not
> > work......
> >
> > I am using rsyslog v 3.21.2
> >
> > I have configured the client rsyslog like this:
> >
> > $ModLoad imuxsock
> > $ModLoad omrelp
> > $WorkDirectory /var/spool/rsyslog
> > $ActionQueueType LinkedList
> > $ActionQueueFileName rsyslog-buffer
> > $ActionResumeRetryCount -1
> > $ActionQueueSaveOnShutdown on
> > *.* :omrelp:172.16.0.43:2514
> >
> > Which is similar to the reliable forwarding configuration from the
> > documentation, except that I prefer to use RELP.
> >
> > But When I test it, by stopping rsyslog on the central log
collector,
> > and
> > running this small shell script on the client:
> >
> > #! /bin/sh
> > for ((i=1;i<=10000;i+=1)); do
> >   logger Log line $i
> > done
> >
> > The shell script does not complete until I restart rsyslog, on the
> > central
> > log collector, and when I look at the timestamps in the logfile on
> the
> > log
> > collector, I see a large time difference between log line 3303 and
> > 3304.
> >
> > My plan is to get rsyslog to spool all log entries locally, while
the
> > connection the the central log collector is interrupted, and then
> flush
> > all
> > the spooled log messages to the central log collector when the
> > communication
> > is restored - while the rest of the client system continues with
> > business as
> > usual.
> >
> > I can send you a debug log is needed.
> >
> >
> > -- David Darville
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Fri Aug  8 12:38:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 12:38:56 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF81@grfint2.intern.adiscon.com>

David,

I have done a quick and dirty (not to stay) fix, which makes imuxsock a
NO_DELAY input:

http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test4.tar.gz

I'd appreciate if you could give it a try and report back.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, August 08, 2008 12:28 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> Thanks, I now received the log files. And, of course, they made
> immediately clear what is going on ;) What you experience is flow
> control kicking in. Let me first elaborate a bit:
> 
> Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> the local socket) and those who can not (like UDP). The idea is that
we
> can slow down senders where this does not result in loss of messages,
> leaving room inside the queue on busy systems for those where we can
> not
> do that. We had cases where not delaying caused a lot of grief.
> 
> There are also two watermarks, the low and high delay mark. If the low
> delay mark is reached, a delayable source is delayed for one second.
It
> is at 70% of the (in-memory) queue capacity. Again, this is all done
> for
> some very good reasons.
> 
> However, when I introduced these features, I did not make them
> configurable. Also, I did classify the local unix socket as one who
can
> be delayed. In your scenario, this does not stop the system, but once
> the local log socket is filled up, it needs to wait one second for
each
> message before it is pulled.
> 
> Again, the rate limiting logic is there for good reasons. So what I
now
> see is that I need to add a few config statements to allow to control
> it
> according to your needs. I am also unsure if I should re-classify
> imuxsock as an input which cannot be delayed - but that also can have
> some drawbacks. Maybe the best thing to do is make its state
> configurable and use "not to be delayed" as default.
> 
> Anyone with a thought on that?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, August 07, 2008 2:19 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Problems with spooling to disk
> >
> > >From what I see, this should work. So please send the debug log,
> > client
> > (more important) and server please :)
> >
> > Thanks,
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of David Darville
> > > Sent: Thursday, August 07, 2008 2:06 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] Problems with spooling to disk
> > >
> > > Hello everybody
> > >
> > > Now I'm trying to configure rsyslog to spool log messages to disk,
> > > while the
> > > connection to the central log collector is down. But it does not
> > > work......
> > >
> > > I am using rsyslog v 3.21.2
> > >
> > > I have configured the client rsyslog like this:
> > >
> > > $ModLoad imuxsock
> > > $ModLoad omrelp
> > > $WorkDirectory /var/spool/rsyslog
> > > $ActionQueueType LinkedList
> > > $ActionQueueFileName rsyslog-buffer
> > > $ActionResumeRetryCount -1
> > > $ActionQueueSaveOnShutdown on
> > > *.* :omrelp:172.16.0.43:2514
> > >
> > > Which is similar to the reliable forwarding configuration from the
> > > documentation, except that I prefer to use RELP.
> > >
> > > But When I test it, by stopping rsyslog on the central log
> collector,
> > > and
> > > running this small shell script on the client:
> > >
> > > #! /bin/sh
> > > for ((i=1;i<=10000;i+=1)); do
> > >   logger Log line $i
> > > done
> > >
> > > The shell script does not complete until I restart rsyslog, on the
> > > central
> > > log collector, and when I look at the timestamps in the logfile on
> > the
> > > log
> > > collector, I see a large time difference between log line 3303 and
> > > 3304.
> > >
> > > My plan is to get rsyslog to spool all log entries locally, while
> the
> > > connection the the central log collector is interrupted, and then
> > flush
> > > all
> > > the spooled log messages to the central log collector when the
> > > communication
> > > is restored - while the rest of the client system continues with
> > > business as
> > > usual.
> > >
> > > I can send you a debug log is needed.
> > >
> > >
> > > -- David Darville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From friedl at hq.adiscon.com  Fri Aug  8 14:18:56 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 8 Aug 2008 14:18:56 +0200
Subject: [rsyslog] rsyslog 3.18.2 released (stable)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF85@grfint2.intern.adiscon.com>

Hi all,

we have just released rsyslog 3.18.2, a version of the v3-stable branch.
There is a single bugfix in this release: IPv6 addresses could not be
specified in forwarding actions, because they contain colons and the
colon character was already used for some other purpose. IPv6 addresses
can now be specified inside of square brackets. This is the same fix
that was released for v2-stable yesterday. This is a recommended update
for all v3-stable branch users.


Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-126.phtml

Changelog:

http://www.rsyslog.com/Article268.phtml


As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From rory at ooma.com  Fri Aug  8 23:32:48 2008
From: rory at ooma.com (Rory Toma)
Date: Fri, 08 Aug 2008 14:32:48 -0700
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>
References: <489A3DDD.4000205@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>
Message-ID: <489CBB80.10507@ooma.com>

Actually, I ended up doing a "LIKE" search on hostname, looking at the 
query via full processlist and running the query by hand, assuming it 
was the same, just without the "LIKE" syntax.

Turns out I had a horked index in a table that MAX(id) exposed. Fixed it 
and moved on.

thx

Andre Lorbach wrote:
> One question, are you searching for a single word or a phrase? 
> If you are searching for a phrase, please use + instead of spaces. 
> For example, you want to find a syslogtag like "mysql safe", then you
> use this search: 
> source:=mysql+safe
>
> If you do not use the + instead of the space, the filter will be
> splitted at the word safe, which will be considered as message filter
> then. 
>
> Maybe this helps - best regards,
> Andre Lorbach
>
>
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> Sent: Thursday, August 07, 2008 2:12 AM
>> To: rsyslog-users
>> Subject: [rsyslog] see sql queries on phplogcon
>>
>> How do I see what queries are being done in phplogcon? I don't have
>>     
> sql
>   
>> logging turned on in my db.
>>
>> the := syntax seems to be no longer working for me. I get an error
>> telling me there were no syslog records found when I try it.
>>
>>
>> I've set up my SystemEvents table as a union and merged all the tables
>> in with data. This shouldn't affect it, I would think. My sql queries
>>     
> by
>   
>> hand seem to work fine.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   



From mmcgrath at redhat.com  Tue Aug 12 17:37:35 2008
From: mmcgrath at redhat.com (Mike McGrath)
Date: Tue, 12 Aug 2008 10:37:35 -0500 (CDT)
Subject: [rsyslog] rsyslog dropping logs
In-Reply-To: <1216896024.7184.189.camel@rgf9dev.intern.adiscon.com>
References: <alpine.LFD.1.10.0807231413270.14749@laptop.mmcgrath.net>
	<1216896024.7184.189.camel@rgf9dev.intern.adiscon.com>
Message-ID: <alpine.LFD.1.10.0808121034550.5157@laptop.mmcgrath.net>

On Thu, 24 Jul 2008, Rainer Gerhards wrote:

> (I am not commenting on v2 vs. v3 as this is already done)
>
> First of all, we need to keep in mind that UDP is inherently lossy. Even
> when a frame is seen received by the local stack, it does not mean that
> it will eventually be forwarded to the application. If message bursts
> come in very quickly and the OS scheduler does not schedule the app fast
> enough to receive this messages (or the app is too slow in itself! ;))
> new frames may overwrite frames inside the stack's receive buffers.
>
> So it is always a good idea to avoid UDP if that's possible.
>
> HOWEVER, I, too, find it somewhat unusual that around 90% of all
> incoming frames are discarded before the rsyslog receiver could process
> them. One explanation I have is that you have bursts (or volume in
> general) that outperforms the configured actions. Having seen the config
> file, and seeing it does not include any database writer, it is hard to
> imagine this should happen, assuming reasonable hardware sizing is used.
> A cause could be excessive synchronous writes. Many rules do not put a
> dash in front of the file name and without it (in v2), every write is
> immediately synced. This is very costly. But still, I have never seen
> that this alone outperforms a system.
>
> To dig deeper into what is happening, a debug log would be most useful,
> together with the information which frames have been seen in tcpdump but
> NOT in one of the log files. You can enable debug mode via -dn command
> line switch and is recommended to run rsyslog interactively while doing
> so. Then, you can simply capture its output via stdout redirection.
> Please note that debug mode generates considerable output, and requires
> considerable additional processing time. In any case, though, it should
> show us where the bottleneck is. Please note that I need a consistent
> excerpt from the debug log that shows how things began and how it worked
> during the fault conditions. Usually, this means I need everything ;)
> Debug logs may also reveal sensitive information, even passwords, so you
> should be careful in what you do. I am used to log files around the size
> of 1GB. With reasonable compression, the transfer is usually not a
> problem (but I suggest you place them on a server for me to download).
> Download links and/or smaller logs you can email me privately at
> rgerhards at gmail.com (please NOT at my primary, adiscon, email address).
>
> I hope this helps and I am looking forward for the additional
> information.

So after a long hiatus and a new config the problems went away but only
temporarily.  I think because of a rebooted box.  They have returned.  I'm
going to mail the logs your way.  I can send you more.  Its very easy to
reproduce.

	-Mike









============== Snip ===================
>
> On Wed, 2008-07-23 at 14:21 -0500, Mike McGrath wrote:
> > I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central
> > logging server.  When running tcpdump I'm seeing all the udp packets
> > coming in but many of them are not getting logged.  And we're talking
> > like 10% or so getting logged (maybe less) and the rest are just lost.
> > I've attached my config file.
> >
> > (side note, if I'm doing something stupid in the config please correct me)
> >
> > 	-Mike
> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>



From julianokyap at gmail.com  Wed Aug 13 12:32:00 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Wed, 13 Aug 2008 00:32:00 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
Message-ID: <d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>

It's kind of strange but I've had this running for about a week now...
 I seem to have had 2 false alerts for no apparent reason.


On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Thanks for the feedback, it will now be part of the next devel release,
> I think some time next week :)
>
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 08, 2008 1:55 AM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> Yep, after further testing this works great!  Thanks Rainer.
>>
>> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
>> wrote:
>> > Rainer,
>> >
>> > Initial testing looks fine.  I'll try some more to see if I can
> break
>> it.
>> >
>> > - Julian
>> >
>> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> >> Julian,
>> >>
>> >> as you have probably seen in my other post, I have implemented the
>> >> functionality. I have now also created a test tarball. I'd
>> appreciate if
>> >> you could obtain it from
>> >>
>> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>> >>
>> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> >> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout.
>> For
>> >> what you intend to do, this should work:
>> >>
>> >> $ActionExecOnlyEveryNthTime 3
>> >> *.* ..your action..
>> >>
>> >> You don't need the timeout, but I have included it for
> completeness.
>> >> Well, actually if I were you I'd think if you really don't need it.
>> Is
>> >> it really OK that "three in a row" means one each day?
>> >>
>> >> Please provide feedback on this feature.
>> >>
>> >> Thanks,
>> >> Rainer
>> >>
>> >>> -----Original Message-----
>> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >>> Sent: Friday, August 01, 2008 12:14 PM
>> >>> To: Rainer Gerhards
>> >>> Cc: rsyslog at lists.adiscon.com
>> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> >> found
>> >>>
>> >>> Roger that Rainer.
>> >>>
>> >>> Thanks,
>> >>> Julian
>> >>>
>> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> >>> <rgerhards at hq.adiscon.com> wrote:
>> >>> > OK, that greatly simplifies things. Actually, it now boils down
>> to
>> >>> > "execute an action only on the n-the time the filter evaluates
> to
>> >>> true".
>> >>> > I think this is quite easy to implement, but I must verify
>> that...
>> >>> >
>> >>> > Rainer
>> >>> >
>> >>> >> -----Original Message-----
>> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >>> >> Sent: Friday, August 01, 2008 11:03 AM
>> >>> >> To: Rainer Gerhards
>> >>> >> Cc: rsyslog at lists.adiscon.com
>> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
>> are
>> >>> > found
>> >>> >>
>> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> >>> >> <rgerhards at hq.adiscon.com> wrote:
>> >>> >> > Just one more re-confirmation:
>> >>> >> >
>> >>> >> >> What I'm looking for:
>> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
> then
>> >>> >> >> :ommail:;mailBody
>> >>> >> >>
>> >>> >> >> This would be nice but is not required since the 'This is
>> really
>> >>> >> bad'
>> >>> >> >> message in my case is very unique:
>> >>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>> >>> >> $program
>> >>> >> >> == 'program') happens 3 times in a row then
> :ommail:;mailBody
>> >>> >> >
>> >>> >> > So you would actually use such a rule. If "this other thing
> is
>> >>> > really
>> >>> >> > bad" happened three times, the rule shall not trigger. Is
> this
>> >>> > right?
>> >>> >>
>> >>> >> Yes, I would use such a rule.  It would make what is already an
>> >>> >> awesome application even more awesome. :P  I am also willing to
>> >> test
>> >>> >> it out and run the latest development version...  Which I'm
>> doing
>> >>> >> anyway.
>> >>> >>
>> >>> >> And yes, what you just wrote is correct.
>> >>> >>
>> >>> >> - Julian
>> >>> >
>> >>
>> >
>


From rgerhards at hq.adiscon.com  Wed Aug 13 12:47:36 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 12:47:36 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
	<d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>

Do you use $ActionExecOnlyEveryNthTimeTimeout?

Rainer


> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Wednesday, August 13, 2008 12:32 PM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> It's kind of strange but I've had this running for about a week now...
>  I seem to have had 2 false alerts for no apparent reason.
> 
> 
> On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Thanks for the feedback, it will now be part of the next devel
> release,
> > I think some time next week :)
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> Sent: Friday, August 08, 2008 1:55 AM
> >> To: Rainer Gerhards
> >> Cc: rsyslog at lists.adiscon.com
> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > found
> >>
> >> Yep, after further testing this works great!  Thanks Rainer.
> >>
> >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
> >> wrote:
> >> > Rainer,
> >> >
> >> > Initial testing looks fine.  I'll try some more to see if I can
> > break
> >> it.
> >> >
> >> > - Julian
> >> >
> >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> >> > <rgerhards at hq.adiscon.com> wrote:
> >> >> Julian,
> >> >>
> >> >> as you have probably seen in my other post, I have implemented
> the
> >> >> functionality. I have now also created a test tarball. I'd
> >> appreciate if
> >> >> you could obtain it from
> >> >>
> >> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> >> >>
> >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> >> >> $ActionExecOnlyEveryNthTime and
> $ActionExecOnlyEveryNthTimeTimeout.
> >> For
> >> >> what you intend to do, this should work:
> >> >>
> >> >> $ActionExecOnlyEveryNthTime 3
> >> >> *.* ..your action..
> >> >>
> >> >> You don't need the timeout, but I have included it for
> > completeness.
> >> >> Well, actually if I were you I'd think if you really don't need
> it.
> >> Is
> >> >> it really OK that "three in a row" means one each day?
> >> >>
> >> >> Please provide feedback on this feature.
> >> >>
> >> >> Thanks,
> >> >> Rainer
> >> >>
> >> >>> -----Original Message-----
> >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> >>> Sent: Friday, August 01, 2008 12:14 PM
> >> >>> To: Rainer Gerhards
> >> >>> Cc: rsyslog at lists.adiscon.com
> >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> are
> >> >> found
> >> >>>
> >> >>> Roger that Rainer.
> >> >>>
> >> >>> Thanks,
> >> >>> Julian
> >> >>>
> >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >> >>> <rgerhards at hq.adiscon.com> wrote:
> >> >>> > OK, that greatly simplifies things. Actually, it now boils
> down
> >> to
> >> >>> > "execute an action only on the n-the time the filter
evaluates
> > to
> >> >>> true".
> >> >>> > I think this is quite easy to implement, but I must verify
> >> that...
> >> >>> >
> >> >>> > Rainer
> >> >>> >
> >> >>> >> -----Original Message-----
> >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> >> >>> >> To: Rainer Gerhards
> >> >>> >> Cc: rsyslog at lists.adiscon.com
> >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
lines
> >> are
> >> >>> > found
> >> >>> >>
> >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >> >>> >> <rgerhards at hq.adiscon.com> wrote:
> >> >>> >> > Just one more re-confirmation:
> >> >>> >> >
> >> >>> >> >> What I'm looking for:
> >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
> > then
> >> >>> >> >> :ommail:;mailBody
> >> >>> >> >>
> >> >>> >> >> This would be nice but is not required since the 'This is
> >> really
> >> >>> >> bad'
> >> >>> >> >> message in my case is very unique:
> >> >>> >> >> if ($msg == 'This is really bad' and $server == 'server'
> and
> >> >>> >> $program
> >> >>> >> >> == 'program') happens 3 times in a row then
> > :ommail:;mailBody
> >> >>> >> >
> >> >>> >> > So you would actually use such a rule. If "this other
thing
> > is
> >> >>> > really
> >> >>> >> > bad" happened three times, the rule shall not trigger. Is
> > this
> >> >>> > right?
> >> >>> >>
> >> >>> >> Yes, I would use such a rule.  It would make what is already
> an
> >> >>> >> awesome application even more awesome. :P  I am also willing
> to
> >> >> test
> >> >>> >> it out and run the latest development version...  Which I'm
> >> doing
> >> >>> >> anyway.
> >> >>> >>
> >> >>> >> And yes, what you just wrote is correct.
> >> >>> >>
> >> >>> >> - Julian
> >> >>> >
> >> >>
> >> >
> >


From r.bhatia at ipax.at  Wed Aug 13 13:04:07 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Aug 2008 13:04:07 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
Message-ID: <48A2BFA7.8000408@ipax.at>

hi,

Rainer Gerhards wrote:
> Thanks, I now received the log files. And, of course, they made
> immediately clear what is going on ;) What you experience is flow
> control kicking in. Let me first elaborate a bit:
> 
> Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> the local socket) and those who can not (like UDP). The idea is that we
> can slow down senders where this does not result in loss of messages,
> leaving room inside the queue on busy systems for those where we can not
> do that. We had cases where not delaying caused a lot of grief.
> 
> There are also two watermarks, the low and high delay mark. If the low
> delay mark is reached, a delayable source is delayed for one second. It
> is at 70% of the (in-memory) queue capacity. Again, this is all done for
> some very good reasons.
> 
> However, when I introduced these features, I did not make them
> configurable. Also, I did classify the local unix socket as one who can
> be delayed. In your scenario, this does not stop the system, but once
> the local log socket is filled up, it needs to wait one second for each
> message before it is pulled.
> 
> Again, the rate limiting logic is there for good reasons. So what I now
> see is that I need to add a few config statements to allow to control it
> according to your needs. I am also unsure if I should re-classify
> imuxsock as an input which cannot be delayed - but that also can have
> some drawbacks. Maybe the best thing to do is make its state
> configurable and use "not to be delayed" as default.

ok, after reading through your explanation, i (almost) understand the
delay mechanism and that at present it is not configurable.

almost, because i do not get the part where you talk about the "70% of
the (in-memory) queue capacity".

anyways, how does this relate to the original problem, that the
following code stalls the logging process.

>>> #! /bin/sh
>>> for ((i=1;i<=10000;i+=1)); do
>>>   logger Log line $i
>>> done

is this a capacity problem? is this another problem?

i really consider this a critical bug, as i yesterday ran out of
diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
rsyslog-client, which logs locally and to the remote destination,
stalled completely.

cheers,
raoul
ps: i am on 3.16.1-1~bpo40+1
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From rgerhards at hq.adiscon.com  Wed Aug 13 14:33:00 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 14:33:00 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <48A2BFA7.8000408@ipax.at>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
	<48A2BFA7.8000408@ipax.at>
Message-ID: <1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>

On Wed, 2008-08-13 at 13:04 +0200, Raoul Bhatia [IPAX] wrote:
> hi,
> 
> Rainer Gerhards wrote:
> > Thanks, I now received the log files. And, of course, they made
> > immediately clear what is going on ;) What you experience is flow
> > control kicking in. Let me first elaborate a bit:
> > 
> > Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> > the local socket) and those who can not (like UDP). The idea is that we
> > can slow down senders where this does not result in loss of messages,
> > leaving room inside the queue on busy systems for those where we can not
> > do that. We had cases where not delaying caused a lot of grief.
> > 
> > There are also two watermarks, the low and high delay mark. If the low
> > delay mark is reached, a delayable source is delayed for one second. It
> > is at 70% of the (in-memory) queue capacity. Again, this is all done for
> > some very good reasons.
> > 
> > However, when I introduced these features, I did not make them
> > configurable. Also, I did classify the local unix socket as one who can
> > be delayed. In your scenario, this does not stop the system, but once
> > the local log socket is filled up, it needs to wait one second for each
> > message before it is pulled.
> > 
> > Again, the rate limiting logic is there for good reasons. So what I now
> > see is that I need to add a few config statements to allow to control it
> > according to your needs. I am also unsure if I should re-classify
> > imuxsock as an input which cannot be delayed - but that also can have
> > some drawbacks. Maybe the best thing to do is make its state
> > configurable and use "not to be delayed" as default.
> 
> ok, after reading through your explanation, i (almost) understand the
> delay mechanism and that at present it is not configurable.
> 
> almost, because i do not get the part where you talk about the "70% of
> the (in-memory) queue capacity".

An example: if the configured queue capacity is 10,000 (the default
value for the main message queue), than no flow control at all happens
until 7,000 messages are inside the queue. Once they are (and only as
long as they are), flow control is applied to those inputs that permit
it (imuxsock permits it).

It doesn't matter here if the queue is disk-assisted or not. Even with
disk-assisted queues, this setting is very important to prevent an input
(e.g. imfile!) to send massive amounts of data to the queue when there
is no need to (because the input can be flow-controlled).

> 
> anyways, how does this relate to the original problem, that the
> following code stalls the logging process.
> 
> >>> #! /bin/sh
> >>> for ((i=1;i<=10000;i+=1)); do
> >>>   logger Log line $i
> >>> done
> 
> is this a capacity problem? is this another problem?

It is going over the flow control watermark, flow control kicks in and
delays imuxsock writes by one second each as long as there are more than
7,000 messages. No problem, intended behavior, but obviously
questionable. In the imuxsock case, there are also some bad cases.

> 
> i really consider this a critical bug, as i yesterday ran out of
> diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
> rsyslog-client, which logs locally and to the remote destination,
> stalled completely.

We'll - not completely, one message per second ;) But that's probably
complete enough ;)

As a quick fix, I think I'll make imuxsock a non-flow-controllable input
in the v3-stable (3.18.x). The devel will have a switch to make it flow
controlled if one desires. I'll also add some other knobs to fine-tune
the flow control algo, but that probably needs to wait until after my
summer break. These tweaks could easily break things and I don't like
the idea of doing them under time pressure ;)

HTH
Rainer
> 
> cheers,
> raoul
> ps: i am on 3.16.1-1~bpo40+1



From rgerhards at hq.adiscon.com  Wed Aug 13 14:39:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 14:39:44 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com><48A2BFA7.8000408@ipax.at>
	<1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>

This is the v3-stable fix:

http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=plugins/imuxsock/imux
sock.c;h=60ccaffbc526a03011be3ce5869e40bc3e32c319;hp=f8798039c5d53e58fc7
fa17807547862e08c999e;hb=8eb888d049da12e1294a7688432b6325794ade32;hpb=02
f768c37dac9dde424bbd31e378482750fc276c

You can also easily apply it to 3.16 - I just won't do it that version
has been superseded.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, August 13, 2008 2:33 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> On Wed, 2008-08-13 at 13:04 +0200, Raoul Bhatia [IPAX] wrote:
> > hi,
> >
> > Rainer Gerhards wrote:
> > > Thanks, I now received the log files. And, of course, they made
> > > immediately clear what is going on ;) What you experience is flow
> > > control kicking in. Let me first elaborate a bit:
> > >
> > > Rsyslog knows inputs which can be delayed a bit (like tcp receiver
> AND
> > > the local socket) and those who can not (like UDP). The idea is
> that we
> > > can slow down senders where this does not result in loss of
> messages,
> > > leaving room inside the queue on busy systems for those where we
> can not
> > > do that. We had cases where not delaying caused a lot of grief.
> > >
> > > There are also two watermarks, the low and high delay mark. If the
> low
> > > delay mark is reached, a delayable source is delayed for one
> second. It
> > > is at 70% of the (in-memory) queue capacity. Again, this is all
> done for
> > > some very good reasons.
> > >
> > > However, when I introduced these features, I did not make them
> > > configurable. Also, I did classify the local unix socket as one
who
> can
> > > be delayed. In your scenario, this does not stop the system, but
> once
> > > the local log socket is filled up, it needs to wait one second for
> each
> > > message before it is pulled.
> > >
> > > Again, the rate limiting logic is there for good reasons. So what
I
> now
> > > see is that I need to add a few config statements to allow to
> control it
> > > according to your needs. I am also unsure if I should re-classify
> > > imuxsock as an input which cannot be delayed - but that also can
> have
> > > some drawbacks. Maybe the best thing to do is make its state
> > > configurable and use "not to be delayed" as default.
> >
> > ok, after reading through your explanation, i (almost) understand
the
> > delay mechanism and that at present it is not configurable.
> >
> > almost, because i do not get the part where you talk about the "70%
> of
> > the (in-memory) queue capacity".
> 
> An example: if the configured queue capacity is 10,000 (the default
> value for the main message queue), than no flow control at all happens
> until 7,000 messages are inside the queue. Once they are (and only as
> long as they are), flow control is applied to those inputs that permit
> it (imuxsock permits it).
> 
> It doesn't matter here if the queue is disk-assisted or not. Even with
> disk-assisted queues, this setting is very important to prevent an
> input
> (e.g. imfile!) to send massive amounts of data to the queue when there
> is no need to (because the input can be flow-controlled).
> 
> >
> > anyways, how does this relate to the original problem, that the
> > following code stalls the logging process.
> >
> > >>> #! /bin/sh
> > >>> for ((i=1;i<=10000;i+=1)); do
> > >>>   logger Log line $i
> > >>> done
> >
> > is this a capacity problem? is this another problem?
> 
> It is going over the flow control watermark, flow control kicks in and
> delays imuxsock writes by one second each as long as there are more
> than
> 7,000 messages. No problem, intended behavior, but obviously
> questionable. In the imuxsock case, there are also some bad cases.
> 
> >
> > i really consider this a critical bug, as i yesterday ran out of
> > diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
> > rsyslog-client, which logs locally and to the remote destination,
> > stalled completely.
> 
> We'll - not completely, one message per second ;) But that's probably
> complete enough ;)
> 
> As a quick fix, I think I'll make imuxsock a non-flow-controllable
> input
> in the v3-stable (3.18.x). The devel will have a switch to make it
flow
> controlled if one desires. I'll also add some other knobs to fine-tune
> the flow control algo, but that probably needs to wait until after my
> summer break. These tweaks could easily break things and I don't like
> the idea of doing them under time pressure ;)
> 
> HTH
> Rainer
> >
> > cheers,
> > raoul
> > ps: i am on 3.16.1-1~bpo40+1
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From r.bhatia at ipax.at  Wed Aug 13 15:13:48 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Aug 2008 15:13:48 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com><48A2BFA7.8000408@ipax.at>	<1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>
Message-ID: <48A2DE0C.6010502@ipax.at>

Rainer Gerhards wrote:
> This is the v3-stable fix:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=plugins/imuxsock/imux
> sock.c;h=60ccaffbc526a03011be3ce5869e40bc3e32c319;hp=f8798039c5d53e58fc7
> fa17807547862e08c999e;hb=8eb888d049da12e1294a7688432b6325794ade32;hpb=02
> f768c37dac9dde424bbd31e378482750fc276c

thank you for your prompt fix. i will try to simulate the failure again.

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From rgerhards at hq.adiscon.com  Thu Aug 14 11:31:03 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Aug 2008 11:31:03 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com><d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com><d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com><d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>

Just to keep everyone update. I got a private mail with some details not
to be posted publically. However, Julian stated that he does not use
$ActionExecOnlyEveryNthTimeTimeout.

With that (and the non-public) info, I think that the missing
$ActionExecOnlyEveryNthTimeTimeout is the culprit. I have not asked how
often the receiving syslogd is restarted or huped. Julian, please let me
know. If it is sufficiently seldom, I think it could be an "old"
message.

I think about this scenario: a message comes it at, eg., 10am. That's
count 1. Then, nothing happens for the next 10 hours. At 8pm, the next
one occurs. That's count 2. Another 5 hours later, the next message
occurs, bringing the total count to 3. Thus, this message now triggers
the rule.

The question is if this is desired behavior? Or should the rule only be
triggered if the messages occur within an e.g. 20 minute window? If the
later is the case, you need a

$ActionExecOnlyEveryNthTimeTimeout 1200

This directive will timeout previous messages seen if they are older
than 20 minutes. In the example above, the count would now be always 1
and consequently no rule would ever be triggered (and that's the reason
why this directive is present;)).

Does this make sense? Does it match what you see?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, August 13, 2008 12:48 PM
> To: Julian Yap
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Do you use $ActionExecOnlyEveryNthTimeTimeout?
> 
> Rainer
> 
> 
> > -----Original Message-----
> > From: Julian Yap [mailto:julianokyap at gmail.com]
> > Sent: Wednesday, August 13, 2008 12:32 PM
> > To: Rainer Gerhards
> > Cc: rsyslog at lists.adiscon.com
> > Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
> >
> > It's kind of strange but I've had this running for about a week
> now...
> >  I seem to have had 2 false alerts for no apparent reason.
> >
> >
> > On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> > > Thanks for the feedback, it will now be part of the next devel
> > release,
> > > I think some time next week :)
> > >
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> Sent: Friday, August 08, 2008 1:55 AM
> > >> To: Rainer Gerhards
> > >> Cc: rsyslog at lists.adiscon.com
> > >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > > found
> > >>
> > >> Yep, after further testing this works great!  Thanks Rainer.
> > >>
> > >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap
> <julianokyap at gmail.com>
> > >> wrote:
> > >> > Rainer,
> > >> >
> > >> > Initial testing looks fine.  I'll try some more to see if I can
> > > break
> > >> it.
> > >> >
> > >> > - Julian
> > >> >
> > >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> > >> > <rgerhards at hq.adiscon.com> wrote:
> > >> >> Julian,
> > >> >>
> > >> >> as you have probably seen in my other post, I have implemented
> > the
> > >> >> functionality. I have now also created a test tarball. I'd
> > >> appreciate if
> > >> >> you could obtain it from
> > >> >>
> > >> >>
http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> > >> >>
> > >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> > >> >> $ActionExecOnlyEveryNthTime and
> > $ActionExecOnlyEveryNthTimeTimeout.
> > >> For
> > >> >> what you intend to do, this should work:
> > >> >>
> > >> >> $ActionExecOnlyEveryNthTime 3
> > >> >> *.* ..your action..
> > >> >>
> > >> >> You don't need the timeout, but I have included it for
> > > completeness.
> > >> >> Well, actually if I were you I'd think if you really don't
need
> > it.
> > >> Is
> > >> >> it really OK that "three in a row" means one each day?
> > >> >>
> > >> >> Please provide feedback on this feature.
> > >> >>
> > >> >> Thanks,
> > >> >> Rainer
> > >> >>
> > >> >>> -----Original Message-----
> > >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> >>> Sent: Friday, August 01, 2008 12:14 PM
> > >> >>> To: Rainer Gerhards
> > >> >>> Cc: rsyslog at lists.adiscon.com
> > >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> > are
> > >> >> found
> > >> >>>
> > >> >>> Roger that Rainer.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> Julian
> > >> >>>
> > >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > >> >>> <rgerhards at hq.adiscon.com> wrote:
> > >> >>> > OK, that greatly simplifies things. Actually, it now boils
> > down
> > >> to
> > >> >>> > "execute an action only on the n-the time the filter
> evaluates
> > > to
> > >> >>> true".
> > >> >>> > I think this is quite easy to implement, but I must verify
> > >> that...
> > >> >>> >
> > >> >>> > Rainer
> > >> >>> >
> > >> >>> >> -----Original Message-----
> > >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> > >> >>> >> To: Rainer Gerhards
> > >> >>> >> Cc: rsyslog at lists.adiscon.com
> > >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
> lines
> > >> are
> > >> >>> > found
> > >> >>> >>
> > >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> > >> >>> >> <rgerhards at hq.adiscon.com> wrote:
> > >> >>> >> > Just one more re-confirmation:
> > >> >>> >> >
> > >> >>> >> >> What I'm looking for:
> > >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a
row
> > > then
> > >> >>> >> >> :ommail:;mailBody
> > >> >>> >> >>
> > >> >>> >> >> This would be nice but is not required since the 'This
> is
> > >> really
> > >> >>> >> bad'
> > >> >>> >> >> message in my case is very unique:
> > >> >>> >> >> if ($msg == 'This is really bad' and $server ==
'server'
> > and
> > >> >>> >> $program
> > >> >>> >> >> == 'program') happens 3 times in a row then
> > > :ommail:;mailBody
> > >> >>> >> >
> > >> >>> >> > So you would actually use such a rule. If "this other
> thing
> > > is
> > >> >>> > really
> > >> >>> >> > bad" happened three times, the rule shall not trigger.
Is
> > > this
> > >> >>> > right?
> > >> >>> >>
> > >> >>> >> Yes, I would use such a rule.  It would make what is
> already
> > an
> > >> >>> >> awesome application even more awesome. :P  I am also
> willing
> > to
> > >> >> test
> > >> >>> >> it out and run the latest development version...  Which
I'm
> > >> doing
> > >> >>> >> anyway.
> > >> >>> >>
> > >> >>> >> And yes, what you just wrote is correct.
> > >> >>> >>
> > >> >>> >> - Julian
> > >> >>> >
> > >> >>
> > >> >
> > >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From julianokyap at gmail.com  Thu Aug 14 12:30:02 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 14 Aug 2008 00:30:02 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
	<d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>
Message-ID: <d66707280808140330i77592c8fk56da76125736da63@mail.gmail.com>

Rainer,

Yes, after looking at it some more, I guess I do need to have the
$ActionExecOnlyEveryNthTimeTimeout setting.  Your explanation below
clarifies things a lot.

I guess I was confused with my existing
$ActionExecOnlyOnceEveryInterval setting and thinking that was
similar.

That should fix my issue.

- Julian

On Wed, Aug 13, 2008 at 11:31 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just to keep everyone update. I got a private mail with some details not
> to be posted publically. However, Julian stated that he does not use
> $ActionExecOnlyEveryNthTimeTimeout.
>
> With that (and the non-public) info, I think that the missing
> $ActionExecOnlyEveryNthTimeTimeout is the culprit. I have not asked how
> often the receiving syslogd is restarted or huped. Julian, please let me
> know. If it is sufficiently seldom, I think it could be an "old"
> message.
>
> I think about this scenario: a message comes it at, eg., 10am. That's
> count 1. Then, nothing happens for the next 10 hours. At 8pm, the next
> one occurs. That's count 2. Another 5 hours later, the next message
> occurs, bringing the total count to 3. Thus, this message now triggers
> the rule.
>
> The question is if this is desired behavior? Or should the rule only be
> triggered if the messages occur within an e.g. 20 minute window? If the
> later is the case, you need a
>
> $ActionExecOnlyEveryNthTimeTimeout 1200
>
> This directive will timeout previous messages seen if they are older
> than 20 minutes. In the example above, the count would now be always 1
> and consequently no rule would ever be triggered (and that's the reason
> why this directive is present;)).
>
> Does this make sense? Does it match what you see?
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Wednesday, August 13, 2008 12:48 PM
>> To: Julian Yap
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> Do you use $ActionExecOnlyEveryNthTimeTimeout?
>>
>> Rainer
>>
>>
>> > -----Original Message-----
>> > From: Julian Yap [mailto:julianokyap at gmail.com]
>> > Sent: Wednesday, August 13, 2008 12:32 PM
>> > To: Rainer Gerhards
>> > Cc: rsyslog at lists.adiscon.com
>> > Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> found
>> >
>> > It's kind of strange but I've had this running for about a week
>> now...
>> >  I seem to have had 2 false alerts for no apparent reason.
>> >
>> >
>> > On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> > > Thanks for the feedback, it will now be part of the next devel
>> > release,
>> > > I think some time next week :)
>> > >
>> > > Rainer
>> > >
>> > >> -----Original Message-----
>> > >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> Sent: Friday, August 08, 2008 1:55 AM
>> > >> To: Rainer Gerhards
>> > >> Cc: rsyslog at lists.adiscon.com
>> > >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> > > found
>> > >>
>> > >> Yep, after further testing this works great!  Thanks Rainer.
>> > >>
>> > >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap
>> <julianokyap at gmail.com>
>> > >> wrote:
>> > >> > Rainer,
>> > >> >
>> > >> > Initial testing looks fine.  I'll try some more to see if I can
>> > > break
>> > >> it.
>> > >> >
>> > >> > - Julian
>> > >> >
>> > >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
>> > >> > <rgerhards at hq.adiscon.com> wrote:
>> > >> >> Julian,
>> > >> >>
>> > >> >> as you have probably seen in my other post, I have implemented
>> > the
>> > >> >> functionality. I have now also created a test tarball. I'd
>> > >> appreciate if
>> > >> >> you could obtain it from
>> > >> >>
>> > >> >>
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>> > >> >>
>> > >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> > >> >> $ActionExecOnlyEveryNthTime and
>> > $ActionExecOnlyEveryNthTimeTimeout.
>> > >> For
>> > >> >> what you intend to do, this should work:
>> > >> >>
>> > >> >> $ActionExecOnlyEveryNthTime 3
>> > >> >> *.* ..your action..
>> > >> >>
>> > >> >> You don't need the timeout, but I have included it for
>> > > completeness.
>> > >> >> Well, actually if I were you I'd think if you really don't
> need
>> > it.
>> > >> Is
>> > >> >> it really OK that "three in a row" means one each day?
>> > >> >>
>> > >> >> Please provide feedback on this feature.
>> > >> >>
>> > >> >> Thanks,
>> > >> >> Rainer
>> > >> >>
>> > >> >>> -----Original Message-----
>> > >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> >>> Sent: Friday, August 01, 2008 12:14 PM
>> > >> >>> To: Rainer Gerhards
>> > >> >>> Cc: rsyslog at lists.adiscon.com
>> > >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
>> > are
>> > >> >> found
>> > >> >>>
>> > >> >>> Roger that Rainer.
>> > >> >>>
>> > >> >>> Thanks,
>> > >> >>> Julian
>> > >> >>>
>> > >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> > >> >>> <rgerhards at hq.adiscon.com> wrote:
>> > >> >>> > OK, that greatly simplifies things. Actually, it now boils
>> > down
>> > >> to
>> > >> >>> > "execute an action only on the n-the time the filter
>> evaluates
>> > > to
>> > >> >>> true".
>> > >> >>> > I think this is quite easy to implement, but I must verify
>> > >> that...
>> > >> >>> >
>> > >> >>> > Rainer
>> > >> >>> >
>> > >> >>> >> -----Original Message-----
>> > >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
>> > >> >>> >> To: Rainer Gerhards
>> > >> >>> >> Cc: rsyslog at lists.adiscon.com
>> > >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
>> lines
>> > >> are
>> > >> >>> > found
>> > >> >>> >>
>> > >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> > >> >>> >> <rgerhards at hq.adiscon.com> wrote:
>> > >> >>> >> > Just one more re-confirmation:
>> > >> >>> >> >
>> > >> >>> >> >> What I'm looking for:
>> > >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a
> row
>> > > then
>> > >> >>> >> >> :ommail:;mailBody
>> > >> >>> >> >>
>> > >> >>> >> >> This would be nice but is not required since the 'This
>> is
>> > >> really
>> > >> >>> >> bad'
>> > >> >>> >> >> message in my case is very unique:
>> > >> >>> >> >> if ($msg == 'This is really bad' and $server ==
> 'server'
>> > and
>> > >> >>> >> $program
>> > >> >>> >> >> == 'program') happens 3 times in a row then
>> > > :ommail:;mailBody
>> > >> >>> >> >
>> > >> >>> >> > So you would actually use such a rule. If "this other
>> thing
>> > > is
>> > >> >>> > really
>> > >> >>> >> > bad" happened three times, the rule shall not trigger.
> Is
>> > > this
>> > >> >>> > right?
>> > >> >>> >>
>> > >> >>> >> Yes, I would use such a rule.  It would make what is
>> already
>> > an
>> > >> >>> >> awesome application even more awesome. :P  I am also
>> willing
>> > to
>> > >> >> test
>> > >> >>> >> it out and run the latest development version...  Which
> I'm
>> > >> doing
>> > >> >>> >> anyway.
>> > >> >>> >>
>> > >> >>> >> And yes, what you just wrote is correct.
>> > >> >>> >>
>> > >> >>> >> - Julian
>> > >> >>> >
>> > >> >>
>> > >> >
>> > >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From klas.lindfors at it.su.se  Fri Aug 15 09:37:14 2008
From: klas.lindfors at it.su.se (Klas Lindfors)
Date: Fri, 15 Aug 2008 09:37:14 +0200
Subject: [rsyslog] patch adding option for testing configuration
Message-ID: <48A5322A.9080400@it.su.se>

Hello,

I've attached a patch to this message adding a config option (-T) to run 
rsyslog with configuration testing, immediatly exiting with 0 if the 
conf is valid. For us this is an important feature to make sure a 
configuration is valid before starting the daemon.

The style of the patch is probably not acceptable for inclusion, but it 
does it's job, so maybe a variant of it can be included.

Regards,

Klas Lindfors
Stockholm University

From rgerhards at hq.adiscon.com  Fri Aug 15 11:37:15 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Aug 2008 11:37:15 +0200
Subject: [rsyslog] patch adding option for testing configuration
In-Reply-To: <48A5322A.9080400@it.su.se>
References: <48A5322A.9080400@it.su.se>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFF0@grfint2.intern.adiscon.com>

Hi Klas,

the patch did not make it to the mailing list (the list processor is
very picky to prevent spreading spam and malware). However, from you
description it sounds like the -N option which has recently (3.21.1)
been introduced. Please see

http://www.rsyslog.com/doc-troubleshoot.html

under "configuration problems". I don't know out of my head if the exit
code works in the way you describe, but that should be an easy add. I
suggest you file an enhancement request if in doubt
(http://bugzilla.adiscon.com). I'll be soon away for summer break, so
that needs to wait until after it ;)

Please let me know if -N goes into the right direction.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Klas Lindfors
> Sent: Friday, August 15, 2008 9:37 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] patch adding option for testing configuration
> 
> Hello,
> 
> I've attached a patch to this message adding a config option (-T) to
> run
> rsyslog with configuration testing, immediatly exiting with 0 if the
> conf is valid. For us this is an important feature to make sure a
> configuration is valid before starting the daemon.
> 
> The style of the patch is probably not acceptable for inclusion, but
it
> does it's job, so maybe a variant of it can be included.
> 
> Regards,
> 
> Klas Lindfors
> Stockholm University


From friedl at hq.adiscon.com  Fri Aug 15 14:07:34 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 15 Aug 2008 14:07:34 +0200
Subject: [rsyslog] rsyslog 3.21.3 (devel) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFFF@grfint2.intern.adiscon.com>

Hi all,

I have just released 3.21.3, a devel branch version. It provides support
for specifying custom MySQL server ports and adds the ability to execute
an action only ever n-th time. Also, flow control for imuxsock messages
can now be turned on or off. There are also some bug fixes, among others
one to fully support IPv6 hosts in forwarding actions. This is a
recommended update for all devel branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-127.phtml

Changelog:

http://www.rsyslog.com/Article270.phtml

As always, feedback is appreciated.

Florian Riedl


From friedl at hq.adiscon.com  Mon Aug 18 17:16:49 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 18 Aug 2008 17:16:49 +0200
Subject: [rsyslog] rsyslog 3.18.3 released (stable)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F006@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 3.18.3, a member of the v3-stable branch,
today. This is primarily a bug-fixing release. It fixes a problem that
causes imfile to segfault on termination, disables flow control in
imuxsock which could lead to system hang conditions and corrects a
misspelled configuration directive. There is a small enhancement which
now permits to use MySQL custom server ports. This is a recommended
update for all v3-stable branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-128.phtml

Changelog:

http://www.rsyslog.com/Article271.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From stefan.wold at it.su.se  Wed Aug 20 07:40:19 2008
From: stefan.wold at it.su.se (Stefan Wold)
Date: Wed, 20 Aug 2008 07:40:19 +0200
Subject: [rsyslog] rsyslog with RELP resolving issue
Message-ID: <1219210819.4967.18.camel@localhost>

Hi,

I've been testing rsyslog versions 3.18.2 and 3.21.3 with RELP. My
problem here is that no matter what I try I can't get the receiving
rsyslog to do hostname resolving. If I disable RELP and go back to
either tcp or udp it will resolve my hosts properly. On the receiving
side I have a template that create one log file per host with the
%HOSTNAME% variable, if I try using %fromhost% and RELP I end up with an
unresolved ip-address.

Is there a hidden option I'm not aware of or is resolving just not
working properly with RELP yet?

Secondly I wonder if adding a option that force rsyslog to use FQDN for
everything including hostnames within the same domain as the logserver
would be plausible? 

-- 
Sincerely,
Stefan Wold
IT Services, Stockholm University, Sweden



From satoru.satoh at gmail.com  Wed Aug 20 17:31:19 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 00:31:19 +0900
Subject: [rsyslog] [PATCH 00/07] man-i18n patches
Message-ID: <20080820153118.GC20566@localhost.localdomain>

Hello / Guten tag,


Rio-san (rfujita at redhat.com) previously told about his idea of
documentation i18n / l10n for rsyslog. I'm one of his colleagues and
originally told him about related topics.


A series of patches I'll post right after this mail are to implement
this i18n / l10n workflow for man pages into normal build process.

These patches enable to 

 * generate original English man pages from DocBook XML files
 * generate translated non-English man pages from translated DocBook
   XML files, which can be generated from PO files (PO is much suitable
   format for translation maintainace than XML, I guess)
 * create POT (PO template)

And this automated man-pages-generation process is only performed when
the configure option "--enable-regenerate-man" is specified. So, side
effects of these patches should be kept to a minimum, I think.

- satoru


From satoru.satoh at gmail.com  Wed Aug 20 19:07:40 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 02:07:40 +0900
Subject: [rsyslog] [PATCH] man-i18n patches summary
Message-ID: <20080820170735.GB20932@localhost.localdomain>

Hello,


I posted the first patch but it seems the patch was too big to post. I
list them here at once.

Could you please take some time to review these and merge them?


I splitted origianl big patch into minimum units. However, to complete
to man-i18n support, the following patches must be applied in this
order.

All of the patches below are available from
http://www.gnome.gr.jp/~ss/t/rsyslog/.


1. 0001-man-i18n-move-man-files-from-tools-to-man-C.patch

This patch just moves man files from tools/ to man/C/ (new dir to hold
man related stuff) and update related Makefile.am files.


2. 0002-man-i18n-add-man-ja.patch

This patch creates new dir, man/ja/ and Makefile.am in it.

There are no man files in it and this should not have harmful effects
for build process at all, I guess.


3. 0003-man-i18n-add-man-ja.patch

This patch adds man files in man/ja and update Makefile.am in it to
install these.


4. 0004-man-i18n-add-docbook-xml-sources-for-man-files.patch

This patch adds docbook XML sources for man pages.


5. 0005-man-i18n-add-m4-macros-in-m4-and-make-aclocal-look.patch

This patch adds m4 macros in m4/ and make aclocal look for this dir.

These m4 files are not evaluated in actual and this should not have
adverse effect at all, I guess.


6. 0006-man-i18n-add-build-process-of-man-pages-in-Makefile.patch

This patch adds transformation process of man pages and po files in
Makefile.am.

Added part are in "if ENABLE_REGENERATE_MAN ... endif" and
ENABLE_REGENERATE_MAN is not defined at this point, so this also should
not have side effects.


7. 0007-man-i18n-add-enable-regenerate-man-option-for-con.patch

This is the last one. This patch adds "--enable-regenerate-man"
configure option to enable man build process.

This option is disabled by default and should not affect normal build
process, I think.


How to build:

To enable the man build process, configure have to be regenerated:

  $ (autoreconf -vfi && \
        ./configure --enable-regenerate-man [...other options] && \
        make && make install


Translator's workflow:

 a. initial translation:

   0. arrange lang dir, that is, cp man/ja to man/YOUR_LANG/ and edit
      Makefile.am; s/ja/YOUR_LANG/.
   1. create POT: make -C man/YOUR_LANG/ create-pot
   2. copy POT to PO and translate it
   3. run make

 b. maintanance:

   1. update PO: make -C man/YOUR_LANG update-po
   2. update translations
   3. just run make


Requirements for man regeneration / translation:
 
   xmllint, xsltproc, xml2po in gnome-doc-utils, DocBook DTD and DocBook
   xslt stylesheet for man pages are required.

   If xmlcatalog (e.g. /etc/xml/catalog) is configured appropriately,
   both xmllint and xsltproc will use local DTD and stylsheet files
   instead of these from remote site
   (http://www.oasis-open.org/docbook/...).


- satoru


From satoru.satoh at gmail.com  Wed Aug 20 19:25:24 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 02:25:24 +0900
Subject: [rsyslog] [PATCH] man-i18n patches summary
In-Reply-To: <20080820170735.GB20932@localhost.localdomain>
References: <20080820170735.GB20932@localhost.localdomain>
Message-ID: <20080820172520.GC20932@localhost.localdomain>

Hello,


On Thu, Aug 21, 2008 at 02:07:40AM +0900, Satoru SATOH wrote:
> Could you please take some time to review these and merge them?

Of course, my patches change many and don't think easy to merge nor am
not sure accetable enhancement in the first place.

Please let me know if you have any issues on them, I'll work to fix
them as far as I can.

- satoru


From rgerhards at hq.adiscon.com  Wed Aug 20 21:25:42 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Aug 2008 21:25:42 +0200
Subject: [rsyslog] [PATCH 00/07] man-i18n patches
In-Reply-To: <20080820153118.GC20566@localhost.localdomain>
References: <20080820153118.GC20566@localhost.localdomain>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F018@grfint2.intern.adiscon.com>

Hi Satoru-san,

Your contribution is greatly appreciated. But I am currently on summer
vacation end will return end of month. Then I'll have a look at the
patch :)

Thanks,
Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Satoru SATOH
> Sent: Wednesday, August 20, 2008 5:31 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] [PATCH 00/07] man-i18n patches
> 
> Hello / Guten tag,
> 
> 
> Rio-san (rfujita at redhat.com) previously told about his idea of
> documentation i18n / l10n for rsyslog. I'm one of his colleagues and
> originally told him about related topics.
> 
> 
> A series of patches I'll post right after this mail are to implement
> this i18n / l10n workflow for man pages into normal build process.
> 
> These patches enable to 
> 
>  * generate original English man pages from DocBook XML files
>  * generate translated non-English man pages from translated DocBook
>    XML files, which can be generated from PO files (PO is 
> much suitable
>    format for translation maintainace than XML, I guess)
>  * create POT (PO template)
> 
> And this automated man-pages-generation process is only performed when
> the configure option "--enable-regenerate-man" is specified. So, side
> effects of these patches should be kept to a minimum, I think.
> 
> - satoru
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> 


From jeffschroed at gmail.com  Thu Aug 21 19:33:51 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Thu, 21 Aug 2008 10:33:51 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
Message-ID: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>

Rsyslog seems to be mangling messages sent from our in-house applications.
We are trying to get the same format as we did with syslog-ng previously and
not having a lot of luck.

The logs are in the format something like this:
TAG APPLICATION: MSG

In syslog-ng, it looks like this locally and when sent to a remote
syslog-ng server:
Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
ops-192.168.101.251:65261 offset changed 0.0004593 seconds!

rsyslog with the TraditionalFileFormat makes it look like this
locally, stripping the hostname:
Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
0.0004593 seconds!

and like this when sent to a remote syslog-ng server. Notice how the
ADMIN tag was removed:
Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
offset changed 0.0004593 seconds!

We have realtime logscraping software that looks at all message tags
and does custom reporting on them.
It keys off the TAG property that rsyslog is stripping off. What can
we do to enable this? We've been playing
with custom $templates in the rsyslog.conf to no real avail.

Thanks!

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From hks.private at gmail.com  Thu Aug 21 19:46:43 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 21 Aug 2008 13:46:43 -0400
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
Message-ID: <ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>

I'm not familiar with syslog-ng, but I suspect this is just a
difference in the formatting/interpretation of the raw packet. Try
capturing a message or two from syslog-ng with tcpdump and compare to
section 4 of RFC 3164.

You can see how rsyslog interprets it with the following template:
$template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
PROGRAM: %programname% MSG: %rawmsg%\n"

This should give you some idea of how to create a template that will
log in the format you need.

-HKS



On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> Rsyslog seems to be mangling messages sent from our in-house applications.
> We are trying to get the same format as we did with syslog-ng previously and
> not having a lot of luck.
>
> The logs are in the format something like this:
> TAG APPLICATION: MSG
>
> In syslog-ng, it looks like this locally and when sent to a remote
> syslog-ng server:
> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>
> rsyslog with the TraditionalFileFormat makes it look like this
> locally, stripping the hostname:
> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
> 0.0004593 seconds!
>
> and like this when sent to a remote syslog-ng server. Notice how the
> ADMIN tag was removed:
> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
> offset changed 0.0004593 seconds!
>
> We have realtime logscraping software that looks at all message tags
> and does custom reporting on them.
> It keys off the TAG property that rsyslog is stripping off. What can
> we do to enable this? We've been playing
> with custom $templates in the rsyslog.conf to no real avail.
>
> Thanks!
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From rgerhards at hq.adiscon.com  Thu Aug 21 19:53:39 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 21 Aug 2008 19:53:39 +0200
Subject: [rsyslog] Problems migrating from syslog-ng
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>

I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

Rainer


-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com on behalf of (private) HKS
Sent: Thu 8/21/2008 7:46 PM
To: rsyslog-users
Subject: Re: [rsyslog] Problems migrating from syslog-ng
 
I'm not familiar with syslog-ng, but I suspect this is just a
difference in the formatting/interpretation of the raw packet. Try
capturing a message or two from syslog-ng with tcpdump and compare to
section 4 of RFC 3164.

You can see how rsyslog interprets it with the following template:
$template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
PROGRAM: %programname% MSG: %rawmsg%\n"

This should give you some idea of how to create a template that will
log in the format you need.

-HKS



On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> Rsyslog seems to be mangling messages sent from our in-house applications.
> We are trying to get the same format as we did with syslog-ng previously and
> not having a lot of luck.
>
> The logs are in the format something like this:
> TAG APPLICATION: MSG
>
> In syslog-ng, it looks like this locally and when sent to a remote
> syslog-ng server:
> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>
> rsyslog with the TraditionalFileFormat makes it look like this
> locally, stripping the hostname:
> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
> 0.0004593 seconds!
>
> and like this when sent to a remote syslog-ng server. Notice how the
> ADMIN tag was removed:
> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
> offset changed 0.0004593 seconds!
>
> We have realtime logscraping software that looks at all message tags
> and does custom reporting on them.
> It keys off the TAG property that rsyslog is stripping off. What can
> we do to enable this? We've been playing
> with custom $templates in the rsyslog.conf to no real avail.
>
> Thanks!
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From jeffschroed at gmail.com  Thu Aug 21 22:44:16 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Thu, 21 Aug 2008 13:44:16 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
Message-ID: <c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>

On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
work with -c3 and I'd
rather not run in compatibility mode.

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com on behalf of (private) HKS
> Sent: Thu 8/21/2008 7:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems migrating from syslog-ng
>
> I'm not familiar with syslog-ng, but I suspect this is just a
> difference in the formatting/interpretation of the raw packet. Try
> capturing a message or two from syslog-ng with tcpdump and compare to
> section 4 of RFC 3164.
>
> You can see how rsyslog interprets it with the following template:
> $template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
> PROGRAM: %programname% MSG: %rawmsg%\n"
>
> This should give you some idea of how to create a template that will
> log in the format you need.
>
> -HKS
>
>
>
> On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
>> Rsyslog seems to be mangling messages sent from our in-house applications.
>> We are trying to get the same format as we did with syslog-ng previously and
>> not having a lot of luck.
>>
>> The logs are in the format something like this:
>> TAG APPLICATION: MSG
>>
>> In syslog-ng, it looks like this locally and when sent to a remote
>> syslog-ng server:
>> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
>> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>>
>> rsyslog with the TraditionalFileFormat makes it look like this
>> locally, stripping the hostname:
>> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
>> 0.0004593 seconds!
>>
>> and like this when sent to a remote syslog-ng server. Notice how the
>> ADMIN tag was removed:
>> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
>> offset changed 0.0004593 seconds!
>>
>> We have realtime logscraping software that looks at all message tags
>> and does custom reporting on them.
>> It keys off the TAG property that rsyslog is stripping off. What can
>> we do to enable this? We've been playing
>> with custom $templates in the rsyslog.conf to no real avail.
>>
>> Thanks!

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From hks.private at gmail.com  Thu Aug 21 23:00:35 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 21 Aug 2008 17:00:35 -0400
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
	<c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
Message-ID: <ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>

On Thu, Aug 21, 2008 at 4:44 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.
>
> Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
> work with -c3 and I'd
> rather not run in compatibility mode.


I don't think so.

-HKS


From jeffschroed at gmail.com  Fri Aug 22 15:21:01 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Fri, 22 Aug 2008 06:21:01 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
	<c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
	<ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>
Message-ID: <c24352ca0808220621s5419322fi3f0820e1dea79da9@mail.gmail.com>

On Thu, Aug 21, 2008 at 2:00 PM, (private) HKS <hks.private at gmail.com> wrote:
> On Thu, Aug 21, 2008 at 4:44 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
>> On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>>> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

You were right Rainer. It looks like the java code which injects the
message is sending malformed
syslog requests. syslog-ng still sends it through and does the correct
things. Is there a way to make
rsyslog a bit less strict about it? Running rsyslog with -c0 defeats
the purpose of using rsyslog.

Until our application has been fixed and rolled out accross our
clusters worldwide, we rolled back to syslog-ng.


>>
>> Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
>> work with -c3 and I'd
>> rather not run in compatibility mode.
>
>
> I don't think so.
>
> -HKS
>



-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From aoz.syn at gmail.com  Sat Aug 23 09:45:41 2008
From: aoz.syn at gmail.com (RB)
Date: Sat, 23 Aug 2008 01:45:41 -0600
Subject: [rsyslog] Running hashes
Message-ID: <4255c2570808230045kb625e11u795435a8a6c8d457@mail.gmail.com>

Acting on an old comment from a friend, I decided to look into what
it'd take to get rsyslog to perform running hashes of logs.
Conceptually, it's pretty simple - every Nth message inject one
message containing the hash of the previous N messages (including the
previous hash message).  It also gave me an excuse to start digging
into the rsyslog code.

At first I thought I could do it with a property replacer, but that
seems a wash since those are wholly message-based and don't [seem to]
give the opportunity to store information (even a running hash) of
prior messages.  A plugin was my next hope, but there doesn't seem to
be a good mechanism to pipeline those together - AFAICT they're
expected to be single ingress/egress points, with no interstitial
stages.  I see the code for loading other objects as Rainer mentioned
in April, but that seems more for central functionality than for
chaining modules together.

This all brings me back to one of my original questions for rsyslog -
is module chaining something that is even on your radar?  I'm thinking
normalization, hashing, encryption, etc.  Almost feels like there
should be another layer here, maybe a "mangle" plugin interface that
could stack in after im* and before om*?


RB


From aoz.syn at gmail.com  Sun Aug 24 08:01:38 2008
From: aoz.syn at gmail.com (RB)
Date: Sun, 24 Aug 2008 00:01:38 -0600
Subject: [rsyslog] broken size_nbr handling
Message-ID: <4255c2570808232301n5bab0c30w549df07c8e5b6c1f@mail.gmail.com>

In implementing my own 'human-readable' size parser (would be nice to
have one as a part of one of the usable objects), I noticed there
seemed to be order-of-magnitude issues on the base-10 size definitions
in cfsysline.c.  Although the base-2 operations properly increase 2^10
(1024) for each SI prefix, the base-10 ones only increase by one order
of magnitude (10) instead of the proper 10^3 per prefix, ending up
with EB only being 10^8 instead of 10^18.

Following is a simple patch that should put things right:
======================================
diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c
index 18643ba..1989247 100644
--- a/runtime/cfsysline.c
+++ b/runtime/cfsysline.c
@@ -215,11 +215,11 @@ static rsRetVal doGetSize(uchar **pp, rsRetVal
(*pSetHdlr)(void*, uid_t), void *
                case 'e': i *= (int64) 1024 * 1024 * 1024 * 1024 *
1024 * 1024; ++(*pp); break; /* exa */
                /* and now the "new" 1000-based definitions */
                case 'K': i *= 1000; ++(*pp); break;
-               case 'M': i *= 10000; ++(*pp); break;
-               case 'G': i *= 100000; ++(*pp); break;
-               case 'T': i *= 1000000; ++(*pp); break; /* tera */
-               case 'P': i *= 10000000; ++(*pp); break; /* peta */
-               case 'E': i *= 100000000; ++(*pp); break; /* exa */
+               case 'M': i *= 1000000; ++(*pp); break;
+               case 'G': i *= 1000000000; ++(*pp); break;
+               case 'T': i *= 1000000000000; ++(*pp); break; /* tera */
+               case 'P': i *= 1000000000000000; ++(*pp); break; /* peta */
+               case 'E': i *= 1000000000000000000; ++(*pp); break; /* exa */
        }

        /* done */
======================================


From friedl at hq.adiscon.com  Mon Aug 25 17:47:31 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 25 Aug 2008 17:47:31 +0200
Subject: [rsyslog] rsyslog 3.19.11 released (beta)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F039@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 3.19.11, a member of the beta branch, today.
This is a refresh of the current beta, bringing it up to date. It
includes a number of bugfixes from the v3-stable branch. It is a
recommended update for all beta branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-129.phtml

Changelog:

http://www.rsyslog.com/Article273.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .




From mattjhell at gmail.com  Tue Aug 26 00:07:37 2008
From: mattjhell at gmail.com (Matt Hellman)
Date: Mon, 25 Aug 2008 17:07:37 -0500
Subject: [rsyslog] gnutls support in 3.18.3?
Message-ID: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>

I don't see the --enable-gnutls option in 3.18.3.  Is this only in dev
branch currently?


From mbiebl at gmail.com  Tue Aug 26 00:39:07 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 26 Aug 2008 00:39:07 +0200
Subject: [rsyslog] gnutls support in 3.18.3?
In-Reply-To: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>
References: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>
Message-ID: <aa76cdac0808251539o5f5508e9qf45c13af2edfcf83@mail.gmail.com>

2008/8/26 Matt Hellman <mattjhell at gmail.com>:
> I don't see the --enable-gnutls option in 3.18.3.  Is this only in dev
> branch currently?

That is correct.
The TLS support is not available in the v3-stable branch.
You should use the beta branch (latest released version is 3.19.11) or
the master aka devel branch (latest released version is 3.21.2)

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From danson at rackspace.com  Wed Aug 27 23:18:40 2008
From: danson at rackspace.com (Daniel Anson)
Date: Wed, 27 Aug 2008 16:18:40 -0500
Subject: [rsyslog] test
Message-ID: <19205_1219871984_m7RLJaDk013323_96AF20FDF4301D419B33CCE8E3A0132B08A296C2@SAT4MX07.RACKSPACE.CORP>

Test.  New to group.

Daniel M. Anson
Linux Systems Engineer
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From rory at ooma.com  Thu Aug 28 00:20:16 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 27 Aug 2008 15:20:16 -0700
Subject: [rsyslog] phpLogCon bug?
Message-ID: <48B5D320.80405@ooma.com>

It seems that if I execute a query, and then close the browser window 
(safari and firefox), the query remains running on the sql server and 
only goes away after it times out, degrading performance.

I'm using 2.5.3 of phpLogCon


From alorbach at ro1.adiscon.com  Thu Aug 28 09:50:39 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 28 Aug 2008 09:50:39 +0200
Subject: [rsyslog] phpLogCon bug?
In-Reply-To: <48B5D320.80405@ooma.com>
References: <48B5D320.80405@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D9D@rmail.adiscon.com>

Hi,

I think this behavior comes from the design of php. Closing the Browser
Window tells the webserver to quit the current connection, but if the
php processor hangs within a working sql query, he has to wait until
this query is done. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 28, 2008 12:20 AM
> To: rsyslog-users
> Subject: [rsyslog] phpLogCon bug?
> 
> It seems that if I execute a query, and then close the browser window
> (safari and firefox), the query remains running on the sql server and
> only goes away after it times out, degrading performance.
> 
> I'm using 2.5.3 of phpLogCon
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug 28 10:20:46 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Aug 2008 10:20:46 +0200
Subject: [rsyslog] phpLogCon bug?
Message-ID: <000c01c908e7$0d9b6f72$060013ac@intern.adiscon.com>

I agree with andre, thats http behaviour. The srver doesnt even know the client is dead until it tries to send data (which it can only after completing the query).
Hth
rainer

----- Urspr?ngliche Nachricht -----
Von: "Andre Lorbach" <alorbach at ro1.adiscon.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 28.08.08 09:50
Betreff: Re: [rsyslog] phpLogCon bug?

Hi,

I think this behavior comes from the design of php. Closing the Browser
Window tells the webserver to quit the current connection, but if the
php processor hangs within a working sql query, he has to wait until
this query is done. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 28, 2008 12:20 AM
> To: rsyslog-users
> Subject: [rsyslog] phpLogCon bug?
> 
> It seems that if I execute a query, and then close the browser window
> (safari and firefox), the query remains running on the sql server and
> only goes away after it times out, degrading performance.
> 
> I'm using 2.5.3 of phpLogCon
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From danson at rackspace.com  Thu Aug 28 16:54:59 2008
From: danson at rackspace.com (Daniel Anson)
Date: Thu, 28 Aug 2008 09:54:59 -0500
Subject: [rsyslog] Memory vanishes
Message-ID: <28252_1219935370_m7SEu5H1031505_96AF20FDF4301D419B33CCE8E3A0132B08A29886@SAT4MX07.RACKSPACE.CORP>

For a bit of background, I have a RHEL5 server acting as an aggregate
point to write rsyslog/syslog info to:
rsyslog.i386                             2.0.0-11.el5
installed       
rsyslog-mysql.i386                       2.0.0-11.el5
installed

I am manipulating this data received with a threaded C program.  So, in
the spirit of testing, I wrote this small code to produce logs so I can
test how many messages that can be processed.  Here is the code for that
program:

#include <syslog.h>

int
main(int argc, char **argv) {
  int num_syslogs = atoi(argv[1]), i;
  openlog("syslog_writer", LOG_CONS | LOG_PID, LOG_USER);
  for(i=0; i < num_syslogs; i++) {
    syslog(LOG_INFO, "syslog_writer: log number %d", i);
  }
  closelog();
  return 1;
}

I run it thusly:
./syslogwriter 300000

After the running of this program, I find that ~30MB of memory just
disappear and never reappear.  I understand that MySQL and other
programs can hold some memory but do end up releasing it.  This is not
the case, this memory is not freed.  I know that I am not using the most
current version but I am using a STABLE release.  Has anyone seen this
behavior before or have an idea why this is happening.  I am at a loss.
Have I merely reached a limitation point in the number of consecutive
logs that can be handled?

Thx,

Daniel M. Anson
Linux Systems Engineer III
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From rgerhards at hq.adiscon.com  Thu Aug 28 21:47:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Aug 2008 21:47:56 +0200
Subject: [rsyslog] Memory vanishes
Message-ID: <000e01c90947$0ce8c017$060013ac@intern.adiscon.com>

2.0.0 is quite old. I think there are some mem leak fixes in the more recent v2-stable builds (see changelog). I suggest updating (from source) to the current v2-stable (2.0.6 or so, not sure out of my head).

Rainer

----- Urspr?ngliche Nachricht -----
Von: "Daniel Anson" <danson at rackspace.com>
An: "rsyslog at lists.adiscon.com" <rsyslog at lists.adiscon.com>
Gesendet: 28.08.08 16:56
Betreff: [rsyslog] Memory vanishes

For a bit of background, I have a RHEL5 server acting as an aggregate
point to write rsyslog/syslog info to:
rsyslog.i386                             2.0.0-11.el5
installed       
rsyslog-mysql.i386                       2.0.0-11.el5
installed

I am manipulating this data received with a threaded C program.  So, in
the spirit of testing, I wrote this small code to produce logs so I can
test how many messages that can be processed.  Here is the code for that
program:

#include <syslog.h>

int
main(int argc, char **argv) {
  int num_syslogs = atoi(argv[1]), i;
  openlog("syslog_writer", LOG_CONS | LOG_PID, LOG_USER);
  for(i=0; i < num_syslogs; i++) {
    syslog(LOG_INFO, "syslog_writer: log number %d", i);
  }
  closelog();
  return 1;
}

I run it thusly:
./syslogwriter 300000

After the running of this program, I find that ~30MB of memory just
disappear and never reappear.  I understand that MySQL and other
programs can hold some memory but do end up releasing it.  This is not
the case, this memory is not freed.  I know that I am not using the most
current version but I am using a STABLE release.  Has anyone seen this
behavior before or have an idea why this is happening.  I am at a loss.
Have I merely reached a limitation point in the number of consecutive
logs that can be handled?

Thx,

Daniel M. Anson
Linux Systems Engineer III
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From julianokyap at gmail.com  Fri Aug  1 01:50:22 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 13:50:22 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
Message-ID: <d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>

Yes, this would be correct.

In my example, a particular message string would be the same.

Example:
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad

So then is could be $msg == 'This is really bad' or perhaps $msg
contains 'really bad'.  But for me, matching the exact $msg would be
fine. :P


On Thu, Jul 31, 2008 at 10:38 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence:
>
> 1. a - a - a - b
> 2. a - a - b - a
>
> Result: in case 1 an alert is triggered, in case 2 not.
>
> Is this understanding correct?
>
> rainer
>
> ----- Urspr?ngliche Nachricht -----
> Von: "Julian Yap" <julianokyap at gmail.com>
> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> Cc: "rgerhards at hq.adiscon.com" <rgerhards at hq.adiscon.com>; "hks.private at gmail.com" <hks.private at gmail.com>
> Gesendet: 31.07.08 21:59
> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
>
> That's pretty much it for now.  I've written Alerts for single line
> events.  But for one particular event, it's only really a factor if it
> happens tree times in a row.
>
>
> On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> What exactly do you need to do except the "three in a row" alert?
>>
>> ----- Urspr?ngliche Nachricht -----
>> Von: "Julian Yap" <julianokyap at gmail.com>
>> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
>> Gesendet: 31.07.08 20:27
>> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> Hmm, Nagios is a pain to set up.  Looking for something more light
>> weight...  Was hoping that I could have consolidated lots of Alerts
>> under Rsyslog.
>>
>> Any other suggestions besides Swatch?
>>
>>
>>
>> On 7/31/08, (private) HKS <hks.private at gmail.com> wrote:
>>> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or
>>> some other monitoring-type software.
>>>
>>> -HKS
>>>
>>> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap <julianokyap at gmail.com> wrote:
>>>> Is there a way to set an Alert when multiple repeated lines are found in a
>>>> log?
>>>>
>>>> I want to spawn an email Alert if a message is received 3 times.
>>>>
>>>> Example log lines:
>>>> Jul 30 04:19:29 localhost program: Error detected
>>>> Jul 30 05:19:29 localhost program: Error detected
>>>> Jul 30 06:19:29 localhost program: Error detected
>>>>
>>>> Thanks,
>>>> Julian
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>
>


From rgerhards at hq.adiscon.com  Fri Aug  1 07:45:15 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 07:45:15 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>

Just to make sure:

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:04 server program: This is really bad

[Note the last timestamp!] would still make up for "three in a row"?

Now, syslog contains not only timestamps, but also hostnames. So how about

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server2 program: This is really bad
Jul 31 13:45:03 server program: This is really bad

And what about this:

Jul 31 13:45:03 server program2: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad

And would that trigger any alert at all:

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server2 program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program2: This is really bad
Jul 31 13:45:03 server program: This is really bad

Finally, does "three in a row" time out? So what would happen in the case below. Watch the timestamps and let's assume there are no other records inside the log:

Jul 29 13:45:03 server program: This is really bad
Jul 30 13:45:03 server program: This is really bad
Jul 31 13:45:04 server program: This is really bad

And a related question. You write:

> So then is could be $msg == 'This is really bad' or perhaps $msg
> contains 'really bad'.  But for me, matching the exact $msg would be
> fine. :P

Does this imply you would like to do the "if $msg == 'this is really bad'" check? So far, I assume you do NOT do this but expect an alert to be raised whenever ANY messages fulfills the identity criterion n times in a row. Please explain.

Thanks,
Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 1:50 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Yes, this would be correct.
> 
> In my example, a particular message string would be the same.
> 
> Example:
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> 
> So then is could be $msg == 'This is really bad' or perhaps $msg
> contains 'really bad'.  But for me, matching the exact $msg would be
> fine. :P
> 
> 
> On Thu, Jul 31, 2008 at 10:38 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > To clarify: be "a" the event in question and "b" any other event. Two
> samples of an event sequence:
> >
> > 1. a - a - a - b
> > 2. a - a - b - a
> >
> > Result: in case 1 an alert is triggered, in case 2 not.
> >
> > Is this understanding correct?
> >
> > rainer
> >
> > ----- Urspr?ngliche Nachricht -----
> > Von: "Julian Yap" <julianokyap at gmail.com>
> > An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> > Cc: "rgerhards at hq.adiscon.com" <rgerhards at hq.adiscon.com>;
> "hks.private at gmail.com" <hks.private at gmail.com>
> > Gesendet: 31.07.08 21:59
> > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
> >
> > That's pretty much it for now.  I've written Alerts for single line
> > events.  But for one particular event, it's only really a factor if
> it
> > happens tree times in a row.
> >
> >
> > On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> What exactly do you need to do except the "three in a row" alert?
> >>
> >> ----- Urspr?ngliche Nachricht -----
> >> Von: "Julian Yap" <julianokyap at gmail.com>
> >> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> >> Gesendet: 31.07.08 20:27
> >> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
> >>
> >> Hmm, Nagios is a pain to set up.  Looking for something more light
> >> weight...  Was hoping that I could have consolidated lots of Alerts
> >> under Rsyslog.
> >>
> >> Any other suggestions besides Swatch?
> >>
> >>
> >>
> >> On 7/31/08, (private) HKS <hks.private at gmail.com> wrote:
> >>> Not in rsyslogd itself, but you could do this with Swatch, Nagios,
> or
> >>> some other monitoring-type software.
> >>>
> >>> -HKS
> >>>
> >>> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap <julianokyap at gmail.com>
> wrote:
> >>>> Is there a way to set an Alert when multiple repeated lines are
> found in a
> >>>> log?
> >>>>
> >>>> I want to spawn an email Alert if a message is received 3 times.
> >>>>
> >>>> Example log lines:
> >>>> Jul 30 04:19:29 localhost program: Error detected
> >>>> Jul 30 05:19:29 localhost program: Error detected
> >>>> Jul 30 06:19:29 localhost program: Error detected
> >>>>
> >>>> Thanks,
> >>>> Julian
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>
> >


From rgerhards at hq.adiscon.com  Fri Aug  1 07:58:10 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 07:58:10 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>

Sorry for being brief, I was on the road and needed to write from a cell phone :(

It also looks like I was wrong ;) I have reviewed the case. If I understand correctly, you get you data from the different unix sockets and then forward via a single rule. So you can not use different templates (which most often are the real solution to any such problem, but are not obvious to most people not directly involved in rsyslog ;)).

Is my understanding correct now?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, July 31, 2008 5:04 PM
> To: David Darville; rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> Use a template with fixed name.
> 
> --- Urspr?ngliche Nachricht ---
> Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> Betreff: Re: [rsyslog] Changing hostname field
> Datum: 31. Juli 2008
> Uhrzeit: 16:46:59
> 
> The jails all have their own unique hostname (and IP), but all share an
> rsyslogd instance running on the main host, and the %hostname% and
> %fromhost% in all the log messages from the jails are set to the
> hostname of
> the main host. And that is what I want to change.
> 
> On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > Do the jails all share the same hostname and IP? If not, you should
> be
> > able to use the %hostname% or %fromhost% properties.
> >
> > If so, are they each running their own instance of (r)syslogd?
> >
> > -HKS
> >
> > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > <ml at darville.vm.bytemark.co.uk> wrote:
> > > Hello everyone
> > >
> > > I am trying to configure rsyslog to service a number of chroot
> jails in
> > > addition to the host itself.
> > >
> > > But I need to change the hostname field of the syslog messages from
> the
> > > different jails, so that I place them in the right log file on the
> central
> > > logging host.
> > >
> > > My current rsyslog.conf is as follows:
> > >
> > > $ModLoad imuxsock
> > > $ModLoad imklog
> > > $ModLoad immark
> > > $ModLoad omrelp
> > >
> > > $AddUnixListenSocket /jail/1/dev/log
> > > $AddUnixListenSocket /jail/2/dev/log
> > >
> > > *.* :omrelp:10.0.0.4:2514
> > >
> > >
> > > Can anyone please advice me on how to do that?
> > >
> > >
> > > ---
> > >
> > > David Darville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From julianokyap at gmail.com  Fri Aug  1 09:47:02 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 21:47:02 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>

Hi Rainer,

Thanks for taking the time to work on my issue.  I'll approach this
from my situation.

On Thu, Jul 31, 2008 at 7:45 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just to make sure:
>
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:04 server program: This is really bad
>
> [Note the last timestamp!] would still make up for "three in a row"?

Yes.  Something that would be nice would be 'three in a row in the
last x minutes'...  But let's keep things simple for now :)

> Now, syslog contains not only timestamps, but also hostnames. So how about
>
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server2 program: This is really bad
> Jul 31 13:45:03 server program: This is really bad

In my particular case, it would only come from the one server.  But if
I had 2 servers logging to the same log file, it should have the
option to filter based on server and/or message.

> And what about this:
>
> Jul 31 13:45:03 server program2: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
>
> And would that trigger any alert at all:

In my particular case, it would only come from the one program.  But
if I had 2 programs logging to the same log file, it should have the
option to filter based on program and/or message.

> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server2 program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program2: This is really bad
> Jul 31 13:45:03 server program: This is really bad
>
> Finally, does "three in a row" time out? So what would happen in the case below. Watch the timestamps and let's assume there are no other records inside the log:

In my particular case, 'three in a row' wouldn't time out.  I would
use ActionMail so I would use a corresponding
$ActionExecOnlyOnceEveryInterval value.

For my particular case, it happens when a process totally locks up.  A
very rare instance which requires manually restarting a process...  So
the 'This is really bad' messages would eventually stop.

> Jul 29 13:45:03 server program: This is really bad
> Jul 30 13:45:03 server program: This is really bad
> Jul 31 13:45:04 server program: This is really bad
>
> And a related question. You write:
>
>> So then is could be $msg == 'This is really bad' or perhaps $msg
>> contains 'really bad'.  But for me, matching the exact $msg would be
>> fine. :P
>
> Does this imply you would like to do the "if $msg == 'this is really bad'" check? So far, I assume you do NOT do this but expect an alert to be raised whenever ANY messages fulfills the identity criterion n times in a row. Please explain.

Yes, you assume correctly.

What I'm looking for:
if $msg == 'This is really bad' happens 3 times in a row then :ommail:;mailBody

This would be nice but is not required since the 'This is really bad'
message in my case is very unique:
if ($msg == 'This is really bad' and $server == 'server' and $program
== 'program') happens 3 times in a row then :ommail:;mailBody

- Julian

> Thanks,
> Rainer


From rgerhards at hq.adiscon.com  Fri Aug  1 10:18:59 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 10:18:59 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>

Just one more re-confirmation:

> What I'm looking for:
> if $msg == 'This is really bad' happens 3 times in a row then
> :ommail:;mailBody
> 
> This would be nice but is not required since the 'This is really bad'
> message in my case is very unique:
> if ($msg == 'This is really bad' and $server == 'server' and $program
> == 'program') happens 3 times in a row then :ommail:;mailBody

So you would actually use such a rule. If "this other thing is really
bad" happened three times, the rule shall not trigger. Is this right?

Rainer


From julianokyap at gmail.com  Fri Aug  1 11:02:32 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 23:02:32 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>

On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just one more re-confirmation:
>
>> What I'm looking for:
>> if $msg == 'This is really bad' happens 3 times in a row then
>> :ommail:;mailBody
>>
>> This would be nice but is not required since the 'This is really bad'
>> message in my case is very unique:
>> if ($msg == 'This is really bad' and $server == 'server' and $program
>> == 'program') happens 3 times in a row then :ommail:;mailBody
>
> So you would actually use such a rule. If "this other thing is really
> bad" happened three times, the rule shall not trigger. Is this right?

Yes, I would use such a rule.  It would make what is already an
awesome application even more awesome. :P  I am also willing to test
it out and run the latest development version...  Which I'm doing
anyway.

And yes, what you just wrote is correct.

- Julian


From rgerhards at hq.adiscon.com  Fri Aug  1 11:58:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 11:58:19 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>

OK, that greatly simplifies things. Actually, it now boils down to
"execute an action only on the n-the time the filter evaluates to true".
I think this is quite easy to implement, but I must verify that...

Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 11:03 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Just one more re-confirmation:
> >
> >> What I'm looking for:
> >> if $msg == 'This is really bad' happens 3 times in a row then
> >> :ommail:;mailBody
> >>
> >> This would be nice but is not required since the 'This is really
> bad'
> >> message in my case is very unique:
> >> if ($msg == 'This is really bad' and $server == 'server' and
> $program
> >> == 'program') happens 3 times in a row then :ommail:;mailBody
> >
> > So you would actually use such a rule. If "this other thing is
really
> > bad" happened three times, the rule shall not trigger. Is this
right?
> 
> Yes, I would use such a rule.  It would make what is already an
> awesome application even more awesome. :P  I am also willing to test
> it out and run the latest development version...  Which I'm doing
> anyway.
> 
> And yes, what you just wrote is correct.
> 
> - Julian


From julianokyap at gmail.com  Fri Aug  1 12:13:50 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Fri, 1 Aug 2008 00:13:50 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>

Roger that Rainer.

Thanks,
Julian

On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> OK, that greatly simplifies things. Actually, it now boils down to
> "execute an action only on the n-the time the filter evaluates to true".
> I think this is quite easy to implement, but I must verify that...
>
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 01, 2008 11:03 AM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>> > Just one more re-confirmation:
>> >
>> >> What I'm looking for:
>> >> if $msg == 'This is really bad' happens 3 times in a row then
>> >> :ommail:;mailBody
>> >>
>> >> This would be nice but is not required since the 'This is really
>> bad'
>> >> message in my case is very unique:
>> >> if ($msg == 'This is really bad' and $server == 'server' and
>> $program
>> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>> >
>> > So you would actually use such a rule. If "this other thing is
> really
>> > bad" happened three times, the rule shall not trigger. Is this
> right?
>>
>> Yes, I would use such a rule.  It would make what is already an
>> awesome application even more awesome. :P  I am also willing to test
>> it out and run the latest development version...  Which I'm doing
>> anyway.
>>
>> And yes, what you just wrote is correct.
>>
>> - Julian
>


From ml at darville.vm.bytemark.co.uk  Fri Aug  1 12:26:50 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Fri, 1 Aug 2008 11:26:50 +0100
Subject: [rsyslog] Changing hostname field
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
Message-ID: <20080801102649.GA27628@darville.vm.bytemark.co.uk>

On Fri, Aug 01, 2008 at 07:58:10AM +0200, Rainer Gerhards wrote:
> Sorry for being brief, I was on the road and needed to write from
> a cell phone :(

Supporting rsyslog on your cell phone - you are very dedicated to you
product ;-)

> It also looks like I was wrong ;) I have reviewed the case. If I
> understand correctly, you get you data from the different unix sockets
> and then forward via a single rule. So you can not use different
> templates (which most often are the real solution to any such problem,
> but are not obvious to most people not directly involved in rsyslog ;)).
>
> Is my understanding correct now?

Yes, and then I'm splitting log messages on the dedicated log host, based on
the hostname field. 

We are keeping the rsyslogd process outside the jails, to to keep the syslog
traffic on an isolated network segment.

-- David

> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, July 31, 2008 5:04 PM
> > To: David Darville; rsyslog-users
> > Subject: Re: [rsyslog] Changing hostname field
> > 
> > Use a template with fixed name.
> > 
> > --- Urspr?ngliche Nachricht ---
> > Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> > Betreff: Re: [rsyslog] Changing hostname field
> > Datum: 31. Juli 2008
> > Uhrzeit: 16:46:59
> > 
> > The jails all have their own unique hostname (and IP), but all share an
> > rsyslogd instance running on the main host, and the %hostname% and
> > %fromhost% in all the log messages from the jails are set to the
> > hostname of
> > the main host. And that is what I want to change.
> > 
> > On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > > Do the jails all share the same hostname and IP? If not, you should
> > be
> > > able to use the %hostname% or %fromhost% properties.
> > >
> > > If so, are they each running their own instance of (r)syslogd?
> > >
> > > -HKS
> > >
> > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > > <ml at darville.vm.bytemark.co.uk> wrote:
> > > > Hello everyone
> > > >
> > > > I am trying to configure rsyslog to service a number of chroot
> > jails in
> > > > addition to the host itself.
> > > >
> > > > But I need to change the hostname field of the syslog messages from
> > the
> > > > different jails, so that I place them in the right log file on the
> > central
> > > > logging host.
> > > >
> > > > My current rsyslog.conf is as follows:
> > > >
> > > > $ModLoad imuxsock
> > > > $ModLoad imklog
> > > > $ModLoad immark
> > > > $ModLoad omrelp
> > > >
> > > > $AddUnixListenSocket /jail/1/dev/log
> > > > $AddUnixListenSocket /jail/2/dev/log
> > > >
> > > > *.* :omrelp:10.0.0.4:2514
> > > >
> > > >
> > > > Can anyone please advice me on how to do that?
> > > >
> > > >
> > > > ---
> > > >
> > > > David Darville
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Fri Aug  1 15:06:43 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 15:06:43 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <20080801102649.GA27628@darville.vm.bytemark.co.uk>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
	<20080801102649.GA27628@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>

Inline...

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Friday, August 01, 2008 12:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> On Fri, Aug 01, 2008 at 07:58:10AM +0200, Rainer Gerhards wrote:
> > Sorry for being brief, I was on the road and needed to write from
> > a cell phone :(
> 
> Supporting rsyslog on your cell phone - you are very dedicated to you
> product ;-)

Thanks - I am trying my best ;) And, if you have not already done so, I'd appreciate if you could rate rsyslog at freshmeat.net, it's just a simple click (even if you need to create an account it doesn't take long). It helps us get momentum and this is as important as code quality ;)

You can do it here (middle of the page):
http://freshmeat.net/projects/rsyslog/

> 
> > It also looks like I was wrong ;) I have reviewed the case. If I
> > understand correctly, you get you data from the different unix
> sockets
> > and then forward via a single rule. So you can not use different
> > templates (which most often are the real solution to any such
> problem,
> > but are not obvious to most people not directly involved in rsyslog
> ;)).
> >
> > Is my understanding correct now?
> 
> Yes, and then I'm splitting log messages on the dedicated log host,
> based on
> the hostname field.
> 
> We are keeping the rsyslogd process outside the jails, to to keep the
> syslog
> traffic on an isolated network segment.

OK, that was fairly easy to do. I have created a special version for you:

http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz

It contains a new config directive that allows you to specify the hostname, see last sample on

http://www.rsyslog.com/doc-imuxsock.html

In my lab it works, but I would appreciate if you could test and verify.

Thanks,
Rainer

> 
> -- David
> 
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > Sent: Thursday, July 31, 2008 5:04 PM
> > > To: David Darville; rsyslog-users
> > > Subject: Re: [rsyslog] Changing hostname field
> > >
> > > Use a template with fixed name.
> > >
> > > --- Urspr?ngliche Nachricht ---
> > > Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> > > Betreff: Re: [rsyslog] Changing hostname field
> > > Datum: 31. Juli 2008
> > > Uhrzeit: 16:46:59
> > >
> > > The jails all have their own unique hostname (and IP), but all
> share an
> > > rsyslogd instance running on the main host, and the %hostname% and
> > > %fromhost% in all the log messages from the jails are set to the
> > > hostname of
> > > the main host. And that is what I want to change.
> > >
> > > On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > > > Do the jails all share the same hostname and IP? If not, you
> should
> > > be
> > > > able to use the %hostname% or %fromhost% properties.
> > > >
> > > > If so, are they each running their own instance of (r)syslogd?
> > > >
> > > > -HKS
> > > >
> > > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > > > <ml at darville.vm.bytemark.co.uk> wrote:
> > > > > Hello everyone
> > > > >
> > > > > I am trying to configure rsyslog to service a number of chroot
> > > jails in
> > > > > addition to the host itself.
> > > > >
> > > > > But I need to change the hostname field of the syslog messages
> from
> > > the
> > > > > different jails, so that I place them in the right log file on
> the
> > > central
> > > > > logging host.
> > > > >
> > > > > My current rsyslog.conf is as follows:
> > > > >
> > > > > $ModLoad imuxsock
> > > > > $ModLoad imklog
> > > > > $ModLoad immark
> > > > > $ModLoad omrelp
> > > > >
> > > > > $AddUnixListenSocket /jail/1/dev/log
> > > > > $AddUnixListenSocket /jail/2/dev/log
> > > > >
> > > > > *.* :omrelp:10.0.0.4:2514
> > > > >
> > > > >
> > > > > Can anyone please advice me on how to do that?
> > > > >
> > > > >
> > > > > ---
> > > > >
> > > > > David Darville
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Fri Aug  1 16:25:02 2008
From: hks.private at gmail.com ((private) HKS)
Date: Fri, 1 Aug 2008 10:25:02 -0400
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
Message-ID: <ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>

This may be a silly question, but is the syslog daemon the proper
place for something like this?

-HKS

On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com> wrote:
> Roger that Rainer.
>
> Thanks,
> Julian
>
> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> OK, that greatly simplifies things. Actually, it now boils down to
>> "execute an action only on the n-the time the filter evaluates to true".
>> I think this is quite easy to implement, but I must verify that...
>>
>> Rainer


From rgerhards at hq.adiscon.com  Fri Aug  1 17:02:32 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 17:02:32 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>

That's not a silly one ;) And it is coming up every now and then. The
last time it came up, I was smart enough to write a blogpost:
http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-plugins.ht
ml

In short, and to this questions: there are different schools of thought.
If you think about a plain ole syslogd shuffling data to disk files, you
do not need that. My vision of the syslogd (actually the "event logging
and alerting") subsystem is much broader. IMHO, it should support
anything that is needed to gather, process and persistently store
events. Also note that I say "events" for a reason - syslog messages are
just a subset of the potential set of events.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Friday, August 01, 2008 4:25 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> This may be a silly question, but is the syslog daemon the proper
> place for something like this?
> 
> -HKS
> 
> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
> wrote:
> > Roger that Rainer.
> >
> > Thanks,
> > Julian
> >
> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> OK, that greatly simplifies things. Actually, it now boils down to
> >> "execute an action only on the n-the time the filter evaluates to
> true".
> >> I think this is quite easy to implement, but I must verify that...
> >>
> >> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Fri Aug  1 17:17:03 2008
From: hks.private at gmail.com ((private) HKS)
Date: Fri, 1 Aug 2008 11:17:03 -0400
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>

Thanks for the link, that post makes a lot of sense. I interpreted
this discussion as moving towards adding alert functionality to the
rsyslogd core - but your stance of keeping the rsyslogd core lean and
efficient while plugins provide a full suite of event handling
processes is reassuring.

I'm looking forward to seeing where you take this plugin architecture.

-HKS

On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> That's not a silly one ;) And it is coming up every now and then. The
> last time it came up, I was smart enough to write a blogpost:
> http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-plugins.ht
> ml
>
> In short, and to this questions: there are different schools of thought.
> If you think about a plain ole syslogd shuffling data to disk files, you
> do not need that. My vision of the syslogd (actually the "event logging
> and alerting") subsystem is much broader. IMHO, it should support
> anything that is needed to gather, process and persistently store
> events. Also note that I say "events" for a reason - syslog messages are
> just a subset of the potential set of events.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of (private) HKS
>> Sent: Friday, August 01, 2008 4:25 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> This may be a silly question, but is the syslog daemon the proper
>> place for something like this?
>>
>> -HKS
>>
>> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
>> wrote:
>> > Roger that Rainer.
>> >
>> > Thanks,
>> > Julian
>> >
>> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> >> OK, that greatly simplifies things. Actually, it now boils down to
>> >> "execute an action only on the n-the time the filter evaluates to
>> true".
>> >> I think this is quite easy to implement, but I must verify that...
>> >>
>> >> Rainer
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From rgerhards at hq.adiscon.com  Fri Aug  1 17:23:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 17:23:35 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
	<ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>

Actually, in this case the (limited) functionality will go into the
core. But that's not because we need it for alerting but it is something
that was scheduled for the core engine at all (because there are other
use cases besides alerting where you need it, e.g. start some corrective
action only after the n-th error indication). Plus, it will be very
limited code. There are some other things, namely the enhanced rate
limiter, which will go to the core. I have some ways to do that via
plug-ins too, but given the expected complexity of this functionality vs
the expected complexity of pluginizing them, the choice to go to core is
really obvious. Another point where one could debate is enhanced
queuing. This, in the long term, is a candidate for being moved to a
plugin because many installations do never use disk-based (or assisted)
queues.

Now comes the important difference: if a generic, any message let me
know when it happens n times in the row filter would have been needed,
that would NOT go into the core. Because it does not belong there. It is
quite complex and even performance intense. For that, future version
will have customer (RainerScript) functions which can be provided by
library plugins. I have plans to implement such a beast (much later),
but it will come as a function that you provide the message to and that
is only loaded on an as-needed basis.

I hope this clarifies.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Friday, August 01, 2008 5:17 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Thanks for the link, that post makes a lot of sense. I interpreted
> this discussion as moving towards adding alert functionality to the
> rsyslogd core - but your stance of keeping the rsyslogd core lean and
> efficient while plugins provide a full suite of event handling
> processes is reassuring.
> 
> I'm looking forward to seeing where you take this plugin architecture.
> 
> -HKS
> 
> On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > That's not a silly one ;) And it is coming up every now and then.
The
> > last time it came up, I was smart enough to write a blogpost:
> > http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-
> plugins.ht
> > ml
> >
> > In short, and to this questions: there are different schools of
> thought.
> > If you think about a plain ole syslogd shuffling data to disk files,
> you
> > do not need that. My vision of the syslogd (actually the "event
> logging
> > and alerting") subsystem is much broader. IMHO, it should support
> > anything that is needed to gather, process and persistently store
> > events. Also note that I say "events" for a reason - syslog messages
> are
> > just a subset of the potential set of events.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> >> Sent: Friday, August 01, 2008 4:25 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> >>
> >> This may be a silly question, but is the syslog daemon the proper
> >> place for something like this?
> >>
> >> -HKS
> >>
> >> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
> >> wrote:
> >> > Roger that Rainer.
> >> >
> >> > Thanks,
> >> > Julian
> >> >
> >> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >> > <rgerhards at hq.adiscon.com> wrote:
> >> >> OK, that greatly simplifies things. Actually, it now boils down
> to
> >> >> "execute an action only on the n-the time the filter evaluates
to
> >> true".
> >> >> I think this is quite easy to implement, but I must verify
> that...
> >> >>
> >> >> Rainer
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From lovely at crm114.net  Mon Aug  4 03:25:28 2008
From: lovely at crm114.net (Julian Tyler)
Date: Mon, 4 Aug 2008 11:25:28 +1000
Subject: [rsyslog] Denial of Service
Message-ID: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>


I have a client/server setup of rsyslog, however if the server becomes  
unreachable
and the client's memory buffers fill up, everything that needs to log  
just hangs/blocks,
such as ssh.

I have enabled disk queueing however it never seems to queue anything  
to disk.

What am i doing wrong ?

---
----> Rsyslog version
---
[app-00-0001 ~]# rsyslogd -v
rsyslogd 3.16.1, compiled with:
	FEATURE_REGEXP:				Yes
	FEATURE_LARGEFILE:			Yes
	FEATURE_NETZIP (message compression):	Yes
	GSSAPI Kerberos 5 support:		No
	FEATURE_DEBUG (debug build, slow code):	No
	Runtime Instrumentation (slow code):	No

---
-----> Here is my rsyslog.conf from the offending box
---
[app-00-0001 ~]# cat /etc/rsyslog.conf
# rsyslog v3: load input modules
# If you do not load inputs, nothing happens!
# You may need to set the module load path if modules are not found.

# Debug Template
$template MyDebug,"Debug line with all properties:\nFROMHOST:  
'%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%',  
PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%',  
APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%', 
\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%', 
\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n"

$template Sys23Forward, "<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME 
% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%"


$ModLoad immark.so # provides --MARK-- message capability
$ModLoad imuxsock.so # provides support for local system logging (e.g.  
via logger command)
$ModLoad imklog.so # kernel logging (formerly provided by rklogd)
$ModLoad lmregexp.so

*.*							/var/log/all

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                -/var/log/ 
messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/ 
maillog


# Log cron stuff
cron.*                                                  -/var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          -/var/log/ 
spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/ 
boot.log

# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/spool/rsyslog/ # where to place spool files
$ActionQueueType LinkedList   # run asynchronously
$ActionQueueFileName remotelog # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as  
possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@(z9)10.238.0.1;RSYSLOG_SyslogProtocol23Format
*.* @@10.238.0.1;Sys23Forward


# ######### Receiving Messages from Remote Hosts ##########
# TCP Syslog Server:
# provides TCP syslog reception and GSS-API (if compiled to support it)
#$ModLoad imtcp.so  # load module
# Note: as of now, you need to use the -t command line option to
# enable TCP reception (e.g. -t514 to run a server at port 514/tcp)
# This will change in later v3 releases.

# UDP Syslog Server:
$ModLoad imudp.so  # provides UDP syslog reception
#$UDPServerRun 514 # start a UDP syslog server at standard port 514
$UDPServerRun 513 # start a UDP syslog server at standard port 514
#$UDPServerAddress



From rgerhards at hq.adiscon.com  Mon Aug  4 11:07:17 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 4 Aug 2008 11:07:17 +0200
Subject: [rsyslog] Denial of Service
In-Reply-To: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>
References: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF16@grfint2.intern.adiscon.com>

I think you are affected by this bug:

http://bugzilla.adiscon.com/show_bug.cgi?id=86

You need to upgrade to the recent v3-stable version.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Julian Tyler
> Sent: Monday, August 04, 2008 3:25 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Denial of Service
> 
> 
> I have a client/server setup of rsyslog, however if the server becomes
> unreachable
> and the client's memory buffers fill up, everything that needs to log
> just hangs/blocks,
> such as ssh.
> 
> I have enabled disk queueing however it never seems to queue anything
> to disk.
> 
> What am i doing wrong ?
> 
> ---
> ----> Rsyslog version
> ---
> [app-00-0001 ~]# rsyslogd -v
> rsyslogd 3.16.1, compiled with:
> 	FEATURE_REGEXP:				Yes
> 	FEATURE_LARGEFILE:			Yes
> 	FEATURE_NETZIP (message compression):	Yes
> 	GSSAPI Kerberos 5 support:		No
> 	FEATURE_DEBUG (debug build, slow code):	No
> 	Runtime Instrumentation (slow code):	No
> 
> ---
> -----> Here is my rsyslog.conf from the offending box
> ---
> [app-00-0001 ~]# cat /etc/rsyslog.conf
> # rsyslog v3: load input modules
> # If you do not load inputs, nothing happens!
> # You may need to set the module load path if modules are not found.
> 
> # Debug Template
> $template MyDebug,"Debug line with all properties:\nFROMHOST:
> '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%',
> PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%',
> APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',
> \nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',
> \nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg:
'%rawmsg%'\n\n"
> 
> $template Sys23Forward, "<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME
> % %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%"
> 
> 
> $ModLoad immark.so # provides --MARK-- message capability
> $ModLoad imuxsock.so # provides support for local system logging (e.g.
> via logger command)
> $ModLoad imklog.so # kernel logging (formerly provided by rklogd)
> $ModLoad lmregexp.so
> 
> *.*							/var/log/all
> 
> # Log all kernel messages to the console.
> # Logging much else clutters up the screen.
> #kern.*                                                 /dev/console
> 
> # Log anything (except mail) of level info or higher.
> # Don't log private authentication messages!
> *.info;mail.none;authpriv.none;cron.none                -/var/log/
> messages
> 
> # The authpriv file has restricted access.
> authpriv.*
/var/log/secure
> 
> # Log all the mail messages in one place.
> mail.*                                                  -/var/log/
> maillog
> 
> 
> # Log cron stuff
> cron.*                                                  -/var/log/cron
> 
> # Everybody gets emergency messages
> *.emerg                                                 *
> 
> # Save news errors of level crit and higher in a special file.
> uucp,news.crit                                          -/var/log/
> spooler
> 
> # Save boot messages also to boot.log
> local7.*                                                /var/log/
> boot.log
> 
> # Remote Logging (we use TCP for reliable delivery)
> # An on-disk queue is created for this action. If the remote host is
> # down, messages are spooled to disk and sent when it is up again.
> $WorkDirectory /var/spool/rsyslog/ # where to place spool files
> $ActionQueueType LinkedList   # run asynchronously
> $ActionQueueFileName remotelog # unique name prefix for spool files
> $ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as
> possible)
> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
> $ActionResumeRetryCount -1    # infinite retries if host is down
> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
> #*.* @@(z9)10.238.0.1;RSYSLOG_SyslogProtocol23Format
> *.* @@10.238.0.1;Sys23Forward
> 
> 
> # ######### Receiving Messages from Remote Hosts ##########
> # TCP Syslog Server:
> # provides TCP syslog reception and GSS-API (if compiled to support
it)
> #$ModLoad imtcp.so  # load module
> # Note: as of now, you need to use the -t command line option to
> # enable TCP reception (e.g. -t514 to run a server at port 514/tcp)
> # This will change in later v3 releases.
> 
> # UDP Syslog Server:
> $ModLoad imudp.so  # provides UDP syslog reception
> #$UDPServerRun 514 # start a UDP syslog server at standard port 514
> $UDPServerRun 513 # start a UDP syslog server at standard port 514
> #$UDPServerAddress
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From friedl at hq.adiscon.com  Mon Aug  4 17:40:28 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 4 Aug 2008 17:40:28 +0200
Subject: [rsyslog] rsyslog 3.21.2 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF2D@grfint2.intern.adiscon.com>

Hi all,

rsyslog 3.21.2, a member of the devel branch, has been released today.
It offers two improvements: Imuxsock now permits to specify an
alternative hostname to be used with messages originating from that
socket. This is useful for logging jail processes. Ommail now offers the
ability to specify multiple email recipients. Also, the troubleshooting
documentation has been enhanced and now contains information on debug
mode. There are no bug fixes in this release. This is a recommended
update for all devel branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-124.phtml

Changelog:

http://www.rsyslog.com/Article264.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .




From alorbach at ro1.adiscon.com  Tue Aug  5 16:25:04 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 5 Aug 2008 16:25:04 +0200
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>

Hi again, 

I want to inform you all that the search within phpLogCon has been
extended to support full and partial searches in Version 2.5.3,
available for download here: http://www.phplogcon.org/downloads

I first tests have shown, that filtering by hostname and other string
based fields performs much better on database sources now. 
You can use the submenu buttons on each field within the messages view
to perform such a search. If you want to do manual searches, it is very
simple. Here is a sample: 

Search for a full hostname: source:=SERVERNAME (Will only find if the
match is 100% = SERVERNAME)
Search for a partial hostname source:SERVER (Willmatch if the hostname
is SERVER or SERVERNAME).  

I hope this helps, 
Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Thursday, July 31, 2008 10:15 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> Hi,
> 
> the like query can indeed have quiet an impact on performance when
doing
> queries on large databases.
> But I think we can expand the syntax, so you can either search by part
> of a string (LIKE '%search%') or the whole string (= 'search'). This
> should be rather easy to implement. I will put this on my todolist, if
> it is as easy as I think, the next minor update of the devel branch
will
> contain this new feature.
> 
> Best regards,
> Andre Lorbach
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > Sent: Thursday, July 31, 2008 4:10 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] tips for managing data
> >
> > OK, so it seems that doing a query from the query line does a LIKE,
> > which can take significantly longer (sample query 8 seconds vs. 50
> msecs...)
> >
> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
> things
> > up quite a but, but I lose some flexibility. Is there some kind of
> > search syntax where I can differentiate between LIKE and =?
> >
> > If not, I'm thinking something like:
> >
> > source:foo.bar.com   # would be using =
> >
> > ~source:foo   # would be using LIKE
> >
> >
> >
> > Rory Toma wrote:
> > > So, my current mysql rsyslog drops about 20 million rows of data
per
> day.
> > >
> > > Over time, this gets slow as tables grow.
> > >
> > > I'm not a dba, so I was wondering if anyone had some suggestions
for
> > > keeping performance still on the order of seconds, and not minutes
> or hours.
> > >
> > > thx
> > >
> > > I did add a key for EventSource, as that is commonly searched.
> However,
> > > using PhpLogCon, it seems that if I search using the web interface
> (i.e.
> > > I click on a host entry and hit the available searches) it is
> relatively
> > > quick. However, changing the text field that is generated and
> hitting
> > > the "search" button is slow. Do these two methods use the same
> query, or
> > > is something else going on?
> > >
> > > thx
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Tue Aug  5 17:09:22 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 5 Aug 2008 11:09:22 -0400
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
Message-ID: <ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>

Will searching for NAME match SERVERNAME?

-HKS

On Tue, Aug 5, 2008 at 10:25 AM, Andre Lorbach <alorbach at ro1.adiscon.com> wrote:
> Hi again,
>
> I want to inform you all that the search within phpLogCon has been
> extended to support full and partial searches in Version 2.5.3,
> available for download here: http://www.phplogcon.org/downloads
>
> I first tests have shown, that filtering by hostname and other string
> based fields performs much better on database sources now.
> You can use the submenu buttons on each field within the messages view
> to perform such a search. If you want to do manual searches, it is very
> simple. Here is a sample:
>
> Search for a full hostname: source:=SERVERNAME (Will only find if the
> match is 100% = SERVERNAME)
> Search for a partial hostname source:SERVER (Willmatch if the hostname
> is SERVER or SERVERNAME).
>
> I hope this helps,
> Best regards,
> Andre Lorbach
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>> Sent: Thursday, July 31, 2008 10:15 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] tips for managing data
>>
>> Hi,
>>
>> the like query can indeed have quiet an impact on performance when
> doing
>> queries on large databases.
>> But I think we can expand the syntax, so you can either search by part
>> of a string (LIKE '%search%') or the whole string (= 'search'). This
>> should be rather easy to implement. I will put this on my todolist, if
>> it is as easy as I think, the next minor update of the devel branch
> will
>> contain this new feature.
>>
>> Best regards,
>> Andre Lorbach
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> > Sent: Thursday, July 31, 2008 4:10 AM
>> > To: rsyslog-users
>> > Subject: Re: [rsyslog] tips for managing data
>> >
>> > OK, so it seems that doing a query from the query line does a LIKE,
>> > which can take significantly longer (sample query 8 seconds vs. 50
>> msecs...)
>> >
>> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
>> things
>> > up quite a but, but I lose some flexibility. Is there some kind of
>> > search syntax where I can differentiate between LIKE and =?
>> >
>> > If not, I'm thinking something like:
>> >
>> > source:foo.bar.com   # would be using =
>> >
>> > ~source:foo   # would be using LIKE
>> >
>> >
>> >
>> > Rory Toma wrote:
>> > > So, my current mysql rsyslog drops about 20 million rows of data
> per
>> day.
>> > >
>> > > Over time, this gets slow as tables grow.
>> > >
>> > > I'm not a dba, so I was wondering if anyone had some suggestions
> for
>> > > keeping performance still on the order of seconds, and not minutes
>> or hours.
>> > >
>> > > thx
>> > >
>> > > I did add a key for EventSource, as that is commonly searched.
>> However,
>> > > using PhpLogCon, it seems that if I search using the web interface
>> (i.e.
>> > > I click on a host entry and hit the available searches) it is
>> relatively
>> > > quick. However, changing the text field that is generated and
>> hitting
>> > > the "search" button is slow. Do these two methods use the same
>> query, or
>> > > is something else going on?
>> > >
>> > > thx
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > >
>> >
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From alorbach at ro1.adiscon.com  Tue Aug  5 17:31:24 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 5 Aug 2008 17:31:24 +0200
Subject: [rsyslog] tips for managing data
In-Reply-To: <ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>
References: <4891042D.9040805@ooma.com>
	<48911EF0.1010004@ooma.com><F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
	<ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D71@rmail.adiscon.com>

Yes if you use a partial search, it will match as well. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Tuesday, August 05, 2008 5:09 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> Will searching for NAME match SERVERNAME?
> 
> -HKS
> 
> On Tue, Aug 5, 2008 at 10:25 AM, Andre Lorbach
<alorbach at ro1.adiscon.com>
> wrote:
> > Hi again,
> >
> > I want to inform you all that the search within phpLogCon has been
> > extended to support full and partial searches in Version 2.5.3,
> > available for download here: http://www.phplogcon.org/downloads
> >
> > I first tests have shown, that filtering by hostname and other
string
> > based fields performs much better on database sources now.
> > You can use the submenu buttons on each field within the messages
view
> > to perform such a search. If you want to do manual searches, it is
very
> > simple. Here is a sample:
> >
> > Search for a full hostname: source:=SERVERNAME (Will only find if
the
> > match is 100% = SERVERNAME)
> > Search for a partial hostname source:SERVER (Willmatch if the
hostname
> > is SERVER or SERVERNAME).
> >
> > I hope this helps,
> > Best regards,
> > Andre Lorbach
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> >> Sent: Thursday, July 31, 2008 10:15 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] tips for managing data
> >>
> >> Hi,
> >>
> >> the like query can indeed have quiet an impact on performance when
> > doing
> >> queries on large databases.
> >> But I think we can expand the syntax, so you can either search by
part
> >> of a string (LIKE '%search%') or the whole string (= 'search').
This
> >> should be rather easy to implement. I will put this on my todolist,
if
> >> it is as easy as I think, the next minor update of the devel branch
> > will
> >> contain this new feature.
> >>
> >> Best regards,
> >> Andre Lorbach
> >>
> >> > -----Original Message-----
> >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> >> > Sent: Thursday, July 31, 2008 4:10 AM
> >> > To: rsyslog-users
> >> > Subject: Re: [rsyslog] tips for managing data
> >> >
> >> > OK, so it seems that doing a query from the query line does a
LIKE,
> >> > which can take significantly longer (sample query 8 seconds vs.
50
> >> msecs...)
> >> >
> >> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
> >> things
> >> > up quite a but, but I lose some flexibility. Is there some kind
of
> >> > search syntax where I can differentiate between LIKE and =?
> >> >
> >> > If not, I'm thinking something like:
> >> >
> >> > source:foo.bar.com   # would be using =
> >> >
> >> > ~source:foo   # would be using LIKE
> >> >
> >> >
> >> >
> >> > Rory Toma wrote:
> >> > > So, my current mysql rsyslog drops about 20 million rows of
data
> > per
> >> day.
> >> > >
> >> > > Over time, this gets slow as tables grow.
> >> > >
> >> > > I'm not a dba, so I was wondering if anyone had some
suggestions
> > for
> >> > > keeping performance still on the order of seconds, and not
minutes
> >> or hours.
> >> > >
> >> > > thx
> >> > >
> >> > > I did add a key for EventSource, as that is commonly searched.
> >> However,
> >> > > using PhpLogCon, it seems that if I search using the web
interface
> >> (i.e.
> >> > > I click on a host entry and hit the available searches) it is
> >> relatively
> >> > > quick. However, changing the text field that is generated and
> >> hitting
> >> > > the "search" button is slow. Do these two methods use the same
> >> query, or
> >> > > is something else going on?
> >> > >
> >> > > thx
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > >
> >> >
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Tue Aug  5 20:45:31 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 11:45:31 -0700
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com>
	<48911EF0.1010004@ooma.com>	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
Message-ID: <48989FCB.9030609@ooma.com>

Andre Lorbach wrote:
> Hi again, 
>
> I want to inform you all that the search within phpLogCon has been
> extended to support full and partial searches in Version 2.5.3,
> available for download here: http://www.phplogcon.org/downloads
>
> I first tests have shown, that filtering by hostname and other string
> based fields performs much better on database sources now. 
> You can use the submenu buttons on each field within the messages view
> to perform such a search. If you want to do manual searches, it is very
> simple. Here is a sample: 
>
> Search for a full hostname: source:=SERVERNAME (Will only find if the
> match is 100% = SERVERNAME)
> Search for a partial hostname source:SERVER (Willmatch if the hostname
> is SERVER or SERVERNAME).  
>   
Thanks for the quick change.

Now I have another question, which is more sql but I thought I'd ask it 
anyway.

What I'd like to do is keep 3 months worth of data, but rotate and merge 
my tables after a set amount of time to keep search times low. Tables 
more than 3 months old would be dropped.

If anyone has a nice, canned script for this, it would be appreciated. 
Otherwise, I'll read the mysql manuals. 8-)

thx


From rory at ooma.com  Tue Aug  5 23:29:40 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:29:40 -0700
Subject: [rsyslog] phplogcon UI question
Message-ID: <4898C644.5060906@ooma.com>

Is there a way to turn off the hover-exploding info windows when you 
move the mouse over the "message" field?

On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
away.

thx


From rgerhards at hq.adiscon.com  Tue Aug  5 23:31:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 5 Aug 2008 23:31:56 +0200
Subject: [rsyslog] phplogcon UI question
Message-ID: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>

There is a config setting. But I thought it would be off by default?

rainer

----- Urspr?ngliche Nachricht -----
Von: "Rory Toma" <rory at ooma.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 05.08.08 23:29
Betreff: [rsyslog] phplogcon UI question

Is there a way to turn off the hover-exploding info windows when you 
move the mouse over the "message" field?

On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
away.

thx
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Tue Aug  5 23:35:42 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:35:42 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <4898C7AE.7030800@ooma.com>

Rory Toma wrote:
> Is there a way to turn off the hover-exploding info windows when you 
> move the mouse over the "message" field?
>
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
> away.
>
> thx
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   
Firefox 3 does not have this issue.

Neither does IE7.


From rory at ooma.com  Tue Aug  5 23:38:24 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:38:24 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
References: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
Message-ID: <4898C850.1000508@ooma.com>

Rainer Gerhards wrote:
> There is a config setting. But I thought it would be off by default?
>
> rainer
>
>   


ViewEnableDetailPopups

thx


From rory at ooma.com  Wed Aug  6 03:07:46 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 18:07:46 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if query returns no
	results
Message-ID: <4898F962.1030200@ooma.com>

Minor issue, but thought I'd report it.

If i do a query that returns no results, I get a bunch of empty rows 
displayed in phpLogCon.




From rgerhards at hq.adiscon.com  Wed Aug  6 07:43:06 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 07:43:06 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if query
	returns noresults
In-Reply-To: <4898F962.1030200@ooma.com>
References: <4898F962.1030200@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>

Which version are you using? There was an issue with postgres, but that
has been fixed in recent (devel) builds.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 3:08 AM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon displays lots of empty rows if query
> returns noresults
> 
> Minor issue, but thought I'd report it.
> 
> If i do a query that returns no results, I get a bunch of empty rows
> displayed in phpLogCon.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Wed Aug  6 07:47:14 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 22:47:14 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if
 query	returns noresults
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
References: <4898F962.1030200@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
Message-ID: <48993AE2.7050607@ooma.com>

I'm running 2.5.3.


Rainer Gerhards wrote:
> Which version are you using? There was an issue with postgres, but that
> has been fixed in recent (devel) builds.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> Sent: Wednesday, August 06, 2008 3:08 AM
>> To: rsyslog-users
>> Subject: [rsyslog] phplogcon displays lots of empty rows if query
>> returns noresults
>>
>> Minor issue, but thought I'd report it.
>>
>> If i do a query that returns no results, I get a bunch of empty rows
>> displayed in phpLogCon.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   



From rgerhards at hq.adiscon.com  Wed Aug  6 08:01:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 08:01:19 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if
	query	returns noresults
In-Reply-To: <48993AE2.7050607@ooma.com>
References: <4898F962.1030200@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
	<48993AE2.7050607@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>

Can you provide the config of the data source so that Andre can create a
testbed?

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 7:47 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon displays lots of empty rows if query
> returns noresults
> 
> I'm running 2.5.3.
> 
> 
> Rainer Gerhards wrote:
> > Which version are you using? There was an issue with postgres, but
> that
> > has been fixed in recent (devel) builds.
> >
> > Rainer
> >
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> >> Sent: Wednesday, August 06, 2008 3:08 AM
> >> To: rsyslog-users
> >> Subject: [rsyslog] phplogcon displays lots of empty rows if query
> >> returns noresults
> >>
> >> Minor issue, but thought I'd report it.
> >>
> >> If i do a query that returns no results, I get a bunch of empty
rows
> >> displayed in phpLogCon.
> >>
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Wed Aug  6 10:20:34 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 6 Aug 2008 10:20:34 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C7AE.7030800@ooma.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:36 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> Rory Toma wrote:
> > Is there a way to turn off the hover-exploding info windows when you
> > move the mouse over the "message" field?
> >
> > On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> > away.
> >
> > thx
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> Firefox 3 does not have this issue.
> 
> Neither does IE7.

I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
unfortanetly don't have a MacOS machine to test Safari Browser. 

Best regards,
Andre Lorbach


From alorbach at ro1.adiscon.com  Wed Aug  6 10:21:32 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 6 Aug 2008 10:21:32 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C850.1000508@ooma.com>
References: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
	<4898C850.1000508@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D76@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:38 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> Rainer Gerhards wrote:
> > There is a config setting. But I thought it would be off by default?
> >
> > rainer
> >
> >
> 
> ViewEnableDetailPopups

If you install phpLogCon with UserDB System, you can turn this feature
off and on within the Admin Center ;).

Best regards,
Andre Lorbach


From rory at ooma.com  Wed Aug  6 22:58:41 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 13:58:41 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
Message-ID: <489A1081.1090609@ooma.com>


> I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
> unfortanetly don't have a MacOS machine to test Safari Browser. 
>
>   
What about the Windows version of Safari?

http://www.apple.com/downloads/macosx/apple/windows/


From rory at ooma.com  Wed Aug  6 22:59:23 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 13:59:23 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows
 if	query	returns noresults
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>
References: <4898F962.1030200@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>	<48993AE2.7050607@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>
Message-ID: <489A10AB.30101@ooma.com>

Rainer Gerhards wrote:
> Can you provide the config of the data source so that Andre can create a
> testbed?
>
> Thanks,
> Rainer
>
>   
>
Exactly what info would you like? I am using the default syslog schema 
in SystemEvents.




From rgerhards at hq.adiscon.com  Wed Aug  6 23:04:59 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 23:04:59 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if	queryreturns
	noresults
Message-ID: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>

The definition of the data source in question - should be in xour config file...

----- Urspr?ngliche Nachricht -----
Von: "Rory Toma" <rory at ooma.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 06.08.08 22:59
Betreff: Re: [rsyslog] phplogcon displays lots of empty rows if	query	returns noresults

Rainer Gerhards wrote:
> Can you provide the config of the data source so that Andre can create a
> testbed?
>
> Thanks,
> Rainer
>
>   
>
Exactly what info would you like? I am using the default syslog schema 
in SystemEvents.


_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Wed Aug  6 23:09:40 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 14:09:40 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if	queryreturns
 noresults
In-Reply-To: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>
References: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>
Message-ID: <489A1314.3040909@ooma.com>

Rainer Gerhards wrote:
> The definition of the data source in question - should be in xour config file...
>
>
>   

$CFG['DefaultSourceID'] = 'Source1';

$CFG['Sources']['Source1']['ID'] = 'Source1';
$CFG['Sources']['Source1']['Name'] = 'rsyslog';
$CFG['Sources']['Source1']['ViewID'] = 'SYSLOG';
$CFG['Sources']['Source1']['SourceType'] = SOURCE_DB;
$CFG['Sources']['Source1']['DBTableType'] = 'monitorware';
$CFG['Sources']['Source1']['DBServer'] = 'rsyslog.corp.ooma.com';
$CFG['Sources']['Source1']['DBName'] = 'Syslog';
$CFG['Sources']['Source1']['DBUser'] = 'xxx';
$CFG['Sources']['Source1']['DBPassword'] = 'xxx';
$CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents';
$CFG['Sources']['Source1']['DBEnableRowCounting'] = false;




From rory at ooma.com  Thu Aug  7 01:53:49 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 16:53:49 -0700
Subject: [rsyslog] ok, here's my sql rotation...
Message-ID: <489A398D.6010402@ooma.com>

OK, so here's what I ended up doing...

I created SystemEvents1 thru SystemEvents90, and create SystemEvents as 
a union of 90 thru 1. (1 being last on the union line so it's the 
"active" table)

In cron.daily, I run

mysql <blah blah blah> < script.mysql

where script.mysql is

use Syslog;

CREATE TABLE `SystemEvents0` ( <table def>) Engine=MyISAM;

RENAME TABLE SystemEvents90 to SystemEvents91,
SystemEvents89 to SystemEvents 90,
SystemEvents88 to SystemEvents 89,
........
SystemEvents1 to SystemEvents2,
SystemEvents0 to SystemEvents1;

DROP TABLE SystemEvents91;

This gives me 90 days of logs at a reasonable performance level.

I'm kinda new to MySQL, so I took the route of ease of implementation.


From rory at ooma.com  Thu Aug  7 02:12:13 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 17:12:13 -0700
Subject: [rsyslog] see sql queries on phplogcon
Message-ID: <489A3DDD.4000205@ooma.com>

How do I see what queries are being done in phplogcon? I don't have sql 
logging turned on in my db.

the := syntax seems to be no longer working for me. I get an error 
telling me there were no syslog records found when I try it.


I've set up my SystemEvents table as a union and merged all the tables 
in with data. This shouldn't affect it, I would think. My sql queries by 
hand seem to work fine.




From mic at npgx.com.au  Thu Aug  7 04:23:29 2008
From: mic at npgx.com.au (Michael Mansour)
Date: Thu, 7 Aug 2008 13:23:29 +1100
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <489A1081.1090609@ooma.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
	<489A1081.1090609@ooma.com>
Message-ID: <20080807022212.M77981@npgx.com.au>

Hi guys,

> > I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
> > unfortanetly don't have a MacOS machine to test Safari Browser. 
> >
> >   
> What about the Windows version of Safari?
> 
> http://www.apple.com/downloads/macosx/apple/windows/

There are websites out there which will freely test your (public) web page on
a number of browsers of your choice, sending you the results after a period of
time.

A web search will be able to find these sites.

Michael.


From rgerhards at hq.adiscon.com  Thu Aug  7 08:06:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 08:06:35 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <489A3DDD.4000205@ooma.com>
References: <489A3DDD.4000205@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>

Andre,

I think this triggers a new feature request: I know you have this in
debugging (at least in times), but could you (easily enough) add an
option that displays the SQL statement, e.g. at the top of the page?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 07, 2008 2:12 AM
> To: rsyslog-users
> Subject: [rsyslog] see sql queries on phplogcon
> 
> How do I see what queries are being done in phplogcon? I don't have
sql
> logging turned on in my db.
> 
> the := syntax seems to be no longer working for me. I get an error
> telling me there were no syslog records found when I try it.
> 
> 
> I've set up my SystemEvents table as a union and merged all the tables
> in with data. This shouldn't affect it, I would think. My sql queries
> by
> hand seem to work fine.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:08:49 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:08:49 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <489A1081.1090609@ooma.com>
References: <4898C644.5060906@ooma.com>
	<4898C7AE.7030800@ooma.com><F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
	<489A1081.1090609@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7B@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 10:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> 
> > I usually test phpLogCon with the Browsers Opera, Firefox and IE7.
But I
> > unfortanetly don't have a MacOS machine to test Safari Browser.
> >
> >
> What about the Windows version of Safari?
> 
> http://www.apple.com/downloads/macosx/apple/windows/

Oh thanks I wasn't aware Safari was available for Windows ;). I will
download and install this browser and see what I can do. 

Best regards,
Andre Lorbach


From alorbach at ro1.adiscon.com  Thu Aug  7 10:11:07 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:11:07 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
References: <489A3DDD.4000205@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>

I agree, I will add some debug output so you can see the SQL queries
which are performed in background.
Currently I could only tell where to uncomment some debug code in the
logstream classes, but I recommend to wait until I added this feature. 

Best regards,
Andre Lorbach


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 8:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] see sql queries on phplogcon
> 
> Andre,
> 
> I think this triggers a new feature request: I know you have this in
> debugging (at least in times), but could you (easily enough) add an
> option that displays the SQL statement, e.g. at the top of the page?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > Sent: Thursday, August 07, 2008 2:12 AM
> > To: rsyslog-users
> > Subject: [rsyslog] see sql queries on phplogcon
> >
> > How do I see what queries are being done in phplogcon? I don't have
> sql
> > logging turned on in my db.
> >
> > the := syntax seems to be no longer working for me. I get an error
> > telling me there were no syslog records found when I try it.
> >
> >
> > I've set up my SystemEvents table as a union and merged all the
tables
> > in with data. This shouldn't affect it, I would think. My sql
queries
> > by
> > hand seem to work fine.
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:14:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:14:25 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <489A3DDD.4000205@ooma.com>
References: <489A3DDD.4000205@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>

One question, are you searching for a single word or a phrase? 
If you are searching for a phrase, please use + instead of spaces. 
For example, you want to find a syslogtag like "mysql safe", then you
use this search: 
source:=mysql+safe

If you do not use the + instead of the space, the filter will be
splitted at the word safe, which will be considered as message filter
then. 

Maybe this helps - best regards,
Andre Lorbach



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 07, 2008 2:12 AM
> To: rsyslog-users
> Subject: [rsyslog] see sql queries on phplogcon
> 
> How do I see what queries are being done in phplogcon? I don't have
sql
> logging turned on in my db.
> 
> the := syntax seems to be no longer working for me. I get an error
> telling me there were no syslog records found when I try it.
> 
> 
> I've set up my SystemEvents table as a union and merged all the tables
> in with data. This shouldn't affect it, I would think. My sql queries
by
> hand seem to work fine.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 10:26:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 10:26:44 +0200
Subject: [rsyslog] debug code in phpLogCon - was: see sql queries on
	phplogcon
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
References: <489A3DDD.4000205@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>

In general, let me recommend something that has been proven quite
valuable in rsyslog: Many debug facilities in rsyslog are not commented
out or removed, but can be activated during runtime by switch(es). This
is quite useful when it comes down to troubleshooting issues with a
user's installation. All that is needed is to activate a debug setting
and you see, in all its glory ;), what is going on in that installation.
It may even be worth a thought if phpLogCon should support creating
debug files, which can be sent to you for hard to find cases. Not sure
if that makes sense for phpLogCon, though...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Thursday, August 07, 2008 10:11 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] see sql queries on phplogcon
> 
> I agree, I will add some debug output so you can see the SQL queries
> which are performed in background.
> Currently I could only tell where to uncomment some debug code in the
> logstream classes, but I recommend to wait until I added this feature.
> 
> Best regards,
> Andre Lorbach
> 
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, August 07, 2008 8:07 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] see sql queries on phplogcon
> >
> > Andre,
> >
> > I think this triggers a new feature request: I know you have this in
> > debugging (at least in times), but could you (easily enough) add an
> > option that displays the SQL statement, e.g. at the top of the page?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > > Sent: Thursday, August 07, 2008 2:12 AM
> > > To: rsyslog-users
> > > Subject: [rsyslog] see sql queries on phplogcon
> > >
> > > How do I see what queries are being done in phplogcon? I don't
have
> > sql
> > > logging turned on in my db.
> > >
> > > the := syntax seems to be no longer working for me. I get an error
> > > telling me there were no syslog records found when I try it.
> > >
> > >
> > > I've set up my SystemEvents table as a union and merged all the
> tables
> > > in with data. This shouldn't affect it, I would think. My sql
> queries
> > > by
> > > hand seem to work fine.
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:32:39 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:32:39 +0200
Subject: [rsyslog] debug code in phpLogCon - was: see sql queries
	onphplogcon
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>
References: <489A3DDD.4000205@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7E@rmail.adiscon.com>

We have some kind of a debug facility already in phpLogCon, it just
needs to be enabled and extended a little bit.
I think this will be one of the next minor new features to do then ;)!

--
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 10:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] debug code in phpLogCon - was: see sql queries
> onphplogcon
> 
> In general, let me recommend something that has been proven quite
> valuable in rsyslog: Many debug facilities in rsyslog are not
commented
> out or removed, but can be activated during runtime by switch(es).
This
> is quite useful when it comes down to troubleshooting issues with a
> user's installation. All that is needed is to activate a debug setting
> and you see, in all its glory ;), what is going on in that
installation.
> It may even be worth a thought if phpLogCon should support creating
> debug files, which can be sent to you for hard to find cases. Not sure
> if that makes sense for phpLogCon, though...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > Sent: Thursday, August 07, 2008 10:11 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] see sql queries on phplogcon
> >
> > I agree, I will add some debug output so you can see the SQL queries
> > which are performed in background.
> > Currently I could only tell where to uncomment some debug code in
the
> > logstream classes, but I recommend to wait until I added this
feature.
> >
> > Best regards,
> > Andre Lorbach
> >
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > Sent: Thursday, August 07, 2008 8:07 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] see sql queries on phplogcon
> > >
> > > Andre,
> > >
> > > I think this triggers a new feature request: I know you have this
in
> > > debugging (at least in times), but could you (easily enough) add
an
> > > option that displays the SQL statement, e.g. at the top of the
page?
> > >
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > > > Sent: Thursday, August 07, 2008 2:12 AM
> > > > To: rsyslog-users
> > > > Subject: [rsyslog] see sql queries on phplogcon
> > > >
> > > > How do I see what queries are being done in phplogcon? I don't
> have
> > > sql
> > > > logging turned on in my db.
> > > >
> > > > the := syntax seems to be no longer working for me. I get an
error
> > > > telling me there were no syslog records found when I try it.
> > > >
> > > >
> > > > I've set up my SystemEvents table as a union and merged all the
> > tables
> > > > in with data. This shouldn't affect it, I would think. My sql
> > queries
> > > > by
> > > > hand seem to work fine.
> > > >
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:36:12 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:36:12 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:30 PM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon UI question
> 
> Is there a way to turn off the hover-exploding info windows when you
> move the mouse over the "message" field?
> 
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> away.

Ok I have Safari installed now, and I see what you mean. 
Actually this is some kind of a redraw issue within Safari, which does
not redraw the content which was behind the popup window. If you resize
the window after some popup messages have been viewed, you will see that
the content is redrawn. 

However I think we are going to work on this popup window anyway, as it
is not perfect yet. 

Best regards,
Andre Lorbach



From ml at darville.vm.bytemark.co.uk  Thu Aug  7 11:12:35 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Thu, 7 Aug 2008 10:12:35 +0100
Subject: [rsyslog] Changing hostname field
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
References: <20080801102649.GA27628@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
Message-ID: <20080807091235.GA16356@darville.vm.bytemark.co.uk>

On Fri, Aug 01, 2008 at 03:06:43PM +0200, Rainer Gerhards wrote:

> > > It also looks like I was wrong ;) I have reviewed the case. If I
> > > understand correctly, you get you data from the different unix
> > sockets
> > > and then forward via a single rule. So you can not use different
> > > templates (which most often are the real solution to any such
> > problem,
> > > but are not obvious to most people not directly involved in rsyslog
> > ;)).
> > >
> > > Is my understanding correct now?
> > 
> > Yes, and then I'm splitting log messages on the dedicated log host,
> > based on
> > the hostname field.
> > 
> > We are keeping the rsyslogd process outside the jails, to to keep the
> > syslog
> > traffic on an isolated network segment.
> 
> OK, that was fairly easy to do. I have created a special version for you:
> 
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz
> 
> It contains a new config directive that allows you to specify the hostname, see last sample on
> 
> http://www.rsyslog.com/doc-imuxsock.html
> 
> In my lab it works, but I would appreciate if you could test and verify.

Thank you very much, that is extremely good service - it works perfectly.

I owe you a beer for that.......


-- David Darville


From rgerhards at hq.adiscon.com  Thu Aug  7 10:42:14 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 10:42:14 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
References: <4898C644.5060906@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>

> Ok I have Safari installed now, and I see what you mean.
> Actually this is some kind of a redraw issue within Safari, which does
> not redraw the content which was behind the popup window. If you
resize
> the window after some popup messages have been viewed, you will see
> that
> the content is redrawn.
> 
> However I think we are going to work on this popup window anyway, as
it
> is not perfect yet.

Let me express my personal opinion in the hope that others will also
provide some input: I do *not* think it is well spent time to enhance
this Window. The nice thing is that it can be turned off and that should
solve most problems ;) Actually, I (again personally!) find the window
in itself quite problematic: if it is turned on, it blocks me from doing
some things in the natural way, because it always pops up and blocks
view of parts of the web page *just* because I happen to have moved the
mouse over some part of the page. For me, this leads to such funny
things that I need to concentrate on moving the mouse from bottom to top
(and not in the opposite direction ;)) when I want to activate some
context menu items - because otherwise the popup would block me from
reaching that point.

Again, this is just my personal view, but I would like to bring that
topic up to a broader audience.

Rainer


From rgerhards at hq.adiscon.com  Thu Aug  7 12:22:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 12:22:19 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <20080807091235.GA16356@darville.vm.bytemark.co.uk>
References: <20080801102649.GA27628@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
	<20080807091235.GA16356@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF67@grfint2.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Thursday, August 07, 2008 11:13 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> On Fri, Aug 01, 2008 at 03:06:43PM +0200, Rainer Gerhards wrote:
> 
> > > > It also looks like I was wrong ;) I have reviewed the case. If I
> > > > understand correctly, you get you data from the different unix
> > > sockets
> > > > and then forward via a single rule. So you can not use different
> > > > templates (which most often are the real solution to any such
> > > problem,
> > > > but are not obvious to most people not directly involved in
> rsyslog
> > > ;)).
> > > >
> > > > Is my understanding correct now?
> > >
> > > Yes, and then I'm splitting log messages on the dedicated log
host,
> > > based on
> > > the hostname field.
> > >
> > > We are keeping the rsyslogd process outside the jails, to to keep
> the
> > > syslog
> > > traffic on an isolated network segment.
> >
> > OK, that was fairly easy to do. I have created a special version for
> you:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz
> >
> > It contains a new config directive that allows you to specify the
> hostname, see last sample on
> >
> > http://www.rsyslog.com/doc-imuxsock.html
> >
> > In my lab it works, but I would appreciate if you could test and
> verify.
> 
> Thank you very much, that is extremely good service - it works
> perfectly.
> 

Excellent!

> I owe you a beer for that.......

Be careful: I extremely well remember some things ;)

Rainer
> 
> 
> -- David Darville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From ml at darville.vm.bytemark.co.uk  Thu Aug  7 14:05:52 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Thu, 7 Aug 2008 13:05:52 +0100
Subject: [rsyslog] Problems with spooling to disk
Message-ID: <20080807120552.GA16724@darville.vm.bytemark.co.uk>

Hello everybody

Now I'm trying to configure rsyslog to spool log messages to disk, while the
connection to the central log collector is down. But it does not work......

I am using rsyslog v 3.21.2

I have configured the client rsyslog like this:

$ModLoad imuxsock
$ModLoad omrelp
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName rsyslog-buffer
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
*.* :omrelp:172.16.0.43:2514

Which is similar to the reliable forwarding configuration from the
documentation, except that I prefer to use RELP.

But When I test it, by stopping rsyslog on the central log collector, and
running this small shell script on the client:

#! /bin/sh
for ((i=1;i<=10000;i+=1)); do
  logger Log line $i
done

The shell script does not complete until I restart rsyslog, on the central
log collector, and when I look at the timestamps in the logfile on the log
collector, I see a large time difference between log line 3303 and 3304.

My plan is to get rsyslog to spool all log entries locally, while the
connection the the central log collector is interrupted, and then flush all
the spooled log messages to the central log collector when the communication
is restored - while the rest of the client system continues with business as
usual.

I can send you a debug log is needed.


-- David Darville


From rgerhards at hq.adiscon.com  Thu Aug  7 14:19:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 14:19:04 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>

>From what I see, this should work. So please send the debug log, client
(more important) and server please :)

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Thursday, August 07, 2008 2:06 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Problems with spooling to disk
> 
> Hello everybody
> 
> Now I'm trying to configure rsyslog to spool log messages to disk,
> while the
> connection to the central log collector is down. But it does not
> work......
> 
> I am using rsyslog v 3.21.2
> 
> I have configured the client rsyslog like this:
> 
> $ModLoad imuxsock
> $ModLoad omrelp
> $WorkDirectory /var/spool/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueFileName rsyslog-buffer
> $ActionResumeRetryCount -1
> $ActionQueueSaveOnShutdown on
> *.* :omrelp:172.16.0.43:2514
> 
> Which is similar to the reliable forwarding configuration from the
> documentation, except that I prefer to use RELP.
> 
> But When I test it, by stopping rsyslog on the central log collector,
> and
> running this small shell script on the client:
> 
> #! /bin/sh
> for ((i=1;i<=10000;i+=1)); do
>   logger Log line $i
> done
> 
> The shell script does not complete until I restart rsyslog, on the
> central
> log collector, and when I look at the timestamps in the logfile on the
> log
> collector, I see a large time difference between log line 3303 and
> 3304.
> 
> My plan is to get rsyslog to spool all log entries locally, while the
> connection the the central log collector is interrupted, and then
flush
> all
> the spooled log messages to the central log collector when the
> communication
> is restored - while the rest of the client system continues with
> business as
> usual.
> 
> I can send you a debug log is needed.
> 
> 
> -- David Darville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 16:41:14 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 16:41:14 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D86@rmail.adiscon.com>

A new Version of the beta branch has been released a few moments ago
(mailing list should be informed very soon).
I have rewritten and added some Javascript code in order to enhance the
detail popup window. It does work now equal and very well in Firefox,
Opera and Internet Explorer. It looks a little bit better in Safari, but
it still has redraw problems. But I would say these are browser problems
with css. 

You can download the new Version from here: 
http://www.phplogcon.org/downloads

If you are working with the devel branch, you will have to wait until
tomorrow. 

Best regards,
Andre Lorbach


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:30 PM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon UI question
> 
> Is there a way to turn off the hover-exploding info windows when you
> move the mouse over the "message" field?
> 
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> away.
> 
> thx
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 17:00:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 17:00:25 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com><ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF6F@grfint2.intern.adiscon.com>

Just to prove the point: I have now implemented it. As expected, it is a
very small code change (once you know where to apply it ;)). Have a look
at the primary patch:

http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=action.c;h=3a2584de0f
24fdf3f1af0c35748d29fe5e3a3845;hp=f72194059d41f011ae6daf6b6aa1a61d147255
3e;hb=1480263b026984a3d48a7f750a78911777464797;hpb=0a7f964436af73f2e7fbd
403b563f8d5a743f4a5

and note that most is comment. The amount of code actually executed each
time is rater limited and does neither bear a large memory footprint,
nor execution footprint. If the feature is not used, it is one simple
branch. Even if it is used, the performance effects are very limited.
Most importantly, a costly call to time() could be avoided by using the
value that was already present (but needed a bit of reordering).

I am going to this detail just to explain the fine difference between
what belongs into the core (even though it is not "real core
functionality") compared to what must not get into it.

For example, if I had implemented that via a plugin, I would have needed
at least one (indirectly addressed) procedure call branch, still an if,
plus a return branch. Not good for speculative execution. Also, the
plumbing would probably have required more code than the full patch
shown (what means bad from a maintenance point of view as well a bad
from a CPU memory cache point of view ;)).

It remains the argument that if the code would not have been introduced,
the core would be a little more slim. That's right ;) An easy solution
would have been conditional compilation, but I have not applied it as I
think the few extra bytes and CPU cycles really don't matter (that
much).

Just for you info...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, August 01, 2008 5:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Actually, in this case the (limited) functionality will go into the
> core. But that's not because we need it for alerting but it is
> something
> that was scheduled for the core engine at all (because there are other
> use cases besides alerting where you need it, e.g. start some
> corrective
> action only after the n-th error indication). Plus, it will be very
> limited code. There are some other things, namely the enhanced rate
> limiter, which will go to the core. I have some ways to do that via
> plug-ins too, but given the expected complexity of this functionality
> vs
> the expected complexity of pluginizing them, the choice to go to core
> is
> really obvious. Another point where one could debate is enhanced
> queuing. This, in the long term, is a candidate for being moved to a
> plugin because many installations do never use disk-based (or
assisted)
> queues.
> 
> Now comes the important difference: if a generic, any message let me
> know when it happens n times in the row filter would have been needed,
> that would NOT go into the core. Because it does not belong there. It
> is
> quite complex and even performance intense. For that, future version
> will have customer (RainerScript) functions which can be provided by
> library plugins. I have plans to implement such a beast (much later),
> but it will come as a function that you provide the message to and
that
> is only loaded on an as-needed basis.
> 
> I hope this clarifies.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of (private) HKS
> > Sent: Friday, August 01, 2008 5:17 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> >
> > Thanks for the link, that post makes a lot of sense. I interpreted
> > this discussion as moving towards adding alert functionality to the
> > rsyslogd core - but your stance of keeping the rsyslogd core lean
and
> > efficient while plugins provide a full suite of event handling
> > processes is reassuring.
> >
> > I'm looking forward to seeing where you take this plugin
> architecture.
> >
> > -HKS
> >
> > On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> > > That's not a silly one ;) And it is coming up every now and then.
> The
> > > last time it came up, I was smart enough to write a blogpost:
> > > http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-
> > plugins.ht
> > > ml
> > >
> > > In short, and to this questions: there are different schools of
> > thought.
> > > If you think about a plain ole syslogd shuffling data to disk
> files,
> > you
> > > do not need that. My vision of the syslogd (actually the "event
> > logging
> > > and alerting") subsystem is much broader. IMHO, it should support
> > > anything that is needed to gather, process and persistently store
> > > events. Also note that I say "events" for a reason - syslog
> messages
> > are
> > > just a subset of the potential set of events.
> > >
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> > >> Sent: Friday, August 01, 2008 4:25 PM
> > >> To: rsyslog-users
> > >> Subject: Re: [rsyslog] Alert when multiple repeated lines are
> found
> > >>
> > >> This may be a silly question, but is the syslog daemon the proper
> > >> place for something like this?
> > >>
> > >> -HKS
> > >>
> > >> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap
<julianokyap at gmail.com>
> > >> wrote:
> > >> > Roger that Rainer.
> > >> >
> > >> > Thanks,
> > >> > Julian
> > >> >
> > >> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > >> > <rgerhards at hq.adiscon.com> wrote:
> > >> >> OK, that greatly simplifies things. Actually, it now boils
down
> > to
> > >> >> "execute an action only on the n-the time the filter evaluates
> to
> > >> true".
> > >> >> I think this is quite easy to implement, but I must verify
> > that...
> > >> >>
> > >> >> Rainer
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 17:08:20 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 17:08:20 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>

Julian,

as you have probably seen in my other post, I have implemented the
functionality. I have now also created a test tarball. I'd appreciate if
you could obtain it from

http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz

and give it a try. Read ./doc/rsyslog_conf.html in regard to
$ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
what you intend to do, this should work:

$ActionExecOnlyEveryNthTime 3
*.* ..your action..

You don't need the timeout, but I have included it for completeness.
Well, actually if I were you I'd think if you really don't need it. Is
it really OK that "three in a row" means one each day?

Please provide feedback on this feature.

Thanks,
Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 12:14 PM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> Roger that Rainer.
> 
> Thanks,
> Julian
> 
> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > OK, that greatly simplifies things. Actually, it now boils down to
> > "execute an action only on the n-the time the filter evaluates to
> true".
> > I think this is quite easy to implement, but I must verify that...
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> Sent: Friday, August 01, 2008 11:03 AM
> >> To: Rainer Gerhards
> >> Cc: rsyslog at lists.adiscon.com
> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > found
> >>
> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >> <rgerhards at hq.adiscon.com> wrote:
> >> > Just one more re-confirmation:
> >> >
> >> >> What I'm looking for:
> >> >> if $msg == 'This is really bad' happens 3 times in a row then
> >> >> :ommail:;mailBody
> >> >>
> >> >> This would be nice but is not required since the 'This is really
> >> bad'
> >> >> message in my case is very unique:
> >> >> if ($msg == 'This is really bad' and $server == 'server' and
> >> $program
> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
> >> >
> >> > So you would actually use such a rule. If "this other thing is
> > really
> >> > bad" happened three times, the rule shall not trigger. Is this
> > right?
> >>
> >> Yes, I would use such a rule.  It would make what is already an
> >> awesome application even more awesome. :P  I am also willing to
test
> >> it out and run the latest development version...  Which I'm doing
> >> anyway.
> >>
> >> And yes, what you just wrote is correct.
> >>
> >> - Julian
> >


From friedl at hq.adiscon.com  Thu Aug  7 17:28:30 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Thu, 7 Aug 2008 17:28:30 +0200
Subject: [rsyslog] rsyslog 2.0.6 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF73@grfint2.intern.adiscon.com>

Hi all,

we have just released rsyslog 2.0.6, a version of the ultra-conservative
v2-stable branch. There is a single bugfix in this release: IPv6
addresses could not be specified in forwarding actions, because they
contain colons and the colon character was already used for some other
purpose. IPv6 addresses can now be specified inside of square brackets.
This is a recommended update for all v2-stable branch users.


Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-125.phtml

Changelog:

http://www.rsyslog.com/Article266.phtml


As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From julianokyap at gmail.com  Fri Aug  8 00:38:07 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 7 Aug 2008 12:38:07 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
Message-ID: <d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>

Rainer,

Initial testing looks fine.  I'll try some more to see if I can break it.

- Julian

On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Julian,
>
> as you have probably seen in my other post, I have implemented the
> functionality. I have now also created a test tarball. I'd appreciate if
> you could obtain it from
>
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>
> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
> what you intend to do, this should work:
>
> $ActionExecOnlyEveryNthTime 3
> *.* ..your action..
>
> You don't need the timeout, but I have included it for completeness.
> Well, actually if I were you I'd think if you really don't need it. Is
> it really OK that "three in a row" means one each day?
>
> Please provide feedback on this feature.
>
> Thanks,
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 01, 2008 12:14 PM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> Roger that Rainer.
>>
>> Thanks,
>> Julian
>>
>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>> > OK, that greatly simplifies things. Actually, it now boils down to
>> > "execute an action only on the n-the time the filter evaluates to
>> true".
>> > I think this is quite easy to implement, but I must verify that...
>> >
>> > Rainer
>> >
>> >> -----Original Message-----
>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >> Sent: Friday, August 01, 2008 11:03 AM
>> >> To: Rainer Gerhards
>> >> Cc: rsyslog at lists.adiscon.com
>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> > found
>> >>
>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> >> <rgerhards at hq.adiscon.com> wrote:
>> >> > Just one more re-confirmation:
>> >> >
>> >> >> What I'm looking for:
>> >> >> if $msg == 'This is really bad' happens 3 times in a row then
>> >> >> :ommail:;mailBody
>> >> >>
>> >> >> This would be nice but is not required since the 'This is really
>> >> bad'
>> >> >> message in my case is very unique:
>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>> >> $program
>> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>> >> >
>> >> > So you would actually use such a rule. If "this other thing is
>> > really
>> >> > bad" happened three times, the rule shall not trigger. Is this
>> > right?
>> >>
>> >> Yes, I would use such a rule.  It would make what is already an
>> >> awesome application even more awesome. :P  I am also willing to
> test
>> >> it out and run the latest development version...  Which I'm doing
>> >> anyway.
>> >>
>> >> And yes, what you just wrote is correct.
>> >>
>> >> - Julian
>> >
>


From julianokyap at gmail.com  Fri Aug  8 01:55:03 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 7 Aug 2008 13:55:03 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
Message-ID: <d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>

Yep, after further testing this works great!  Thanks Rainer.

On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com> wrote:
> Rainer,
>
> Initial testing looks fine.  I'll try some more to see if I can break it.
>
> - Julian
>
> On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> Julian,
>>
>> as you have probably seen in my other post, I have implemented the
>> functionality. I have now also created a test tarball. I'd appreciate if
>> you could obtain it from
>>
>> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>>
>> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
>> what you intend to do, this should work:
>>
>> $ActionExecOnlyEveryNthTime 3
>> *.* ..your action..
>>
>> You don't need the timeout, but I have included it for completeness.
>> Well, actually if I were you I'd think if you really don't need it. Is
>> it really OK that "three in a row" means one each day?
>>
>> Please provide feedback on this feature.
>>
>> Thanks,
>> Rainer
>>
>>> -----Original Message-----
>>> From: Julian Yap [mailto:julianokyap at gmail.com]
>>> Sent: Friday, August 01, 2008 12:14 PM
>>> To: Rainer Gerhards
>>> Cc: rsyslog at lists.adiscon.com
>>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> found
>>>
>>> Roger that Rainer.
>>>
>>> Thanks,
>>> Julian
>>>
>>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>>> <rgerhards at hq.adiscon.com> wrote:
>>> > OK, that greatly simplifies things. Actually, it now boils down to
>>> > "execute an action only on the n-the time the filter evaluates to
>>> true".
>>> > I think this is quite easy to implement, but I must verify that...
>>> >
>>> > Rainer
>>> >
>>> >> -----Original Message-----
>>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>>> >> Sent: Friday, August 01, 2008 11:03 AM
>>> >> To: Rainer Gerhards
>>> >> Cc: rsyslog at lists.adiscon.com
>>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>>> > found
>>> >>
>>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>>> >> <rgerhards at hq.adiscon.com> wrote:
>>> >> > Just one more re-confirmation:
>>> >> >
>>> >> >> What I'm looking for:
>>> >> >> if $msg == 'This is really bad' happens 3 times in a row then
>>> >> >> :ommail:;mailBody
>>> >> >>
>>> >> >> This would be nice but is not required since the 'This is really
>>> >> bad'
>>> >> >> message in my case is very unique:
>>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>>> >> $program
>>> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>>> >> >
>>> >> > So you would actually use such a rule. If "this other thing is
>>> > really
>>> >> > bad" happened three times, the rule shall not trigger. Is this
>>> > right?
>>> >>
>>> >> Yes, I would use such a rule.  It would make what is already an
>>> >> awesome application even more awesome. :P  I am also willing to
>> test
>>> >> it out and run the latest development version...  Which I'm doing
>>> >> anyway.
>>> >>
>>> >> And yes, what you just wrote is correct.
>>> >>
>>> >> - Julian
>>> >
>>
>


From mic at npgx.com.au  Fri Aug  8 03:58:18 2008
From: mic at npgx.com.au (Michael Mansour)
Date: Fri, 8 Aug 2008 12:58:18 +1100
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>
References: <4898C644.5060906@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>
Message-ID: <20080808014639.M81869@npgx.com.au>

Hi,

> > Ok I have Safari installed now, and I see what you mean.
> > Actually this is some kind of a redraw issue within Safari, which does
> > not redraw the content which was behind the popup window. If you
> resize
> > the window after some popup messages have been viewed, you will see
> > that
> > the content is redrawn.
> > 
> > However I think we are going to work on this popup window anyway, as
> it
> > is not perfect yet.
> 
> Let me express my personal opinion in the hope that others will also
> provide some input: I do *not* think it is well spent time to enhance
> this Window. The nice thing is that it can be turned off and that should
> solve most problems ;) Actually, I (again personally!) find the 
> window in itself quite problematic: if it is turned on, it blocks me 
> from doing some things in the natural way, because it always pops up 
> and blocks view of parts of the web page *just* because I happen to 
> have moved the mouse over some part of the page. For me, this leads 
> to such funny things that I need to concentrate on moving the mouse 
> from bottom to top
> (and not in the opposite direction ;)) when I want to activate some 
> context menu items - because otherwise the popup would block me from 
> reaching that point.
> 
> Again, this is just my personal view, but I would like to bring that
> topic up to a broader audience.

I can only describe my opinion by giving an example of a firefox extension I
like to use. 

I use an extension (or Add-on in FF3 terminology) called Cooliris Previews.
This extension is nifty in that it allows you to open a sub window (pop up
window) for URL's on web pages. From there you can "lock" the sub window down
and mouse over other URL's and it updates the sub windows.

I find this very handy when going through helpdesk cases in our helpdesk
system or emails in my webmial system.

The problem is, before bringing up the sub window, each URL you mouse over
brings up the cooliris previews icon to the right of the link (location
configurable). By default, mousing over the icon brings up the sub window.
People that use cooliris would quickly find this very annoying because sub
windows will constantly open up by accident. So after changing the default to
only allow opening of a sub window by clicking the cooliris icon, it's more
pleasant to use.

After a while though even this becomes annoying, because 90% of my browsing
time I don't use/need cooliris previews, and many times I accidentally click
the icon and bring up sub windows by accident again (you'll realise how easy
it is to make this mistake when using it).

So now I have the bottom right toolbar which enables/disables the extension,
so I no longer see the cooliris icon to the right of every URL I mouse over.

Sometimes though (when I have multiple FF Windows open) that toolbar icon
turns itself on automatically, and does so in every tab that's open in every
window. It's not perfect (and maybe the developers concentrate more on having
this feature enabled in a FF browser than not) but I have come to the point of
considering removing the extension because the pop ups can be annoying when
browsing.

Regards,

Michael.

> Rainer



From rgerhards at hq.adiscon.com  Fri Aug  8 10:39:14 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 10:39:14 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>

Thanks for the feedback, it will now be part of the next devel release,
I think some time next week :)

Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 08, 2008 1:55 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> Yep, after further testing this works great!  Thanks Rainer.
> 
> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
> wrote:
> > Rainer,
> >
> > Initial testing looks fine.  I'll try some more to see if I can
break
> it.
> >
> > - Julian
> >
> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> Julian,
> >>
> >> as you have probably seen in my other post, I have implemented the
> >> functionality. I have now also created a test tarball. I'd
> appreciate if
> >> you could obtain it from
> >>
> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> >>
> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> >> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout.
> For
> >> what you intend to do, this should work:
> >>
> >> $ActionExecOnlyEveryNthTime 3
> >> *.* ..your action..
> >>
> >> You don't need the timeout, but I have included it for
completeness.
> >> Well, actually if I were you I'd think if you really don't need it.
> Is
> >> it really OK that "three in a row" means one each day?
> >>
> >> Please provide feedback on this feature.
> >>
> >> Thanks,
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> >>> Sent: Friday, August 01, 2008 12:14 PM
> >>> To: Rainer Gerhards
> >>> Cc: rsyslog at lists.adiscon.com
> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> >> found
> >>>
> >>> Roger that Rainer.
> >>>
> >>> Thanks,
> >>> Julian
> >>>
> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >>> <rgerhards at hq.adiscon.com> wrote:
> >>> > OK, that greatly simplifies things. Actually, it now boils down
> to
> >>> > "execute an action only on the n-the time the filter evaluates
to
> >>> true".
> >>> > I think this is quite easy to implement, but I must verify
> that...
> >>> >
> >>> > Rainer
> >>> >
> >>> >> -----Original Message-----
> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> >>> >> To: Rainer Gerhards
> >>> >> Cc: rsyslog at lists.adiscon.com
> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> are
> >>> > found
> >>> >>
> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >>> >> <rgerhards at hq.adiscon.com> wrote:
> >>> >> > Just one more re-confirmation:
> >>> >> >
> >>> >> >> What I'm looking for:
> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
then
> >>> >> >> :ommail:;mailBody
> >>> >> >>
> >>> >> >> This would be nice but is not required since the 'This is
> really
> >>> >> bad'
> >>> >> >> message in my case is very unique:
> >>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
> >>> >> $program
> >>> >> >> == 'program') happens 3 times in a row then
:ommail:;mailBody
> >>> >> >
> >>> >> > So you would actually use such a rule. If "this other thing
is
> >>> > really
> >>> >> > bad" happened three times, the rule shall not trigger. Is
this
> >>> > right?
> >>> >>
> >>> >> Yes, I would use such a rule.  It would make what is already an
> >>> >> awesome application even more awesome. :P  I am also willing to
> >> test
> >>> >> it out and run the latest development version...  Which I'm
> doing
> >>> >> anyway.
> >>> >>
> >>> >> And yes, what you just wrote is correct.
> >>> >>
> >>> >> - Julian
> >>> >
> >>
> >


From rgerhards at hq.adiscon.com  Fri Aug  8 12:28:11 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 12:28:11 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>

Thanks, I now received the log files. And, of course, they made
immediately clear what is going on ;) What you experience is flow
control kicking in. Let me first elaborate a bit:

Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
the local socket) and those who can not (like UDP). The idea is that we
can slow down senders where this does not result in loss of messages,
leaving room inside the queue on busy systems for those where we can not
do that. We had cases where not delaying caused a lot of grief.

There are also two watermarks, the low and high delay mark. If the low
delay mark is reached, a delayable source is delayed for one second. It
is at 70% of the (in-memory) queue capacity. Again, this is all done for
some very good reasons.

However, when I introduced these features, I did not make them
configurable. Also, I did classify the local unix socket as one who can
be delayed. In your scenario, this does not stop the system, but once
the local log socket is filled up, it needs to wait one second for each
message before it is pulled.

Again, the rate limiting logic is there for good reasons. So what I now
see is that I need to add a few config statements to allow to control it
according to your needs. I am also unsure if I should re-classify
imuxsock as an input which cannot be delayed - but that also can have
some drawbacks. Maybe the best thing to do is make its state
configurable and use "not to be delayed" as default.

Anyone with a thought on that?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 2:19 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> >From what I see, this should work. So please send the debug log,
> client
> (more important) and server please :)
> 
> Thanks,
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of David Darville
> > Sent: Thursday, August 07, 2008 2:06 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Problems with spooling to disk
> >
> > Hello everybody
> >
> > Now I'm trying to configure rsyslog to spool log messages to disk,
> > while the
> > connection to the central log collector is down. But it does not
> > work......
> >
> > I am using rsyslog v 3.21.2
> >
> > I have configured the client rsyslog like this:
> >
> > $ModLoad imuxsock
> > $ModLoad omrelp
> > $WorkDirectory /var/spool/rsyslog
> > $ActionQueueType LinkedList
> > $ActionQueueFileName rsyslog-buffer
> > $ActionResumeRetryCount -1
> > $ActionQueueSaveOnShutdown on
> > *.* :omrelp:172.16.0.43:2514
> >
> > Which is similar to the reliable forwarding configuration from the
> > documentation, except that I prefer to use RELP.
> >
> > But When I test it, by stopping rsyslog on the central log
collector,
> > and
> > running this small shell script on the client:
> >
> > #! /bin/sh
> > for ((i=1;i<=10000;i+=1)); do
> >   logger Log line $i
> > done
> >
> > The shell script does not complete until I restart rsyslog, on the
> > central
> > log collector, and when I look at the timestamps in the logfile on
> the
> > log
> > collector, I see a large time difference between log line 3303 and
> > 3304.
> >
> > My plan is to get rsyslog to spool all log entries locally, while
the
> > connection the the central log collector is interrupted, and then
> flush
> > all
> > the spooled log messages to the central log collector when the
> > communication
> > is restored - while the rest of the client system continues with
> > business as
> > usual.
> >
> > I can send you a debug log is needed.
> >
> >
> > -- David Darville
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Fri Aug  8 12:38:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 12:38:56 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF81@grfint2.intern.adiscon.com>

David,

I have done a quick and dirty (not to stay) fix, which makes imuxsock a
NO_DELAY input:

http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test4.tar.gz

I'd appreciate if you could give it a try and report back.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, August 08, 2008 12:28 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> Thanks, I now received the log files. And, of course, they made
> immediately clear what is going on ;) What you experience is flow
> control kicking in. Let me first elaborate a bit:
> 
> Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> the local socket) and those who can not (like UDP). The idea is that
we
> can slow down senders where this does not result in loss of messages,
> leaving room inside the queue on busy systems for those where we can
> not
> do that. We had cases where not delaying caused a lot of grief.
> 
> There are also two watermarks, the low and high delay mark. If the low
> delay mark is reached, a delayable source is delayed for one second.
It
> is at 70% of the (in-memory) queue capacity. Again, this is all done
> for
> some very good reasons.
> 
> However, when I introduced these features, I did not make them
> configurable. Also, I did classify the local unix socket as one who
can
> be delayed. In your scenario, this does not stop the system, but once
> the local log socket is filled up, it needs to wait one second for
each
> message before it is pulled.
> 
> Again, the rate limiting logic is there for good reasons. So what I
now
> see is that I need to add a few config statements to allow to control
> it
> according to your needs. I am also unsure if I should re-classify
> imuxsock as an input which cannot be delayed - but that also can have
> some drawbacks. Maybe the best thing to do is make its state
> configurable and use "not to be delayed" as default.
> 
> Anyone with a thought on that?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, August 07, 2008 2:19 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Problems with spooling to disk
> >
> > >From what I see, this should work. So please send the debug log,
> > client
> > (more important) and server please :)
> >
> > Thanks,
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of David Darville
> > > Sent: Thursday, August 07, 2008 2:06 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] Problems with spooling to disk
> > >
> > > Hello everybody
> > >
> > > Now I'm trying to configure rsyslog to spool log messages to disk,
> > > while the
> > > connection to the central log collector is down. But it does not
> > > work......
> > >
> > > I am using rsyslog v 3.21.2
> > >
> > > I have configured the client rsyslog like this:
> > >
> > > $ModLoad imuxsock
> > > $ModLoad omrelp
> > > $WorkDirectory /var/spool/rsyslog
> > > $ActionQueueType LinkedList
> > > $ActionQueueFileName rsyslog-buffer
> > > $ActionResumeRetryCount -1
> > > $ActionQueueSaveOnShutdown on
> > > *.* :omrelp:172.16.0.43:2514
> > >
> > > Which is similar to the reliable forwarding configuration from the
> > > documentation, except that I prefer to use RELP.
> > >
> > > But When I test it, by stopping rsyslog on the central log
> collector,
> > > and
> > > running this small shell script on the client:
> > >
> > > #! /bin/sh
> > > for ((i=1;i<=10000;i+=1)); do
> > >   logger Log line $i
> > > done
> > >
> > > The shell script does not complete until I restart rsyslog, on the
> > > central
> > > log collector, and when I look at the timestamps in the logfile on
> > the
> > > log
> > > collector, I see a large time difference between log line 3303 and
> > > 3304.
> > >
> > > My plan is to get rsyslog to spool all log entries locally, while
> the
> > > connection the the central log collector is interrupted, and then
> > flush
> > > all
> > > the spooled log messages to the central log collector when the
> > > communication
> > > is restored - while the rest of the client system continues with
> > > business as
> > > usual.
> > >
> > > I can send you a debug log is needed.
> > >
> > >
> > > -- David Darville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From friedl at hq.adiscon.com  Fri Aug  8 14:18:56 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 8 Aug 2008 14:18:56 +0200
Subject: [rsyslog] rsyslog 3.18.2 released (stable)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF85@grfint2.intern.adiscon.com>

Hi all,

we have just released rsyslog 3.18.2, a version of the v3-stable branch.
There is a single bugfix in this release: IPv6 addresses could not be
specified in forwarding actions, because they contain colons and the
colon character was already used for some other purpose. IPv6 addresses
can now be specified inside of square brackets. This is the same fix
that was released for v2-stable yesterday. This is a recommended update
for all v3-stable branch users.


Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-126.phtml

Changelog:

http://www.rsyslog.com/Article268.phtml


As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From rory at ooma.com  Fri Aug  8 23:32:48 2008
From: rory at ooma.com (Rory Toma)
Date: Fri, 08 Aug 2008 14:32:48 -0700
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>
References: <489A3DDD.4000205@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>
Message-ID: <489CBB80.10507@ooma.com>

Actually, I ended up doing a "LIKE" search on hostname, looking at the 
query via full processlist and running the query by hand, assuming it 
was the same, just without the "LIKE" syntax.

Turns out I had a horked index in a table that MAX(id) exposed. Fixed it 
and moved on.

thx

Andre Lorbach wrote:
> One question, are you searching for a single word or a phrase? 
> If you are searching for a phrase, please use + instead of spaces. 
> For example, you want to find a syslogtag like "mysql safe", then you
> use this search: 
> source:=mysql+safe
>
> If you do not use the + instead of the space, the filter will be
> splitted at the word safe, which will be considered as message filter
> then. 
>
> Maybe this helps - best regards,
> Andre Lorbach
>
>
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> Sent: Thursday, August 07, 2008 2:12 AM
>> To: rsyslog-users
>> Subject: [rsyslog] see sql queries on phplogcon
>>
>> How do I see what queries are being done in phplogcon? I don't have
>>     
> sql
>   
>> logging turned on in my db.
>>
>> the := syntax seems to be no longer working for me. I get an error
>> telling me there were no syslog records found when I try it.
>>
>>
>> I've set up my SystemEvents table as a union and merged all the tables
>> in with data. This shouldn't affect it, I would think. My sql queries
>>     
> by
>   
>> hand seem to work fine.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   



From mmcgrath at redhat.com  Tue Aug 12 17:37:35 2008
From: mmcgrath at redhat.com (Mike McGrath)
Date: Tue, 12 Aug 2008 10:37:35 -0500 (CDT)
Subject: [rsyslog] rsyslog dropping logs
In-Reply-To: <1216896024.7184.189.camel@rgf9dev.intern.adiscon.com>
References: <alpine.LFD.1.10.0807231413270.14749@laptop.mmcgrath.net>
	<1216896024.7184.189.camel@rgf9dev.intern.adiscon.com>
Message-ID: <alpine.LFD.1.10.0808121034550.5157@laptop.mmcgrath.net>

On Thu, 24 Jul 2008, Rainer Gerhards wrote:

> (I am not commenting on v2 vs. v3 as this is already done)
>
> First of all, we need to keep in mind that UDP is inherently lossy. Even
> when a frame is seen received by the local stack, it does not mean that
> it will eventually be forwarded to the application. If message bursts
> come in very quickly and the OS scheduler does not schedule the app fast
> enough to receive this messages (or the app is too slow in itself! ;))
> new frames may overwrite frames inside the stack's receive buffers.
>
> So it is always a good idea to avoid UDP if that's possible.
>
> HOWEVER, I, too, find it somewhat unusual that around 90% of all
> incoming frames are discarded before the rsyslog receiver could process
> them. One explanation I have is that you have bursts (or volume in
> general) that outperforms the configured actions. Having seen the config
> file, and seeing it does not include any database writer, it is hard to
> imagine this should happen, assuming reasonable hardware sizing is used.
> A cause could be excessive synchronous writes. Many rules do not put a
> dash in front of the file name and without it (in v2), every write is
> immediately synced. This is very costly. But still, I have never seen
> that this alone outperforms a system.
>
> To dig deeper into what is happening, a debug log would be most useful,
> together with the information which frames have been seen in tcpdump but
> NOT in one of the log files. You can enable debug mode via -dn command
> line switch and is recommended to run rsyslog interactively while doing
> so. Then, you can simply capture its output via stdout redirection.
> Please note that debug mode generates considerable output, and requires
> considerable additional processing time. In any case, though, it should
> show us where the bottleneck is. Please note that I need a consistent
> excerpt from the debug log that shows how things began and how it worked
> during the fault conditions. Usually, this means I need everything ;)
> Debug logs may also reveal sensitive information, even passwords, so you
> should be careful in what you do. I am used to log files around the size
> of 1GB. With reasonable compression, the transfer is usually not a
> problem (but I suggest you place them on a server for me to download).
> Download links and/or smaller logs you can email me privately at
> rgerhards at gmail.com (please NOT at my primary, adiscon, email address).
>
> I hope this helps and I am looking forward for the additional
> information.

So after a long hiatus and a new config the problems went away but only
temporarily.  I think because of a rebooted box.  They have returned.  I'm
going to mail the logs your way.  I can send you more.  Its very easy to
reproduce.

	-Mike









============== Snip ===================
>
> On Wed, 2008-07-23 at 14:21 -0500, Mike McGrath wrote:
> > I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central
> > logging server.  When running tcpdump I'm seeing all the udp packets
> > coming in but many of them are not getting logged.  And we're talking
> > like 10% or so getting logged (maybe less) and the rest are just lost.
> > I've attached my config file.
> >
> > (side note, if I'm doing something stupid in the config please correct me)
> >
> > 	-Mike
> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>



From julianokyap at gmail.com  Wed Aug 13 12:32:00 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Wed, 13 Aug 2008 00:32:00 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
Message-ID: <d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>

It's kind of strange but I've had this running for about a week now...
 I seem to have had 2 false alerts for no apparent reason.


On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Thanks for the feedback, it will now be part of the next devel release,
> I think some time next week :)
>
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 08, 2008 1:55 AM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> Yep, after further testing this works great!  Thanks Rainer.
>>
>> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
>> wrote:
>> > Rainer,
>> >
>> > Initial testing looks fine.  I'll try some more to see if I can
> break
>> it.
>> >
>> > - Julian
>> >
>> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> >> Julian,
>> >>
>> >> as you have probably seen in my other post, I have implemented the
>> >> functionality. I have now also created a test tarball. I'd
>> appreciate if
>> >> you could obtain it from
>> >>
>> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>> >>
>> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> >> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout.
>> For
>> >> what you intend to do, this should work:
>> >>
>> >> $ActionExecOnlyEveryNthTime 3
>> >> *.* ..your action..
>> >>
>> >> You don't need the timeout, but I have included it for
> completeness.
>> >> Well, actually if I were you I'd think if you really don't need it.
>> Is
>> >> it really OK that "three in a row" means one each day?
>> >>
>> >> Please provide feedback on this feature.
>> >>
>> >> Thanks,
>> >> Rainer
>> >>
>> >>> -----Original Message-----
>> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >>> Sent: Friday, August 01, 2008 12:14 PM
>> >>> To: Rainer Gerhards
>> >>> Cc: rsyslog at lists.adiscon.com
>> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> >> found
>> >>>
>> >>> Roger that Rainer.
>> >>>
>> >>> Thanks,
>> >>> Julian
>> >>>
>> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> >>> <rgerhards at hq.adiscon.com> wrote:
>> >>> > OK, that greatly simplifies things. Actually, it now boils down
>> to
>> >>> > "execute an action only on the n-the time the filter evaluates
> to
>> >>> true".
>> >>> > I think this is quite easy to implement, but I must verify
>> that...
>> >>> >
>> >>> > Rainer
>> >>> >
>> >>> >> -----Original Message-----
>> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >>> >> Sent: Friday, August 01, 2008 11:03 AM
>> >>> >> To: Rainer Gerhards
>> >>> >> Cc: rsyslog at lists.adiscon.com
>> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
>> are
>> >>> > found
>> >>> >>
>> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> >>> >> <rgerhards at hq.adiscon.com> wrote:
>> >>> >> > Just one more re-confirmation:
>> >>> >> >
>> >>> >> >> What I'm looking for:
>> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
> then
>> >>> >> >> :ommail:;mailBody
>> >>> >> >>
>> >>> >> >> This would be nice but is not required since the 'This is
>> really
>> >>> >> bad'
>> >>> >> >> message in my case is very unique:
>> >>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>> >>> >> $program
>> >>> >> >> == 'program') happens 3 times in a row then
> :ommail:;mailBody
>> >>> >> >
>> >>> >> > So you would actually use such a rule. If "this other thing
> is
>> >>> > really
>> >>> >> > bad" happened three times, the rule shall not trigger. Is
> this
>> >>> > right?
>> >>> >>
>> >>> >> Yes, I would use such a rule.  It would make what is already an
>> >>> >> awesome application even more awesome. :P  I am also willing to
>> >> test
>> >>> >> it out and run the latest development version...  Which I'm
>> doing
>> >>> >> anyway.
>> >>> >>
>> >>> >> And yes, what you just wrote is correct.
>> >>> >>
>> >>> >> - Julian
>> >>> >
>> >>
>> >
>


From rgerhards at hq.adiscon.com  Wed Aug 13 12:47:36 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 12:47:36 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
	<d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>

Do you use $ActionExecOnlyEveryNthTimeTimeout?

Rainer


> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Wednesday, August 13, 2008 12:32 PM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> It's kind of strange but I've had this running for about a week now...
>  I seem to have had 2 false alerts for no apparent reason.
> 
> 
> On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Thanks for the feedback, it will now be part of the next devel
> release,
> > I think some time next week :)
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> Sent: Friday, August 08, 2008 1:55 AM
> >> To: Rainer Gerhards
> >> Cc: rsyslog at lists.adiscon.com
> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > found
> >>
> >> Yep, after further testing this works great!  Thanks Rainer.
> >>
> >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
> >> wrote:
> >> > Rainer,
> >> >
> >> > Initial testing looks fine.  I'll try some more to see if I can
> > break
> >> it.
> >> >
> >> > - Julian
> >> >
> >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> >> > <rgerhards at hq.adiscon.com> wrote:
> >> >> Julian,
> >> >>
> >> >> as you have probably seen in my other post, I have implemented
> the
> >> >> functionality. I have now also created a test tarball. I'd
> >> appreciate if
> >> >> you could obtain it from
> >> >>
> >> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> >> >>
> >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> >> >> $ActionExecOnlyEveryNthTime and
> $ActionExecOnlyEveryNthTimeTimeout.
> >> For
> >> >> what you intend to do, this should work:
> >> >>
> >> >> $ActionExecOnlyEveryNthTime 3
> >> >> *.* ..your action..
> >> >>
> >> >> You don't need the timeout, but I have included it for
> > completeness.
> >> >> Well, actually if I were you I'd think if you really don't need
> it.
> >> Is
> >> >> it really OK that "three in a row" means one each day?
> >> >>
> >> >> Please provide feedback on this feature.
> >> >>
> >> >> Thanks,
> >> >> Rainer
> >> >>
> >> >>> -----Original Message-----
> >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> >>> Sent: Friday, August 01, 2008 12:14 PM
> >> >>> To: Rainer Gerhards
> >> >>> Cc: rsyslog at lists.adiscon.com
> >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> are
> >> >> found
> >> >>>
> >> >>> Roger that Rainer.
> >> >>>
> >> >>> Thanks,
> >> >>> Julian
> >> >>>
> >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >> >>> <rgerhards at hq.adiscon.com> wrote:
> >> >>> > OK, that greatly simplifies things. Actually, it now boils
> down
> >> to
> >> >>> > "execute an action only on the n-the time the filter
evaluates
> > to
> >> >>> true".
> >> >>> > I think this is quite easy to implement, but I must verify
> >> that...
> >> >>> >
> >> >>> > Rainer
> >> >>> >
> >> >>> >> -----Original Message-----
> >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> >> >>> >> To: Rainer Gerhards
> >> >>> >> Cc: rsyslog at lists.adiscon.com
> >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
lines
> >> are
> >> >>> > found
> >> >>> >>
> >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >> >>> >> <rgerhards at hq.adiscon.com> wrote:
> >> >>> >> > Just one more re-confirmation:
> >> >>> >> >
> >> >>> >> >> What I'm looking for:
> >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
> > then
> >> >>> >> >> :ommail:;mailBody
> >> >>> >> >>
> >> >>> >> >> This would be nice but is not required since the 'This is
> >> really
> >> >>> >> bad'
> >> >>> >> >> message in my case is very unique:
> >> >>> >> >> if ($msg == 'This is really bad' and $server == 'server'
> and
> >> >>> >> $program
> >> >>> >> >> == 'program') happens 3 times in a row then
> > :ommail:;mailBody
> >> >>> >> >
> >> >>> >> > So you would actually use such a rule. If "this other
thing
> > is
> >> >>> > really
> >> >>> >> > bad" happened three times, the rule shall not trigger. Is
> > this
> >> >>> > right?
> >> >>> >>
> >> >>> >> Yes, I would use such a rule.  It would make what is already
> an
> >> >>> >> awesome application even more awesome. :P  I am also willing
> to
> >> >> test
> >> >>> >> it out and run the latest development version...  Which I'm
> >> doing
> >> >>> >> anyway.
> >> >>> >>
> >> >>> >> And yes, what you just wrote is correct.
> >> >>> >>
> >> >>> >> - Julian
> >> >>> >
> >> >>
> >> >
> >


From r.bhatia at ipax.at  Wed Aug 13 13:04:07 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Aug 2008 13:04:07 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
Message-ID: <48A2BFA7.8000408@ipax.at>

hi,

Rainer Gerhards wrote:
> Thanks, I now received the log files. And, of course, they made
> immediately clear what is going on ;) What you experience is flow
> control kicking in. Let me first elaborate a bit:
> 
> Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> the local socket) and those who can not (like UDP). The idea is that we
> can slow down senders where this does not result in loss of messages,
> leaving room inside the queue on busy systems for those where we can not
> do that. We had cases where not delaying caused a lot of grief.
> 
> There are also two watermarks, the low and high delay mark. If the low
> delay mark is reached, a delayable source is delayed for one second. It
> is at 70% of the (in-memory) queue capacity. Again, this is all done for
> some very good reasons.
> 
> However, when I introduced these features, I did not make them
> configurable. Also, I did classify the local unix socket as one who can
> be delayed. In your scenario, this does not stop the system, but once
> the local log socket is filled up, it needs to wait one second for each
> message before it is pulled.
> 
> Again, the rate limiting logic is there for good reasons. So what I now
> see is that I need to add a few config statements to allow to control it
> according to your needs. I am also unsure if I should re-classify
> imuxsock as an input which cannot be delayed - but that also can have
> some drawbacks. Maybe the best thing to do is make its state
> configurable and use "not to be delayed" as default.

ok, after reading through your explanation, i (almost) understand the
delay mechanism and that at present it is not configurable.

almost, because i do not get the part where you talk about the "70% of
the (in-memory) queue capacity".

anyways, how does this relate to the original problem, that the
following code stalls the logging process.

>>> #! /bin/sh
>>> for ((i=1;i<=10000;i+=1)); do
>>>   logger Log line $i
>>> done

is this a capacity problem? is this another problem?

i really consider this a critical bug, as i yesterday ran out of
diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
rsyslog-client, which logs locally and to the remote destination,
stalled completely.

cheers,
raoul
ps: i am on 3.16.1-1~bpo40+1
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From rgerhards at hq.adiscon.com  Wed Aug 13 14:33:00 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 14:33:00 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <48A2BFA7.8000408@ipax.at>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
	<48A2BFA7.8000408@ipax.at>
Message-ID: <1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>

On Wed, 2008-08-13 at 13:04 +0200, Raoul Bhatia [IPAX] wrote:
> hi,
> 
> Rainer Gerhards wrote:
> > Thanks, I now received the log files. And, of course, they made
> > immediately clear what is going on ;) What you experience is flow
> > control kicking in. Let me first elaborate a bit:
> > 
> > Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> > the local socket) and those who can not (like UDP). The idea is that we
> > can slow down senders where this does not result in loss of messages,
> > leaving room inside the queue on busy systems for those where we can not
> > do that. We had cases where not delaying caused a lot of grief.
> > 
> > There are also two watermarks, the low and high delay mark. If the low
> > delay mark is reached, a delayable source is delayed for one second. It
> > is at 70% of the (in-memory) queue capacity. Again, this is all done for
> > some very good reasons.
> > 
> > However, when I introduced these features, I did not make them
> > configurable. Also, I did classify the local unix socket as one who can
> > be delayed. In your scenario, this does not stop the system, but once
> > the local log socket is filled up, it needs to wait one second for each
> > message before it is pulled.
> > 
> > Again, the rate limiting logic is there for good reasons. So what I now
> > see is that I need to add a few config statements to allow to control it
> > according to your needs. I am also unsure if I should re-classify
> > imuxsock as an input which cannot be delayed - but that also can have
> > some drawbacks. Maybe the best thing to do is make its state
> > configurable and use "not to be delayed" as default.
> 
> ok, after reading through your explanation, i (almost) understand the
> delay mechanism and that at present it is not configurable.
> 
> almost, because i do not get the part where you talk about the "70% of
> the (in-memory) queue capacity".

An example: if the configured queue capacity is 10,000 (the default
value for the main message queue), than no flow control at all happens
until 7,000 messages are inside the queue. Once they are (and only as
long as they are), flow control is applied to those inputs that permit
it (imuxsock permits it).

It doesn't matter here if the queue is disk-assisted or not. Even with
disk-assisted queues, this setting is very important to prevent an input
(e.g. imfile!) to send massive amounts of data to the queue when there
is no need to (because the input can be flow-controlled).

> 
> anyways, how does this relate to the original problem, that the
> following code stalls the logging process.
> 
> >>> #! /bin/sh
> >>> for ((i=1;i<=10000;i+=1)); do
> >>>   logger Log line $i
> >>> done
> 
> is this a capacity problem? is this another problem?

It is going over the flow control watermark, flow control kicks in and
delays imuxsock writes by one second each as long as there are more than
7,000 messages. No problem, intended behavior, but obviously
questionable. In the imuxsock case, there are also some bad cases.

> 
> i really consider this a critical bug, as i yesterday ran out of
> diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
> rsyslog-client, which logs locally and to the remote destination,
> stalled completely.

We'll - not completely, one message per second ;) But that's probably
complete enough ;)

As a quick fix, I think I'll make imuxsock a non-flow-controllable input
in the v3-stable (3.18.x). The devel will have a switch to make it flow
controlled if one desires. I'll also add some other knobs to fine-tune
the flow control algo, but that probably needs to wait until after my
summer break. These tweaks could easily break things and I don't like
the idea of doing them under time pressure ;)

HTH
Rainer
> 
> cheers,
> raoul
> ps: i am on 3.16.1-1~bpo40+1



From rgerhards at hq.adiscon.com  Wed Aug 13 14:39:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 14:39:44 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com><48A2BFA7.8000408@ipax.at>
	<1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>

This is the v3-stable fix:

http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=plugins/imuxsock/imux
sock.c;h=60ccaffbc526a03011be3ce5869e40bc3e32c319;hp=f8798039c5d53e58fc7
fa17807547862e08c999e;hb=8eb888d049da12e1294a7688432b6325794ade32;hpb=02
f768c37dac9dde424bbd31e378482750fc276c

You can also easily apply it to 3.16 - I just won't do it that version
has been superseded.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, August 13, 2008 2:33 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> On Wed, 2008-08-13 at 13:04 +0200, Raoul Bhatia [IPAX] wrote:
> > hi,
> >
> > Rainer Gerhards wrote:
> > > Thanks, I now received the log files. And, of course, they made
> > > immediately clear what is going on ;) What you experience is flow
> > > control kicking in. Let me first elaborate a bit:
> > >
> > > Rsyslog knows inputs which can be delayed a bit (like tcp receiver
> AND
> > > the local socket) and those who can not (like UDP). The idea is
> that we
> > > can slow down senders where this does not result in loss of
> messages,
> > > leaving room inside the queue on busy systems for those where we
> can not
> > > do that. We had cases where not delaying caused a lot of grief.
> > >
> > > There are also two watermarks, the low and high delay mark. If the
> low
> > > delay mark is reached, a delayable source is delayed for one
> second. It
> > > is at 70% of the (in-memory) queue capacity. Again, this is all
> done for
> > > some very good reasons.
> > >
> > > However, when I introduced these features, I did not make them
> > > configurable. Also, I did classify the local unix socket as one
who
> can
> > > be delayed. In your scenario, this does not stop the system, but
> once
> > > the local log socket is filled up, it needs to wait one second for
> each
> > > message before it is pulled.
> > >
> > > Again, the rate limiting logic is there for good reasons. So what
I
> now
> > > see is that I need to add a few config statements to allow to
> control it
> > > according to your needs. I am also unsure if I should re-classify
> > > imuxsock as an input which cannot be delayed - but that also can
> have
> > > some drawbacks. Maybe the best thing to do is make its state
> > > configurable and use "not to be delayed" as default.
> >
> > ok, after reading through your explanation, i (almost) understand
the
> > delay mechanism and that at present it is not configurable.
> >
> > almost, because i do not get the part where you talk about the "70%
> of
> > the (in-memory) queue capacity".
> 
> An example: if the configured queue capacity is 10,000 (the default
> value for the main message queue), than no flow control at all happens
> until 7,000 messages are inside the queue. Once they are (and only as
> long as they are), flow control is applied to those inputs that permit
> it (imuxsock permits it).
> 
> It doesn't matter here if the queue is disk-assisted or not. Even with
> disk-assisted queues, this setting is very important to prevent an
> input
> (e.g. imfile!) to send massive amounts of data to the queue when there
> is no need to (because the input can be flow-controlled).
> 
> >
> > anyways, how does this relate to the original problem, that the
> > following code stalls the logging process.
> >
> > >>> #! /bin/sh
> > >>> for ((i=1;i<=10000;i+=1)); do
> > >>>   logger Log line $i
> > >>> done
> >
> > is this a capacity problem? is this another problem?
> 
> It is going over the flow control watermark, flow control kicks in and
> delays imuxsock writes by one second each as long as there are more
> than
> 7,000 messages. No problem, intended behavior, but obviously
> questionable. In the imuxsock case, there are also some bad cases.
> 
> >
> > i really consider this a critical bug, as i yesterday ran out of
> > diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
> > rsyslog-client, which logs locally and to the remote destination,
> > stalled completely.
> 
> We'll - not completely, one message per second ;) But that's probably
> complete enough ;)
> 
> As a quick fix, I think I'll make imuxsock a non-flow-controllable
> input
> in the v3-stable (3.18.x). The devel will have a switch to make it
flow
> controlled if one desires. I'll also add some other knobs to fine-tune
> the flow control algo, but that probably needs to wait until after my
> summer break. These tweaks could easily break things and I don't like
> the idea of doing them under time pressure ;)
> 
> HTH
> Rainer
> >
> > cheers,
> > raoul
> > ps: i am on 3.16.1-1~bpo40+1
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From r.bhatia at ipax.at  Wed Aug 13 15:13:48 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Aug 2008 15:13:48 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com><48A2BFA7.8000408@ipax.at>	<1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>
Message-ID: <48A2DE0C.6010502@ipax.at>

Rainer Gerhards wrote:
> This is the v3-stable fix:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=plugins/imuxsock/imux
> sock.c;h=60ccaffbc526a03011be3ce5869e40bc3e32c319;hp=f8798039c5d53e58fc7
> fa17807547862e08c999e;hb=8eb888d049da12e1294a7688432b6325794ade32;hpb=02
> f768c37dac9dde424bbd31e378482750fc276c

thank you for your prompt fix. i will try to simulate the failure again.

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From rgerhards at hq.adiscon.com  Thu Aug 14 11:31:03 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Aug 2008 11:31:03 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com><d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com><d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com><d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>

Just to keep everyone update. I got a private mail with some details not
to be posted publically. However, Julian stated that he does not use
$ActionExecOnlyEveryNthTimeTimeout.

With that (and the non-public) info, I think that the missing
$ActionExecOnlyEveryNthTimeTimeout is the culprit. I have not asked how
often the receiving syslogd is restarted or huped. Julian, please let me
know. If it is sufficiently seldom, I think it could be an "old"
message.

I think about this scenario: a message comes it at, eg., 10am. That's
count 1. Then, nothing happens for the next 10 hours. At 8pm, the next
one occurs. That's count 2. Another 5 hours later, the next message
occurs, bringing the total count to 3. Thus, this message now triggers
the rule.

The question is if this is desired behavior? Or should the rule only be
triggered if the messages occur within an e.g. 20 minute window? If the
later is the case, you need a

$ActionExecOnlyEveryNthTimeTimeout 1200

This directive will timeout previous messages seen if they are older
than 20 minutes. In the example above, the count would now be always 1
and consequently no rule would ever be triggered (and that's the reason
why this directive is present;)).

Does this make sense? Does it match what you see?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, August 13, 2008 12:48 PM
> To: Julian Yap
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Do you use $ActionExecOnlyEveryNthTimeTimeout?
> 
> Rainer
> 
> 
> > -----Original Message-----
> > From: Julian Yap [mailto:julianokyap at gmail.com]
> > Sent: Wednesday, August 13, 2008 12:32 PM
> > To: Rainer Gerhards
> > Cc: rsyslog at lists.adiscon.com
> > Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
> >
> > It's kind of strange but I've had this running for about a week
> now...
> >  I seem to have had 2 false alerts for no apparent reason.
> >
> >
> > On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> > > Thanks for the feedback, it will now be part of the next devel
> > release,
> > > I think some time next week :)
> > >
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> Sent: Friday, August 08, 2008 1:55 AM
> > >> To: Rainer Gerhards
> > >> Cc: rsyslog at lists.adiscon.com
> > >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > > found
> > >>
> > >> Yep, after further testing this works great!  Thanks Rainer.
> > >>
> > >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap
> <julianokyap at gmail.com>
> > >> wrote:
> > >> > Rainer,
> > >> >
> > >> > Initial testing looks fine.  I'll try some more to see if I can
> > > break
> > >> it.
> > >> >
> > >> > - Julian
> > >> >
> > >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> > >> > <rgerhards at hq.adiscon.com> wrote:
> > >> >> Julian,
> > >> >>
> > >> >> as you have probably seen in my other post, I have implemented
> > the
> > >> >> functionality. I have now also created a test tarball. I'd
> > >> appreciate if
> > >> >> you could obtain it from
> > >> >>
> > >> >>
http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> > >> >>
> > >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> > >> >> $ActionExecOnlyEveryNthTime and
> > $ActionExecOnlyEveryNthTimeTimeout.
> > >> For
> > >> >> what you intend to do, this should work:
> > >> >>
> > >> >> $ActionExecOnlyEveryNthTime 3
> > >> >> *.* ..your action..
> > >> >>
> > >> >> You don't need the timeout, but I have included it for
> > > completeness.
> > >> >> Well, actually if I were you I'd think if you really don't
need
> > it.
> > >> Is
> > >> >> it really OK that "three in a row" means one each day?
> > >> >>
> > >> >> Please provide feedback on this feature.
> > >> >>
> > >> >> Thanks,
> > >> >> Rainer
> > >> >>
> > >> >>> -----Original Message-----
> > >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> >>> Sent: Friday, August 01, 2008 12:14 PM
> > >> >>> To: Rainer Gerhards
> > >> >>> Cc: rsyslog at lists.adiscon.com
> > >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> > are
> > >> >> found
> > >> >>>
> > >> >>> Roger that Rainer.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> Julian
> > >> >>>
> > >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > >> >>> <rgerhards at hq.adiscon.com> wrote:
> > >> >>> > OK, that greatly simplifies things. Actually, it now boils
> > down
> > >> to
> > >> >>> > "execute an action only on the n-the time the filter
> evaluates
> > > to
> > >> >>> true".
> > >> >>> > I think this is quite easy to implement, but I must verify
> > >> that...
> > >> >>> >
> > >> >>> > Rainer
> > >> >>> >
> > >> >>> >> -----Original Message-----
> > >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> > >> >>> >> To: Rainer Gerhards
> > >> >>> >> Cc: rsyslog at lists.adiscon.com
> > >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
> lines
> > >> are
> > >> >>> > found
> > >> >>> >>
> > >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> > >> >>> >> <rgerhards at hq.adiscon.com> wrote:
> > >> >>> >> > Just one more re-confirmation:
> > >> >>> >> >
> > >> >>> >> >> What I'm looking for:
> > >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a
row
> > > then
> > >> >>> >> >> :ommail:;mailBody
> > >> >>> >> >>
> > >> >>> >> >> This would be nice but is not required since the 'This
> is
> > >> really
> > >> >>> >> bad'
> > >> >>> >> >> message in my case is very unique:
> > >> >>> >> >> if ($msg == 'This is really bad' and $server ==
'server'
> > and
> > >> >>> >> $program
> > >> >>> >> >> == 'program') happens 3 times in a row then
> > > :ommail:;mailBody
> > >> >>> >> >
> > >> >>> >> > So you would actually use such a rule. If "this other
> thing
> > > is
> > >> >>> > really
> > >> >>> >> > bad" happened three times, the rule shall not trigger.
Is
> > > this
> > >> >>> > right?
> > >> >>> >>
> > >> >>> >> Yes, I would use such a rule.  It would make what is
> already
> > an
> > >> >>> >> awesome application even more awesome. :P  I am also
> willing
> > to
> > >> >> test
> > >> >>> >> it out and run the latest development version...  Which
I'm
> > >> doing
> > >> >>> >> anyway.
> > >> >>> >>
> > >> >>> >> And yes, what you just wrote is correct.
> > >> >>> >>
> > >> >>> >> - Julian
> > >> >>> >
> > >> >>
> > >> >
> > >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From julianokyap at gmail.com  Thu Aug 14 12:30:02 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 14 Aug 2008 00:30:02 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
	<d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>
Message-ID: <d66707280808140330i77592c8fk56da76125736da63@mail.gmail.com>

Rainer,

Yes, after looking at it some more, I guess I do need to have the
$ActionExecOnlyEveryNthTimeTimeout setting.  Your explanation below
clarifies things a lot.

I guess I was confused with my existing
$ActionExecOnlyOnceEveryInterval setting and thinking that was
similar.

That should fix my issue.

- Julian

On Wed, Aug 13, 2008 at 11:31 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just to keep everyone update. I got a private mail with some details not
> to be posted publically. However, Julian stated that he does not use
> $ActionExecOnlyEveryNthTimeTimeout.
>
> With that (and the non-public) info, I think that the missing
> $ActionExecOnlyEveryNthTimeTimeout is the culprit. I have not asked how
> often the receiving syslogd is restarted or huped. Julian, please let me
> know. If it is sufficiently seldom, I think it could be an "old"
> message.
>
> I think about this scenario: a message comes it at, eg., 10am. That's
> count 1. Then, nothing happens for the next 10 hours. At 8pm, the next
> one occurs. That's count 2. Another 5 hours later, the next message
> occurs, bringing the total count to 3. Thus, this message now triggers
> the rule.
>
> The question is if this is desired behavior? Or should the rule only be
> triggered if the messages occur within an e.g. 20 minute window? If the
> later is the case, you need a
>
> $ActionExecOnlyEveryNthTimeTimeout 1200
>
> This directive will timeout previous messages seen if they are older
> than 20 minutes. In the example above, the count would now be always 1
> and consequently no rule would ever be triggered (and that's the reason
> why this directive is present;)).
>
> Does this make sense? Does it match what you see?
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Wednesday, August 13, 2008 12:48 PM
>> To: Julian Yap
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> Do you use $ActionExecOnlyEveryNthTimeTimeout?
>>
>> Rainer
>>
>>
>> > -----Original Message-----
>> > From: Julian Yap [mailto:julianokyap at gmail.com]
>> > Sent: Wednesday, August 13, 2008 12:32 PM
>> > To: Rainer Gerhards
>> > Cc: rsyslog at lists.adiscon.com
>> > Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> found
>> >
>> > It's kind of strange but I've had this running for about a week
>> now...
>> >  I seem to have had 2 false alerts for no apparent reason.
>> >
>> >
>> > On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> > > Thanks for the feedback, it will now be part of the next devel
>> > release,
>> > > I think some time next week :)
>> > >
>> > > Rainer
>> > >
>> > >> -----Original Message-----
>> > >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> Sent: Friday, August 08, 2008 1:55 AM
>> > >> To: Rainer Gerhards
>> > >> Cc: rsyslog at lists.adiscon.com
>> > >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> > > found
>> > >>
>> > >> Yep, after further testing this works great!  Thanks Rainer.
>> > >>
>> > >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap
>> <julianokyap at gmail.com>
>> > >> wrote:
>> > >> > Rainer,
>> > >> >
>> > >> > Initial testing looks fine.  I'll try some more to see if I can
>> > > break
>> > >> it.
>> > >> >
>> > >> > - Julian
>> > >> >
>> > >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
>> > >> > <rgerhards at hq.adiscon.com> wrote:
>> > >> >> Julian,
>> > >> >>
>> > >> >> as you have probably seen in my other post, I have implemented
>> > the
>> > >> >> functionality. I have now also created a test tarball. I'd
>> > >> appreciate if
>> > >> >> you could obtain it from
>> > >> >>
>> > >> >>
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>> > >> >>
>> > >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> > >> >> $ActionExecOnlyEveryNthTime and
>> > $ActionExecOnlyEveryNthTimeTimeout.
>> > >> For
>> > >> >> what you intend to do, this should work:
>> > >> >>
>> > >> >> $ActionExecOnlyEveryNthTime 3
>> > >> >> *.* ..your action..
>> > >> >>
>> > >> >> You don't need the timeout, but I have included it for
>> > > completeness.
>> > >> >> Well, actually if I were you I'd think if you really don't
> need
>> > it.
>> > >> Is
>> > >> >> it really OK that "three in a row" means one each day?
>> > >> >>
>> > >> >> Please provide feedback on this feature.
>> > >> >>
>> > >> >> Thanks,
>> > >> >> Rainer
>> > >> >>
>> > >> >>> -----Original Message-----
>> > >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> >>> Sent: Friday, August 01, 2008 12:14 PM
>> > >> >>> To: Rainer Gerhards
>> > >> >>> Cc: rsyslog at lists.adiscon.com
>> > >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
>> > are
>> > >> >> found
>> > >> >>>
>> > >> >>> Roger that Rainer.
>> > >> >>>
>> > >> >>> Thanks,
>> > >> >>> Julian
>> > >> >>>
>> > >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> > >> >>> <rgerhards at hq.adiscon.com> wrote:
>> > >> >>> > OK, that greatly simplifies things. Actually, it now boils
>> > down
>> > >> to
>> > >> >>> > "execute an action only on the n-the time the filter
>> evaluates
>> > > to
>> > >> >>> true".
>> > >> >>> > I think this is quite easy to implement, but I must verify
>> > >> that...
>> > >> >>> >
>> > >> >>> > Rainer
>> > >> >>> >
>> > >> >>> >> -----Original Message-----
>> > >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
>> > >> >>> >> To: Rainer Gerhards
>> > >> >>> >> Cc: rsyslog at lists.adiscon.com
>> > >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
>> lines
>> > >> are
>> > >> >>> > found
>> > >> >>> >>
>> > >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> > >> >>> >> <rgerhards at hq.adiscon.com> wrote:
>> > >> >>> >> > Just one more re-confirmation:
>> > >> >>> >> >
>> > >> >>> >> >> What I'm looking for:
>> > >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a
> row
>> > > then
>> > >> >>> >> >> :ommail:;mailBody
>> > >> >>> >> >>
>> > >> >>> >> >> This would be nice but is not required since the 'This
>> is
>> > >> really
>> > >> >>> >> bad'
>> > >> >>> >> >> message in my case is very unique:
>> > >> >>> >> >> if ($msg == 'This is really bad' and $server ==
> 'server'
>> > and
>> > >> >>> >> $program
>> > >> >>> >> >> == 'program') happens 3 times in a row then
>> > > :ommail:;mailBody
>> > >> >>> >> >
>> > >> >>> >> > So you would actually use such a rule. If "this other
>> thing
>> > > is
>> > >> >>> > really
>> > >> >>> >> > bad" happened three times, the rule shall not trigger.
> Is
>> > > this
>> > >> >>> > right?
>> > >> >>> >>
>> > >> >>> >> Yes, I would use such a rule.  It would make what is
>> already
>> > an
>> > >> >>> >> awesome application even more awesome. :P  I am also
>> willing
>> > to
>> > >> >> test
>> > >> >>> >> it out and run the latest development version...  Which
> I'm
>> > >> doing
>> > >> >>> >> anyway.
>> > >> >>> >>
>> > >> >>> >> And yes, what you just wrote is correct.
>> > >> >>> >>
>> > >> >>> >> - Julian
>> > >> >>> >
>> > >> >>
>> > >> >
>> > >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From klas.lindfors at it.su.se  Fri Aug 15 09:37:14 2008
From: klas.lindfors at it.su.se (Klas Lindfors)
Date: Fri, 15 Aug 2008 09:37:14 +0200
Subject: [rsyslog] patch adding option for testing configuration
Message-ID: <48A5322A.9080400@it.su.se>

Hello,

I've attached a patch to this message adding a config option (-T) to run 
rsyslog with configuration testing, immediatly exiting with 0 if the 
conf is valid. For us this is an important feature to make sure a 
configuration is valid before starting the daemon.

The style of the patch is probably not acceptable for inclusion, but it 
does it's job, so maybe a variant of it can be included.

Regards,

Klas Lindfors
Stockholm University

From rgerhards at hq.adiscon.com  Fri Aug 15 11:37:15 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Aug 2008 11:37:15 +0200
Subject: [rsyslog] patch adding option for testing configuration
In-Reply-To: <48A5322A.9080400@it.su.se>
References: <48A5322A.9080400@it.su.se>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFF0@grfint2.intern.adiscon.com>

Hi Klas,

the patch did not make it to the mailing list (the list processor is
very picky to prevent spreading spam and malware). However, from you
description it sounds like the -N option which has recently (3.21.1)
been introduced. Please see

http://www.rsyslog.com/doc-troubleshoot.html

under "configuration problems". I don't know out of my head if the exit
code works in the way you describe, but that should be an easy add. I
suggest you file an enhancement request if in doubt
(http://bugzilla.adiscon.com). I'll be soon away for summer break, so
that needs to wait until after it ;)

Please let me know if -N goes into the right direction.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Klas Lindfors
> Sent: Friday, August 15, 2008 9:37 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] patch adding option for testing configuration
> 
> Hello,
> 
> I've attached a patch to this message adding a config option (-T) to
> run
> rsyslog with configuration testing, immediatly exiting with 0 if the
> conf is valid. For us this is an important feature to make sure a
> configuration is valid before starting the daemon.
> 
> The style of the patch is probably not acceptable for inclusion, but
it
> does it's job, so maybe a variant of it can be included.
> 
> Regards,
> 
> Klas Lindfors
> Stockholm University


From friedl at hq.adiscon.com  Fri Aug 15 14:07:34 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 15 Aug 2008 14:07:34 +0200
Subject: [rsyslog] rsyslog 3.21.3 (devel) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFFF@grfint2.intern.adiscon.com>

Hi all,

I have just released 3.21.3, a devel branch version. It provides support
for specifying custom MySQL server ports and adds the ability to execute
an action only ever n-th time. Also, flow control for imuxsock messages
can now be turned on or off. There are also some bug fixes, among others
one to fully support IPv6 hosts in forwarding actions. This is a
recommended update for all devel branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-127.phtml

Changelog:

http://www.rsyslog.com/Article270.phtml

As always, feedback is appreciated.

Florian Riedl


From friedl at hq.adiscon.com  Mon Aug 18 17:16:49 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 18 Aug 2008 17:16:49 +0200
Subject: [rsyslog] rsyslog 3.18.3 released (stable)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F006@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 3.18.3, a member of the v3-stable branch,
today. This is primarily a bug-fixing release. It fixes a problem that
causes imfile to segfault on termination, disables flow control in
imuxsock which could lead to system hang conditions and corrects a
misspelled configuration directive. There is a small enhancement which
now permits to use MySQL custom server ports. This is a recommended
update for all v3-stable branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-128.phtml

Changelog:

http://www.rsyslog.com/Article271.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From stefan.wold at it.su.se  Wed Aug 20 07:40:19 2008
From: stefan.wold at it.su.se (Stefan Wold)
Date: Wed, 20 Aug 2008 07:40:19 +0200
Subject: [rsyslog] rsyslog with RELP resolving issue
Message-ID: <1219210819.4967.18.camel@localhost>

Hi,

I've been testing rsyslog versions 3.18.2 and 3.21.3 with RELP. My
problem here is that no matter what I try I can't get the receiving
rsyslog to do hostname resolving. If I disable RELP and go back to
either tcp or udp it will resolve my hosts properly. On the receiving
side I have a template that create one log file per host with the
%HOSTNAME% variable, if I try using %fromhost% and RELP I end up with an
unresolved ip-address.

Is there a hidden option I'm not aware of or is resolving just not
working properly with RELP yet?

Secondly I wonder if adding a option that force rsyslog to use FQDN for
everything including hostnames within the same domain as the logserver
would be plausible? 

-- 
Sincerely,
Stefan Wold
IT Services, Stockholm University, Sweden



From satoru.satoh at gmail.com  Wed Aug 20 17:31:19 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 00:31:19 +0900
Subject: [rsyslog] [PATCH 00/07] man-i18n patches
Message-ID: <20080820153118.GC20566@localhost.localdomain>

Hello / Guten tag,


Rio-san (rfujita at redhat.com) previously told about his idea of
documentation i18n / l10n for rsyslog. I'm one of his colleagues and
originally told him about related topics.


A series of patches I'll post right after this mail are to implement
this i18n / l10n workflow for man pages into normal build process.

These patches enable to 

 * generate original English man pages from DocBook XML files
 * generate translated non-English man pages from translated DocBook
   XML files, which can be generated from PO files (PO is much suitable
   format for translation maintainace than XML, I guess)
 * create POT (PO template)

And this automated man-pages-generation process is only performed when
the configure option "--enable-regenerate-man" is specified. So, side
effects of these patches should be kept to a minimum, I think.

- satoru


From satoru.satoh at gmail.com  Wed Aug 20 19:07:40 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 02:07:40 +0900
Subject: [rsyslog] [PATCH] man-i18n patches summary
Message-ID: <20080820170735.GB20932@localhost.localdomain>

Hello,


I posted the first patch but it seems the patch was too big to post. I
list them here at once.

Could you please take some time to review these and merge them?


I splitted origianl big patch into minimum units. However, to complete
to man-i18n support, the following patches must be applied in this
order.

All of the patches below are available from
http://www.gnome.gr.jp/~ss/t/rsyslog/.


1. 0001-man-i18n-move-man-files-from-tools-to-man-C.patch

This patch just moves man files from tools/ to man/C/ (new dir to hold
man related stuff) and update related Makefile.am files.


2. 0002-man-i18n-add-man-ja.patch

This patch creates new dir, man/ja/ and Makefile.am in it.

There are no man files in it and this should not have harmful effects
for build process at all, I guess.


3. 0003-man-i18n-add-man-ja.patch

This patch adds man files in man/ja and update Makefile.am in it to
install these.


4. 0004-man-i18n-add-docbook-xml-sources-for-man-files.patch

This patch adds docbook XML sources for man pages.


5. 0005-man-i18n-add-m4-macros-in-m4-and-make-aclocal-look.patch

This patch adds m4 macros in m4/ and make aclocal look for this dir.

These m4 files are not evaluated in actual and this should not have
adverse effect at all, I guess.


6. 0006-man-i18n-add-build-process-of-man-pages-in-Makefile.patch

This patch adds transformation process of man pages and po files in
Makefile.am.

Added part are in "if ENABLE_REGENERATE_MAN ... endif" and
ENABLE_REGENERATE_MAN is not defined at this point, so this also should
not have side effects.


7. 0007-man-i18n-add-enable-regenerate-man-option-for-con.patch

This is the last one. This patch adds "--enable-regenerate-man"
configure option to enable man build process.

This option is disabled by default and should not affect normal build
process, I think.


How to build:

To enable the man build process, configure have to be regenerated:

  $ (autoreconf -vfi && \
        ./configure --enable-regenerate-man [...other options] && \
        make && make install


Translator's workflow:

 a. initial translation:

   0. arrange lang dir, that is, cp man/ja to man/YOUR_LANG/ and edit
      Makefile.am; s/ja/YOUR_LANG/.
   1. create POT: make -C man/YOUR_LANG/ create-pot
   2. copy POT to PO and translate it
   3. run make

 b. maintanance:

   1. update PO: make -C man/YOUR_LANG update-po
   2. update translations
   3. just run make


Requirements for man regeneration / translation:
 
   xmllint, xsltproc, xml2po in gnome-doc-utils, DocBook DTD and DocBook
   xslt stylesheet for man pages are required.

   If xmlcatalog (e.g. /etc/xml/catalog) is configured appropriately,
   both xmllint and xsltproc will use local DTD and stylsheet files
   instead of these from remote site
   (http://www.oasis-open.org/docbook/...).


- satoru


From satoru.satoh at gmail.com  Wed Aug 20 19:25:24 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 02:25:24 +0900
Subject: [rsyslog] [PATCH] man-i18n patches summary
In-Reply-To: <20080820170735.GB20932@localhost.localdomain>
References: <20080820170735.GB20932@localhost.localdomain>
Message-ID: <20080820172520.GC20932@localhost.localdomain>

Hello,


On Thu, Aug 21, 2008 at 02:07:40AM +0900, Satoru SATOH wrote:
> Could you please take some time to review these and merge them?

Of course, my patches change many and don't think easy to merge nor am
not sure accetable enhancement in the first place.

Please let me know if you have any issues on them, I'll work to fix
them as far as I can.

- satoru


From rgerhards at hq.adiscon.com  Wed Aug 20 21:25:42 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Aug 2008 21:25:42 +0200
Subject: [rsyslog] [PATCH 00/07] man-i18n patches
In-Reply-To: <20080820153118.GC20566@localhost.localdomain>
References: <20080820153118.GC20566@localhost.localdomain>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F018@grfint2.intern.adiscon.com>

Hi Satoru-san,

Your contribution is greatly appreciated. But I am currently on summer
vacation end will return end of month. Then I'll have a look at the
patch :)

Thanks,
Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Satoru SATOH
> Sent: Wednesday, August 20, 2008 5:31 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] [PATCH 00/07] man-i18n patches
> 
> Hello / Guten tag,
> 
> 
> Rio-san (rfujita at redhat.com) previously told about his idea of
> documentation i18n / l10n for rsyslog. I'm one of his colleagues and
> originally told him about related topics.
> 
> 
> A series of patches I'll post right after this mail are to implement
> this i18n / l10n workflow for man pages into normal build process.
> 
> These patches enable to 
> 
>  * generate original English man pages from DocBook XML files
>  * generate translated non-English man pages from translated DocBook
>    XML files, which can be generated from PO files (PO is 
> much suitable
>    format for translation maintainace than XML, I guess)
>  * create POT (PO template)
> 
> And this automated man-pages-generation process is only performed when
> the configure option "--enable-regenerate-man" is specified. So, side
> effects of these patches should be kept to a minimum, I think.
> 
> - satoru
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> 


From jeffschroed at gmail.com  Thu Aug 21 19:33:51 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Thu, 21 Aug 2008 10:33:51 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
Message-ID: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>

Rsyslog seems to be mangling messages sent from our in-house applications.
We are trying to get the same format as we did with syslog-ng previously and
not having a lot of luck.

The logs are in the format something like this:
TAG APPLICATION: MSG

In syslog-ng, it looks like this locally and when sent to a remote
syslog-ng server:
Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
ops-192.168.101.251:65261 offset changed 0.0004593 seconds!

rsyslog with the TraditionalFileFormat makes it look like this
locally, stripping the hostname:
Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
0.0004593 seconds!

and like this when sent to a remote syslog-ng server. Notice how the
ADMIN tag was removed:
Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
offset changed 0.0004593 seconds!

We have realtime logscraping software that looks at all message tags
and does custom reporting on them.
It keys off the TAG property that rsyslog is stripping off. What can
we do to enable this? We've been playing
with custom $templates in the rsyslog.conf to no real avail.

Thanks!

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From hks.private at gmail.com  Thu Aug 21 19:46:43 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 21 Aug 2008 13:46:43 -0400
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
Message-ID: <ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>

I'm not familiar with syslog-ng, but I suspect this is just a
difference in the formatting/interpretation of the raw packet. Try
capturing a message or two from syslog-ng with tcpdump and compare to
section 4 of RFC 3164.

You can see how rsyslog interprets it with the following template:
$template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
PROGRAM: %programname% MSG: %rawmsg%\n"

This should give you some idea of how to create a template that will
log in the format you need.

-HKS



On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> Rsyslog seems to be mangling messages sent from our in-house applications.
> We are trying to get the same format as we did with syslog-ng previously and
> not having a lot of luck.
>
> The logs are in the format something like this:
> TAG APPLICATION: MSG
>
> In syslog-ng, it looks like this locally and when sent to a remote
> syslog-ng server:
> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>
> rsyslog with the TraditionalFileFormat makes it look like this
> locally, stripping the hostname:
> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
> 0.0004593 seconds!
>
> and like this when sent to a remote syslog-ng server. Notice how the
> ADMIN tag was removed:
> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
> offset changed 0.0004593 seconds!
>
> We have realtime logscraping software that looks at all message tags
> and does custom reporting on them.
> It keys off the TAG property that rsyslog is stripping off. What can
> we do to enable this? We've been playing
> with custom $templates in the rsyslog.conf to no real avail.
>
> Thanks!
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From rgerhards at hq.adiscon.com  Thu Aug 21 19:53:39 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 21 Aug 2008 19:53:39 +0200
Subject: [rsyslog] Problems migrating from syslog-ng
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>

I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

Rainer


-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com on behalf of (private) HKS
Sent: Thu 8/21/2008 7:46 PM
To: rsyslog-users
Subject: Re: [rsyslog] Problems migrating from syslog-ng
 
I'm not familiar with syslog-ng, but I suspect this is just a
difference in the formatting/interpretation of the raw packet. Try
capturing a message or two from syslog-ng with tcpdump and compare to
section 4 of RFC 3164.

You can see how rsyslog interprets it with the following template:
$template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
PROGRAM: %programname% MSG: %rawmsg%\n"

This should give you some idea of how to create a template that will
log in the format you need.

-HKS



On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> Rsyslog seems to be mangling messages sent from our in-house applications.
> We are trying to get the same format as we did with syslog-ng previously and
> not having a lot of luck.
>
> The logs are in the format something like this:
> TAG APPLICATION: MSG
>
> In syslog-ng, it looks like this locally and when sent to a remote
> syslog-ng server:
> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>
> rsyslog with the TraditionalFileFormat makes it look like this
> locally, stripping the hostname:
> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
> 0.0004593 seconds!
>
> and like this when sent to a remote syslog-ng server. Notice how the
> ADMIN tag was removed:
> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
> offset changed 0.0004593 seconds!
>
> We have realtime logscraping software that looks at all message tags
> and does custom reporting on them.
> It keys off the TAG property that rsyslog is stripping off. What can
> we do to enable this? We've been playing
> with custom $templates in the rsyslog.conf to no real avail.
>
> Thanks!
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From jeffschroed at gmail.com  Thu Aug 21 22:44:16 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Thu, 21 Aug 2008 13:44:16 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
Message-ID: <c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>

On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
work with -c3 and I'd
rather not run in compatibility mode.

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com on behalf of (private) HKS
> Sent: Thu 8/21/2008 7:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems migrating from syslog-ng
>
> I'm not familiar with syslog-ng, but I suspect this is just a
> difference in the formatting/interpretation of the raw packet. Try
> capturing a message or two from syslog-ng with tcpdump and compare to
> section 4 of RFC 3164.
>
> You can see how rsyslog interprets it with the following template:
> $template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
> PROGRAM: %programname% MSG: %rawmsg%\n"
>
> This should give you some idea of how to create a template that will
> log in the format you need.
>
> -HKS
>
>
>
> On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
>> Rsyslog seems to be mangling messages sent from our in-house applications.
>> We are trying to get the same format as we did with syslog-ng previously and
>> not having a lot of luck.
>>
>> The logs are in the format something like this:
>> TAG APPLICATION: MSG
>>
>> In syslog-ng, it looks like this locally and when sent to a remote
>> syslog-ng server:
>> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
>> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>>
>> rsyslog with the TraditionalFileFormat makes it look like this
>> locally, stripping the hostname:
>> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
>> 0.0004593 seconds!
>>
>> and like this when sent to a remote syslog-ng server. Notice how the
>> ADMIN tag was removed:
>> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
>> offset changed 0.0004593 seconds!
>>
>> We have realtime logscraping software that looks at all message tags
>> and does custom reporting on them.
>> It keys off the TAG property that rsyslog is stripping off. What can
>> we do to enable this? We've been playing
>> with custom $templates in the rsyslog.conf to no real avail.
>>
>> Thanks!

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From hks.private at gmail.com  Thu Aug 21 23:00:35 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 21 Aug 2008 17:00:35 -0400
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
	<c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
Message-ID: <ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>

On Thu, Aug 21, 2008 at 4:44 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.
>
> Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
> work with -c3 and I'd
> rather not run in compatibility mode.


I don't think so.

-HKS


From jeffschroed at gmail.com  Fri Aug 22 15:21:01 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Fri, 22 Aug 2008 06:21:01 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
	<c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
	<ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>
Message-ID: <c24352ca0808220621s5419322fi3f0820e1dea79da9@mail.gmail.com>

On Thu, Aug 21, 2008 at 2:00 PM, (private) HKS <hks.private at gmail.com> wrote:
> On Thu, Aug 21, 2008 at 4:44 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
>> On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>>> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

You were right Rainer. It looks like the java code which injects the
message is sending malformed
syslog requests. syslog-ng still sends it through and does the correct
things. Is there a way to make
rsyslog a bit less strict about it? Running rsyslog with -c0 defeats
the purpose of using rsyslog.

Until our application has been fixed and rolled out accross our
clusters worldwide, we rolled back to syslog-ng.


>>
>> Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
>> work with -c3 and I'd
>> rather not run in compatibility mode.
>
>
> I don't think so.
>
> -HKS
>



-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From aoz.syn at gmail.com  Sat Aug 23 09:45:41 2008
From: aoz.syn at gmail.com (RB)
Date: Sat, 23 Aug 2008 01:45:41 -0600
Subject: [rsyslog] Running hashes
Message-ID: <4255c2570808230045kb625e11u795435a8a6c8d457@mail.gmail.com>

Acting on an old comment from a friend, I decided to look into what
it'd take to get rsyslog to perform running hashes of logs.
Conceptually, it's pretty simple - every Nth message inject one
message containing the hash of the previous N messages (including the
previous hash message).  It also gave me an excuse to start digging
into the rsyslog code.

At first I thought I could do it with a property replacer, but that
seems a wash since those are wholly message-based and don't [seem to]
give the opportunity to store information (even a running hash) of
prior messages.  A plugin was my next hope, but there doesn't seem to
be a good mechanism to pipeline those together - AFAICT they're
expected to be single ingress/egress points, with no interstitial
stages.  I see the code for loading other objects as Rainer mentioned
in April, but that seems more for central functionality than for
chaining modules together.

This all brings me back to one of my original questions for rsyslog -
is module chaining something that is even on your radar?  I'm thinking
normalization, hashing, encryption, etc.  Almost feels like there
should be another layer here, maybe a "mangle" plugin interface that
could stack in after im* and before om*?


RB


From aoz.syn at gmail.com  Sun Aug 24 08:01:38 2008
From: aoz.syn at gmail.com (RB)
Date: Sun, 24 Aug 2008 00:01:38 -0600
Subject: [rsyslog] broken size_nbr handling
Message-ID: <4255c2570808232301n5bab0c30w549df07c8e5b6c1f@mail.gmail.com>

In implementing my own 'human-readable' size parser (would be nice to
have one as a part of one of the usable objects), I noticed there
seemed to be order-of-magnitude issues on the base-10 size definitions
in cfsysline.c.  Although the base-2 operations properly increase 2^10
(1024) for each SI prefix, the base-10 ones only increase by one order
of magnitude (10) instead of the proper 10^3 per prefix, ending up
with EB only being 10^8 instead of 10^18.

Following is a simple patch that should put things right:
======================================
diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c
index 18643ba..1989247 100644
--- a/runtime/cfsysline.c
+++ b/runtime/cfsysline.c
@@ -215,11 +215,11 @@ static rsRetVal doGetSize(uchar **pp, rsRetVal
(*pSetHdlr)(void*, uid_t), void *
                case 'e': i *= (int64) 1024 * 1024 * 1024 * 1024 *
1024 * 1024; ++(*pp); break; /* exa */
                /* and now the "new" 1000-based definitions */
                case 'K': i *= 1000; ++(*pp); break;
-               case 'M': i *= 10000; ++(*pp); break;
-               case 'G': i *= 100000; ++(*pp); break;
-               case 'T': i *= 1000000; ++(*pp); break; /* tera */
-               case 'P': i *= 10000000; ++(*pp); break; /* peta */
-               case 'E': i *= 100000000; ++(*pp); break; /* exa */
+               case 'M': i *= 1000000; ++(*pp); break;
+               case 'G': i *= 1000000000; ++(*pp); break;
+               case 'T': i *= 1000000000000; ++(*pp); break; /* tera */
+               case 'P': i *= 1000000000000000; ++(*pp); break; /* peta */
+               case 'E': i *= 1000000000000000000; ++(*pp); break; /* exa */
        }

        /* done */
======================================


From friedl at hq.adiscon.com  Mon Aug 25 17:47:31 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 25 Aug 2008 17:47:31 +0200
Subject: [rsyslog] rsyslog 3.19.11 released (beta)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F039@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 3.19.11, a member of the beta branch, today.
This is a refresh of the current beta, bringing it up to date. It
includes a number of bugfixes from the v3-stable branch. It is a
recommended update for all beta branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-129.phtml

Changelog:

http://www.rsyslog.com/Article273.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .




From mattjhell at gmail.com  Tue Aug 26 00:07:37 2008
From: mattjhell at gmail.com (Matt Hellman)
Date: Mon, 25 Aug 2008 17:07:37 -0500
Subject: [rsyslog] gnutls support in 3.18.3?
Message-ID: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>

I don't see the --enable-gnutls option in 3.18.3.  Is this only in dev
branch currently?


From mbiebl at gmail.com  Tue Aug 26 00:39:07 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 26 Aug 2008 00:39:07 +0200
Subject: [rsyslog] gnutls support in 3.18.3?
In-Reply-To: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>
References: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>
Message-ID: <aa76cdac0808251539o5f5508e9qf45c13af2edfcf83@mail.gmail.com>

2008/8/26 Matt Hellman <mattjhell at gmail.com>:
> I don't see the --enable-gnutls option in 3.18.3.  Is this only in dev
> branch currently?

That is correct.
The TLS support is not available in the v3-stable branch.
You should use the beta branch (latest released version is 3.19.11) or
the master aka devel branch (latest released version is 3.21.2)

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From danson at rackspace.com  Wed Aug 27 23:18:40 2008
From: danson at rackspace.com (Daniel Anson)
Date: Wed, 27 Aug 2008 16:18:40 -0500
Subject: [rsyslog] test
Message-ID: <19205_1219871984_m7RLJaDk013323_96AF20FDF4301D419B33CCE8E3A0132B08A296C2@SAT4MX07.RACKSPACE.CORP>

Test.  New to group.

Daniel M. Anson
Linux Systems Engineer
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From rory at ooma.com  Thu Aug 28 00:20:16 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 27 Aug 2008 15:20:16 -0700
Subject: [rsyslog] phpLogCon bug?
Message-ID: <48B5D320.80405@ooma.com>

It seems that if I execute a query, and then close the browser window 
(safari and firefox), the query remains running on the sql server and 
only goes away after it times out, degrading performance.

I'm using 2.5.3 of phpLogCon


From alorbach at ro1.adiscon.com  Thu Aug 28 09:50:39 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 28 Aug 2008 09:50:39 +0200
Subject: [rsyslog] phpLogCon bug?
In-Reply-To: <48B5D320.80405@ooma.com>
References: <48B5D320.80405@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D9D@rmail.adiscon.com>

Hi,

I think this behavior comes from the design of php. Closing the Browser
Window tells the webserver to quit the current connection, but if the
php processor hangs within a working sql query, he has to wait until
this query is done. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 28, 2008 12:20 AM
> To: rsyslog-users
> Subject: [rsyslog] phpLogCon bug?
> 
> It seems that if I execute a query, and then close the browser window
> (safari and firefox), the query remains running on the sql server and
> only goes away after it times out, degrading performance.
> 
> I'm using 2.5.3 of phpLogCon
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug 28 10:20:46 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Aug 2008 10:20:46 +0200
Subject: [rsyslog] phpLogCon bug?
Message-ID: <000c01c908e7$0d9b6f72$060013ac@intern.adiscon.com>

I agree with andre, thats http behaviour. The srver doesnt even know the client is dead until it tries to send data (which it can only after completing the query).
Hth
rainer

----- Urspr?ngliche Nachricht -----
Von: "Andre Lorbach" <alorbach at ro1.adiscon.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 28.08.08 09:50
Betreff: Re: [rsyslog] phpLogCon bug?

Hi,

I think this behavior comes from the design of php. Closing the Browser
Window tells the webserver to quit the current connection, but if the
php processor hangs within a working sql query, he has to wait until
this query is done. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 28, 2008 12:20 AM
> To: rsyslog-users
> Subject: [rsyslog] phpLogCon bug?
> 
> It seems that if I execute a query, and then close the browser window
> (safari and firefox), the query remains running on the sql server and
> only goes away after it times out, degrading performance.
> 
> I'm using 2.5.3 of phpLogCon
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From danson at rackspace.com  Thu Aug 28 16:54:59 2008
From: danson at rackspace.com (Daniel Anson)
Date: Thu, 28 Aug 2008 09:54:59 -0500
Subject: [rsyslog] Memory vanishes
Message-ID: <28252_1219935370_m7SEu5H1031505_96AF20FDF4301D419B33CCE8E3A0132B08A29886@SAT4MX07.RACKSPACE.CORP>

For a bit of background, I have a RHEL5 server acting as an aggregate
point to write rsyslog/syslog info to:
rsyslog.i386                             2.0.0-11.el5
installed       
rsyslog-mysql.i386                       2.0.0-11.el5
installed

I am manipulating this data received with a threaded C program.  So, in
the spirit of testing, I wrote this small code to produce logs so I can
test how many messages that can be processed.  Here is the code for that
program:

#include <syslog.h>

int
main(int argc, char **argv) {
  int num_syslogs = atoi(argv[1]), i;
  openlog("syslog_writer", LOG_CONS | LOG_PID, LOG_USER);
  for(i=0; i < num_syslogs; i++) {
    syslog(LOG_INFO, "syslog_writer: log number %d", i);
  }
  closelog();
  return 1;
}

I run it thusly:
./syslogwriter 300000

After the running of this program, I find that ~30MB of memory just
disappear and never reappear.  I understand that MySQL and other
programs can hold some memory but do end up releasing it.  This is not
the case, this memory is not freed.  I know that I am not using the most
current version but I am using a STABLE release.  Has anyone seen this
behavior before or have an idea why this is happening.  I am at a loss.
Have I merely reached a limitation point in the number of consecutive
logs that can be handled?

Thx,

Daniel M. Anson
Linux Systems Engineer III
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From rgerhards at hq.adiscon.com  Thu Aug 28 21:47:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Aug 2008 21:47:56 +0200
Subject: [rsyslog] Memory vanishes
Message-ID: <000e01c90947$0ce8c017$060013ac@intern.adiscon.com>

2.0.0 is quite old. I think there are some mem leak fixes in the more recent v2-stable builds (see changelog). I suggest updating (from source) to the current v2-stable (2.0.6 or so, not sure out of my head).

Rainer

----- Urspr?ngliche Nachricht -----
Von: "Daniel Anson" <danson at rackspace.com>
An: "rsyslog at lists.adiscon.com" <rsyslog at lists.adiscon.com>
Gesendet: 28.08.08 16:56
Betreff: [rsyslog] Memory vanishes

For a bit of background, I have a RHEL5 server acting as an aggregate
point to write rsyslog/syslog info to:
rsyslog.i386                             2.0.0-11.el5
installed       
rsyslog-mysql.i386                       2.0.0-11.el5
installed

I am manipulating this data received with a threaded C program.  So, in
the spirit of testing, I wrote this small code to produce logs so I can
test how many messages that can be processed.  Here is the code for that
program:

#include <syslog.h>

int
main(int argc, char **argv) {
  int num_syslogs = atoi(argv[1]), i;
  openlog("syslog_writer", LOG_CONS | LOG_PID, LOG_USER);
  for(i=0; i < num_syslogs; i++) {
    syslog(LOG_INFO, "syslog_writer: log number %d", i);
  }
  closelog();
  return 1;
}

I run it thusly:
./syslogwriter 300000

After the running of this program, I find that ~30MB of memory just
disappear and never reappear.  I understand that MySQL and other
programs can hold some memory but do end up releasing it.  This is not
the case, this memory is not freed.  I know that I am not using the most
current version but I am using a STABLE release.  Has anyone seen this
behavior before or have an idea why this is happening.  I am at a loss.
Have I merely reached a limitation point in the number of consecutive
logs that can be handled?

Thx,

Daniel M. Anson
Linux Systems Engineer III
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From julianokyap at gmail.com  Fri Aug  1 01:50:22 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 13:50:22 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
Message-ID: <d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>

Yes, this would be correct.

In my example, a particular message string would be the same.

Example:
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad

So then is could be $msg == 'This is really bad' or perhaps $msg
contains 'really bad'.  But for me, matching the exact $msg would be
fine. :P


On Thu, Jul 31, 2008 at 10:38 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> To clarify: be "a" the event in question and "b" any other event. Two samples of an event sequence:
>
> 1. a - a - a - b
> 2. a - a - b - a
>
> Result: in case 1 an alert is triggered, in case 2 not.
>
> Is this understanding correct?
>
> rainer
>
> ----- Urspr?ngliche Nachricht -----
> Von: "Julian Yap" <julianokyap at gmail.com>
> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> Cc: "rgerhards at hq.adiscon.com" <rgerhards at hq.adiscon.com>; "hks.private at gmail.com" <hks.private at gmail.com>
> Gesendet: 31.07.08 21:59
> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
>
> That's pretty much it for now.  I've written Alerts for single line
> events.  But for one particular event, it's only really a factor if it
> happens tree times in a row.
>
>
> On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> What exactly do you need to do except the "three in a row" alert?
>>
>> ----- Urspr?ngliche Nachricht -----
>> Von: "Julian Yap" <julianokyap at gmail.com>
>> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
>> Gesendet: 31.07.08 20:27
>> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> Hmm, Nagios is a pain to set up.  Looking for something more light
>> weight...  Was hoping that I could have consolidated lots of Alerts
>> under Rsyslog.
>>
>> Any other suggestions besides Swatch?
>>
>>
>>
>> On 7/31/08, (private) HKS <hks.private at gmail.com> wrote:
>>> Not in rsyslogd itself, but you could do this with Swatch, Nagios, or
>>> some other monitoring-type software.
>>>
>>> -HKS
>>>
>>> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap <julianokyap at gmail.com> wrote:
>>>> Is there a way to set an Alert when multiple repeated lines are found in a
>>>> log?
>>>>
>>>> I want to spawn an email Alert if a message is received 3 times.
>>>>
>>>> Example log lines:
>>>> Jul 30 04:19:29 localhost program: Error detected
>>>> Jul 30 05:19:29 localhost program: Error detected
>>>> Jul 30 06:19:29 localhost program: Error detected
>>>>
>>>> Thanks,
>>>> Julian
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>
>


From rgerhards at hq.adiscon.com  Fri Aug  1 07:45:15 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 07:45:15 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>

Just to make sure:

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:04 server program: This is really bad

[Note the last timestamp!] would still make up for "three in a row"?

Now, syslog contains not only timestamps, but also hostnames. So how about

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server2 program: This is really bad
Jul 31 13:45:03 server program: This is really bad

And what about this:

Jul 31 13:45:03 server program2: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program: This is really bad

And would that trigger any alert at all:

Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server2 program: This is really bad
Jul 31 13:45:03 server program: This is really bad
Jul 31 13:45:03 server program2: This is really bad
Jul 31 13:45:03 server program: This is really bad

Finally, does "three in a row" time out? So what would happen in the case below. Watch the timestamps and let's assume there are no other records inside the log:

Jul 29 13:45:03 server program: This is really bad
Jul 30 13:45:03 server program: This is really bad
Jul 31 13:45:04 server program: This is really bad

And a related question. You write:

> So then is could be $msg == 'This is really bad' or perhaps $msg
> contains 'really bad'.  But for me, matching the exact $msg would be
> fine. :P

Does this imply you would like to do the "if $msg == 'this is really bad'" check? So far, I assume you do NOT do this but expect an alert to be raised whenever ANY messages fulfills the identity criterion n times in a row. Please explain.

Thanks,
Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 1:50 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Yes, this would be correct.
> 
> In my example, a particular message string would be the same.
> 
> Example:
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> 
> So then is could be $msg == 'This is really bad' or perhaps $msg
> contains 'really bad'.  But for me, matching the exact $msg would be
> fine. :P
> 
> 
> On Thu, Jul 31, 2008 at 10:38 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > To clarify: be "a" the event in question and "b" any other event. Two
> samples of an event sequence:
> >
> > 1. a - a - a - b
> > 2. a - a - b - a
> >
> > Result: in case 1 an alert is triggered, in case 2 not.
> >
> > Is this understanding correct?
> >
> > rainer
> >
> > ----- Urspr?ngliche Nachricht -----
> > Von: "Julian Yap" <julianokyap at gmail.com>
> > An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> > Cc: "rgerhards at hq.adiscon.com" <rgerhards at hq.adiscon.com>;
> "hks.private at gmail.com" <hks.private at gmail.com>
> > Gesendet: 31.07.08 21:59
> > Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
> >
> > That's pretty much it for now.  I've written Alerts for single line
> > events.  But for one particular event, it's only really a factor if
> it
> > happens tree times in a row.
> >
> >
> > On Thu, Jul 31, 2008 at 8:37 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> What exactly do you need to do except the "three in a row" alert?
> >>
> >> ----- Urspr?ngliche Nachricht -----
> >> Von: "Julian Yap" <julianokyap at gmail.com>
> >> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> >> Gesendet: 31.07.08 20:27
> >> Betreff: Re: [rsyslog] Alert when multiple repeated lines are found
> >>
> >> Hmm, Nagios is a pain to set up.  Looking for something more light
> >> weight...  Was hoping that I could have consolidated lots of Alerts
> >> under Rsyslog.
> >>
> >> Any other suggestions besides Swatch?
> >>
> >>
> >>
> >> On 7/31/08, (private) HKS <hks.private at gmail.com> wrote:
> >>> Not in rsyslogd itself, but you could do this with Swatch, Nagios,
> or
> >>> some other monitoring-type software.
> >>>
> >>> -HKS
> >>>
> >>> On Wed, Jul 30, 2008 at 6:18 PM, Julian Yap <julianokyap at gmail.com>
> wrote:
> >>>> Is there a way to set an Alert when multiple repeated lines are
> found in a
> >>>> log?
> >>>>
> >>>> I want to spawn an email Alert if a message is received 3 times.
> >>>>
> >>>> Example log lines:
> >>>> Jul 30 04:19:29 localhost program: Error detected
> >>>> Jul 30 05:19:29 localhost program: Error detected
> >>>> Jul 30 06:19:29 localhost program: Error detected
> >>>>
> >>>> Thanks,
> >>>> Julian
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>
> >


From rgerhards at hq.adiscon.com  Fri Aug  1 07:58:10 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 07:58:10 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>

Sorry for being brief, I was on the road and needed to write from a cell phone :(

It also looks like I was wrong ;) I have reviewed the case. If I understand correctly, you get you data from the different unix sockets and then forward via a single rule. So you can not use different templates (which most often are the real solution to any such problem, but are not obvious to most people not directly involved in rsyslog ;)).

Is my understanding correct now?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, July 31, 2008 5:04 PM
> To: David Darville; rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> Use a template with fixed name.
> 
> --- Urspr?ngliche Nachricht ---
> Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> Betreff: Re: [rsyslog] Changing hostname field
> Datum: 31. Juli 2008
> Uhrzeit: 16:46:59
> 
> The jails all have their own unique hostname (and IP), but all share an
> rsyslogd instance running on the main host, and the %hostname% and
> %fromhost% in all the log messages from the jails are set to the
> hostname of
> the main host. And that is what I want to change.
> 
> On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > Do the jails all share the same hostname and IP? If not, you should
> be
> > able to use the %hostname% or %fromhost% properties.
> >
> > If so, are they each running their own instance of (r)syslogd?
> >
> > -HKS
> >
> > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > <ml at darville.vm.bytemark.co.uk> wrote:
> > > Hello everyone
> > >
> > > I am trying to configure rsyslog to service a number of chroot
> jails in
> > > addition to the host itself.
> > >
> > > But I need to change the hostname field of the syslog messages from
> the
> > > different jails, so that I place them in the right log file on the
> central
> > > logging host.
> > >
> > > My current rsyslog.conf is as follows:
> > >
> > > $ModLoad imuxsock
> > > $ModLoad imklog
> > > $ModLoad immark
> > > $ModLoad omrelp
> > >
> > > $AddUnixListenSocket /jail/1/dev/log
> > > $AddUnixListenSocket /jail/2/dev/log
> > >
> > > *.* :omrelp:10.0.0.4:2514
> > >
> > >
> > > Can anyone please advice me on how to do that?
> > >
> > >
> > > ---
> > >
> > > David Darville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From julianokyap at gmail.com  Fri Aug  1 09:47:02 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 21:47:02 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>

Hi Rainer,

Thanks for taking the time to work on my issue.  I'll approach this
from my situation.

On Thu, Jul 31, 2008 at 7:45 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just to make sure:
>
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:04 server program: This is really bad
>
> [Note the last timestamp!] would still make up for "three in a row"?

Yes.  Something that would be nice would be 'three in a row in the
last x minutes'...  But let's keep things simple for now :)

> Now, syslog contains not only timestamps, but also hostnames. So how about
>
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server2 program: This is really bad
> Jul 31 13:45:03 server program: This is really bad

In my particular case, it would only come from the one server.  But if
I had 2 servers logging to the same log file, it should have the
option to filter based on server and/or message.

> And what about this:
>
> Jul 31 13:45:03 server program2: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
>
> And would that trigger any alert at all:

In my particular case, it would only come from the one program.  But
if I had 2 programs logging to the same log file, it should have the
option to filter based on program and/or message.

> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server2 program: This is really bad
> Jul 31 13:45:03 server program: This is really bad
> Jul 31 13:45:03 server program2: This is really bad
> Jul 31 13:45:03 server program: This is really bad
>
> Finally, does "three in a row" time out? So what would happen in the case below. Watch the timestamps and let's assume there are no other records inside the log:

In my particular case, 'three in a row' wouldn't time out.  I would
use ActionMail so I would use a corresponding
$ActionExecOnlyOnceEveryInterval value.

For my particular case, it happens when a process totally locks up.  A
very rare instance which requires manually restarting a process...  So
the 'This is really bad' messages would eventually stop.

> Jul 29 13:45:03 server program: This is really bad
> Jul 30 13:45:03 server program: This is really bad
> Jul 31 13:45:04 server program: This is really bad
>
> And a related question. You write:
>
>> So then is could be $msg == 'This is really bad' or perhaps $msg
>> contains 'really bad'.  But for me, matching the exact $msg would be
>> fine. :P
>
> Does this imply you would like to do the "if $msg == 'this is really bad'" check? So far, I assume you do NOT do this but expect an alert to be raised whenever ANY messages fulfills the identity criterion n times in a row. Please explain.

Yes, you assume correctly.

What I'm looking for:
if $msg == 'This is really bad' happens 3 times in a row then :ommail:;mailBody

This would be nice but is not required since the 'This is really bad'
message in my case is very unique:
if ($msg == 'This is really bad' and $server == 'server' and $program
== 'program') happens 3 times in a row then :ommail:;mailBody

- Julian

> Thanks,
> Rainer


From rgerhards at hq.adiscon.com  Fri Aug  1 10:18:59 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 10:18:59 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>

Just one more re-confirmation:

> What I'm looking for:
> if $msg == 'This is really bad' happens 3 times in a row then
> :ommail:;mailBody
> 
> This would be nice but is not required since the 'This is really bad'
> message in my case is very unique:
> if ($msg == 'This is really bad' and $server == 'server' and $program
> == 'program') happens 3 times in a row then :ommail:;mailBody

So you would actually use such a rule. If "this other thing is really
bad" happened three times, the rule shall not trigger. Is this right?

Rainer


From julianokyap at gmail.com  Fri Aug  1 11:02:32 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 31 Jul 2008 23:02:32 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>

On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just one more re-confirmation:
>
>> What I'm looking for:
>> if $msg == 'This is really bad' happens 3 times in a row then
>> :ommail:;mailBody
>>
>> This would be nice but is not required since the 'This is really bad'
>> message in my case is very unique:
>> if ($msg == 'This is really bad' and $server == 'server' and $program
>> == 'program') happens 3 times in a row then :ommail:;mailBody
>
> So you would actually use such a rule. If "this other thing is really
> bad" happened three times, the rule shall not trigger. Is this right?

Yes, I would use such a rule.  It would make what is already an
awesome application even more awesome. :P  I am also willing to test
it out and run the latest development version...  Which I'm doing
anyway.

And yes, what you just wrote is correct.

- Julian


From rgerhards at hq.adiscon.com  Fri Aug  1 11:58:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 11:58:19 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>

OK, that greatly simplifies things. Actually, it now boils down to
"execute an action only on the n-the time the filter evaluates to true".
I think this is quite easy to implement, but I must verify that...

Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 11:03 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Just one more re-confirmation:
> >
> >> What I'm looking for:
> >> if $msg == 'This is really bad' happens 3 times in a row then
> >> :ommail:;mailBody
> >>
> >> This would be nice but is not required since the 'This is really
> bad'
> >> message in my case is very unique:
> >> if ($msg == 'This is really bad' and $server == 'server' and
> $program
> >> == 'program') happens 3 times in a row then :ommail:;mailBody
> >
> > So you would actually use such a rule. If "this other thing is
really
> > bad" happened three times, the rule shall not trigger. Is this
right?
> 
> Yes, I would use such a rule.  It would make what is already an
> awesome application even more awesome. :P  I am also willing to test
> it out and run the latest development version...  Which I'm doing
> anyway.
> 
> And yes, what you just wrote is correct.
> 
> - Julian


From julianokyap at gmail.com  Fri Aug  1 12:13:50 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Fri, 1 Aug 2008 00:13:50 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
Message-ID: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>

Roger that Rainer.

Thanks,
Julian

On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> OK, that greatly simplifies things. Actually, it now boils down to
> "execute an action only on the n-the time the filter evaluates to true".
> I think this is quite easy to implement, but I must verify that...
>
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 01, 2008 11:03 AM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>> > Just one more re-confirmation:
>> >
>> >> What I'm looking for:
>> >> if $msg == 'This is really bad' happens 3 times in a row then
>> >> :ommail:;mailBody
>> >>
>> >> This would be nice but is not required since the 'This is really
>> bad'
>> >> message in my case is very unique:
>> >> if ($msg == 'This is really bad' and $server == 'server' and
>> $program
>> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>> >
>> > So you would actually use such a rule. If "this other thing is
> really
>> > bad" happened three times, the rule shall not trigger. Is this
> right?
>>
>> Yes, I would use such a rule.  It would make what is already an
>> awesome application even more awesome. :P  I am also willing to test
>> it out and run the latest development version...  Which I'm doing
>> anyway.
>>
>> And yes, what you just wrote is correct.
>>
>> - Julian
>


From ml at darville.vm.bytemark.co.uk  Fri Aug  1 12:26:50 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Fri, 1 Aug 2008 11:26:50 +0100
Subject: [rsyslog] Changing hostname field
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
Message-ID: <20080801102649.GA27628@darville.vm.bytemark.co.uk>

On Fri, Aug 01, 2008 at 07:58:10AM +0200, Rainer Gerhards wrote:
> Sorry for being brief, I was on the road and needed to write from
> a cell phone :(

Supporting rsyslog on your cell phone - you are very dedicated to you
product ;-)

> It also looks like I was wrong ;) I have reviewed the case. If I
> understand correctly, you get you data from the different unix sockets
> and then forward via a single rule. So you can not use different
> templates (which most often are the real solution to any such problem,
> but are not obvious to most people not directly involved in rsyslog ;)).
>
> Is my understanding correct now?

Yes, and then I'm splitting log messages on the dedicated log host, based on
the hostname field. 

We are keeping the rsyslogd process outside the jails, to to keep the syslog
traffic on an isolated network segment.

-- David

> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, July 31, 2008 5:04 PM
> > To: David Darville; rsyslog-users
> > Subject: Re: [rsyslog] Changing hostname field
> > 
> > Use a template with fixed name.
> > 
> > --- Urspr?ngliche Nachricht ---
> > Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> > Betreff: Re: [rsyslog] Changing hostname field
> > Datum: 31. Juli 2008
> > Uhrzeit: 16:46:59
> > 
> > The jails all have their own unique hostname (and IP), but all share an
> > rsyslogd instance running on the main host, and the %hostname% and
> > %fromhost% in all the log messages from the jails are set to the
> > hostname of
> > the main host. And that is what I want to change.
> > 
> > On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > > Do the jails all share the same hostname and IP? If not, you should
> > be
> > > able to use the %hostname% or %fromhost% properties.
> > >
> > > If so, are they each running their own instance of (r)syslogd?
> > >
> > > -HKS
> > >
> > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > > <ml at darville.vm.bytemark.co.uk> wrote:
> > > > Hello everyone
> > > >
> > > > I am trying to configure rsyslog to service a number of chroot
> > jails in
> > > > addition to the host itself.
> > > >
> > > > But I need to change the hostname field of the syslog messages from
> > the
> > > > different jails, so that I place them in the right log file on the
> > central
> > > > logging host.
> > > >
> > > > My current rsyslog.conf is as follows:
> > > >
> > > > $ModLoad imuxsock
> > > > $ModLoad imklog
> > > > $ModLoad immark
> > > > $ModLoad omrelp
> > > >
> > > > $AddUnixListenSocket /jail/1/dev/log
> > > > $AddUnixListenSocket /jail/2/dev/log
> > > >
> > > > *.* :omrelp:10.0.0.4:2514
> > > >
> > > >
> > > > Can anyone please advice me on how to do that?
> > > >
> > > >
> > > > ---
> > > >
> > > > David Darville
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Fri Aug  1 15:06:43 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 15:06:43 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <20080801102649.GA27628@darville.vm.bytemark.co.uk>
References: <001301c8f31e$b2dd0f13$060013ac@intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EEFA@grfint2.intern.adiscon.com>
	<20080801102649.GA27628@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>

Inline...

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Friday, August 01, 2008 12:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> On Fri, Aug 01, 2008 at 07:58:10AM +0200, Rainer Gerhards wrote:
> > Sorry for being brief, I was on the road and needed to write from
> > a cell phone :(
> 
> Supporting rsyslog on your cell phone - you are very dedicated to you
> product ;-)

Thanks - I am trying my best ;) And, if you have not already done so, I'd appreciate if you could rate rsyslog at freshmeat.net, it's just a simple click (even if you need to create an account it doesn't take long). It helps us get momentum and this is as important as code quality ;)

You can do it here (middle of the page):
http://freshmeat.net/projects/rsyslog/

> 
> > It also looks like I was wrong ;) I have reviewed the case. If I
> > understand correctly, you get you data from the different unix
> sockets
> > and then forward via a single rule. So you can not use different
> > templates (which most often are the real solution to any such
> problem,
> > but are not obvious to most people not directly involved in rsyslog
> ;)).
> >
> > Is my understanding correct now?
> 
> Yes, and then I'm splitting log messages on the dedicated log host,
> based on
> the hostname field.
> 
> We are keeping the rsyslogd process outside the jails, to to keep the
> syslog
> traffic on an isolated network segment.

OK, that was fairly easy to do. I have created a special version for you:

http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz

It contains a new config directive that allows you to specify the hostname, see last sample on

http://www.rsyslog.com/doc-imuxsock.html

In my lab it works, but I would appreciate if you could test and verify.

Thanks,
Rainer

> 
> -- David
> 
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > Sent: Thursday, July 31, 2008 5:04 PM
> > > To: David Darville; rsyslog-users
> > > Subject: Re: [rsyslog] Changing hostname field
> > >
> > > Use a template with fixed name.
> > >
> > > --- Urspr?ngliche Nachricht ---
> > > Von: "David Darville" <ml at darville.vm.bytemark.co.uk>
> > > Betreff: Re: [rsyslog] Changing hostname field
> > > Datum: 31. Juli 2008
> > > Uhrzeit: 16:46:59
> > >
> > > The jails all have their own unique hostname (and IP), but all
> share an
> > > rsyslogd instance running on the main host, and the %hostname% and
> > > %fromhost% in all the log messages from the jails are set to the
> > > hostname of
> > > the main host. And that is what I want to change.
> > >
> > > On Thu, Jul 31, 2008 at 10:00:09AM -0400, (private) HKS wrote:
> > > > Do the jails all share the same hostname and IP? If not, you
> should
> > > be
> > > > able to use the %hostname% or %fromhost% properties.
> > > >
> > > > If so, are they each running their own instance of (r)syslogd?
> > > >
> > > > -HKS
> > > >
> > > > On Thu, Jul 31, 2008 at 7:11 AM, David Darville
> > > > <ml at darville.vm.bytemark.co.uk> wrote:
> > > > > Hello everyone
> > > > >
> > > > > I am trying to configure rsyslog to service a number of chroot
> > > jails in
> > > > > addition to the host itself.
> > > > >
> > > > > But I need to change the hostname field of the syslog messages
> from
> > > the
> > > > > different jails, so that I place them in the right log file on
> the
> > > central
> > > > > logging host.
> > > > >
> > > > > My current rsyslog.conf is as follows:
> > > > >
> > > > > $ModLoad imuxsock
> > > > > $ModLoad imklog
> > > > > $ModLoad immark
> > > > > $ModLoad omrelp
> > > > >
> > > > > $AddUnixListenSocket /jail/1/dev/log
> > > > > $AddUnixListenSocket /jail/2/dev/log
> > > > >
> > > > > *.* :omrelp:10.0.0.4:2514
> > > > >
> > > > >
> > > > > Can anyone please advice me on how to do that?
> > > > >
> > > > >
> > > > > ---
> > > > >
> > > > > David Darville
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Fri Aug  1 16:25:02 2008
From: hks.private at gmail.com ((private) HKS)
Date: Fri, 1 Aug 2008 10:25:02 -0400
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
Message-ID: <ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>

This may be a silly question, but is the syslog daemon the proper
place for something like this?

-HKS

On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com> wrote:
> Roger that Rainer.
>
> Thanks,
> Julian
>
> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> OK, that greatly simplifies things. Actually, it now boils down to
>> "execute an action only on the n-the time the filter evaluates to true".
>> I think this is quite easy to implement, but I must verify that...
>>
>> Rainer


From rgerhards at hq.adiscon.com  Fri Aug  1 17:02:32 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 17:02:32 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>

That's not a silly one ;) And it is coming up every now and then. The
last time it came up, I was smart enough to write a blogpost:
http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-plugins.ht
ml

In short, and to this questions: there are different schools of thought.
If you think about a plain ole syslogd shuffling data to disk files, you
do not need that. My vision of the syslogd (actually the "event logging
and alerting") subsystem is much broader. IMHO, it should support
anything that is needed to gather, process and persistently store
events. Also note that I say "events" for a reason - syslog messages are
just a subset of the potential set of events.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Friday, August 01, 2008 4:25 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> This may be a silly question, but is the syslog daemon the proper
> place for something like this?
> 
> -HKS
> 
> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
> wrote:
> > Roger that Rainer.
> >
> > Thanks,
> > Julian
> >
> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> OK, that greatly simplifies things. Actually, it now boils down to
> >> "execute an action only on the n-the time the filter evaluates to
> true".
> >> I think this is quite easy to implement, but I must verify that...
> >>
> >> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Fri Aug  1 17:17:03 2008
From: hks.private at gmail.com ((private) HKS)
Date: Fri, 1 Aug 2008 11:17:03 -0400
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>

Thanks for the link, that post makes a lot of sense. I interpreted
this discussion as moving towards adding alert functionality to the
rsyslogd core - but your stance of keeping the rsyslogd core lean and
efficient while plugins provide a full suite of event handling
processes is reassuring.

I'm looking forward to seeing where you take this plugin architecture.

-HKS

On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> That's not a silly one ;) And it is coming up every now and then. The
> last time it came up, I was smart enough to write a blogpost:
> http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-plugins.ht
> ml
>
> In short, and to this questions: there are different schools of thought.
> If you think about a plain ole syslogd shuffling data to disk files, you
> do not need that. My vision of the syslogd (actually the "event logging
> and alerting") subsystem is much broader. IMHO, it should support
> anything that is needed to gather, process and persistently store
> events. Also note that I say "events" for a reason - syslog messages are
> just a subset of the potential set of events.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of (private) HKS
>> Sent: Friday, August 01, 2008 4:25 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> This may be a silly question, but is the syslog daemon the proper
>> place for something like this?
>>
>> -HKS
>>
>> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
>> wrote:
>> > Roger that Rainer.
>> >
>> > Thanks,
>> > Julian
>> >
>> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> >> OK, that greatly simplifies things. Actually, it now boils down to
>> >> "execute an action only on the n-the time the filter evaluates to
>> true".
>> >> I think this is quite easy to implement, but I must verify that...
>> >>
>> >> Rainer
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From rgerhards at hq.adiscon.com  Fri Aug  1 17:23:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Aug 2008 17:23:35 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com>
	<ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>

Actually, in this case the (limited) functionality will go into the
core. But that's not because we need it for alerting but it is something
that was scheduled for the core engine at all (because there are other
use cases besides alerting where you need it, e.g. start some corrective
action only after the n-th error indication). Plus, it will be very
limited code. There are some other things, namely the enhanced rate
limiter, which will go to the core. I have some ways to do that via
plug-ins too, but given the expected complexity of this functionality vs
the expected complexity of pluginizing them, the choice to go to core is
really obvious. Another point where one could debate is enhanced
queuing. This, in the long term, is a candidate for being moved to a
plugin because many installations do never use disk-based (or assisted)
queues.

Now comes the important difference: if a generic, any message let me
know when it happens n times in the row filter would have been needed,
that would NOT go into the core. Because it does not belong there. It is
quite complex and even performance intense. For that, future version
will have customer (RainerScript) functions which can be provided by
library plugins. I have plans to implement such a beast (much later),
but it will come as a function that you provide the message to and that
is only loaded on an as-needed basis.

I hope this clarifies.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Friday, August 01, 2008 5:17 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Thanks for the link, that post makes a lot of sense. I interpreted
> this discussion as moving towards adding alert functionality to the
> rsyslogd core - but your stance of keeping the rsyslogd core lean and
> efficient while plugins provide a full suite of event handling
> processes is reassuring.
> 
> I'm looking forward to seeing where you take this plugin architecture.
> 
> -HKS
> 
> On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > That's not a silly one ;) And it is coming up every now and then.
The
> > last time it came up, I was smart enough to write a blogpost:
> > http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-
> plugins.ht
> > ml
> >
> > In short, and to this questions: there are different schools of
> thought.
> > If you think about a plain ole syslogd shuffling data to disk files,
> you
> > do not need that. My vision of the syslogd (actually the "event
> logging
> > and alerting") subsystem is much broader. IMHO, it should support
> > anything that is needed to gather, process and persistently store
> > events. Also note that I say "events" for a reason - syslog messages
> are
> > just a subset of the potential set of events.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> >> Sent: Friday, August 01, 2008 4:25 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> >>
> >> This may be a silly question, but is the syslog daemon the proper
> >> place for something like this?
> >>
> >> -HKS
> >>
> >> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap <julianokyap at gmail.com>
> >> wrote:
> >> > Roger that Rainer.
> >> >
> >> > Thanks,
> >> > Julian
> >> >
> >> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >> > <rgerhards at hq.adiscon.com> wrote:
> >> >> OK, that greatly simplifies things. Actually, it now boils down
> to
> >> >> "execute an action only on the n-the time the filter evaluates
to
> >> true".
> >> >> I think this is quite easy to implement, but I must verify
> that...
> >> >>
> >> >> Rainer
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From lovely at crm114.net  Mon Aug  4 03:25:28 2008
From: lovely at crm114.net (Julian Tyler)
Date: Mon, 4 Aug 2008 11:25:28 +1000
Subject: [rsyslog] Denial of Service
Message-ID: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>


I have a client/server setup of rsyslog, however if the server becomes  
unreachable
and the client's memory buffers fill up, everything that needs to log  
just hangs/blocks,
such as ssh.

I have enabled disk queueing however it never seems to queue anything  
to disk.

What am i doing wrong ?

---
----> Rsyslog version
---
[app-00-0001 ~]# rsyslogd -v
rsyslogd 3.16.1, compiled with:
	FEATURE_REGEXP:				Yes
	FEATURE_LARGEFILE:			Yes
	FEATURE_NETZIP (message compression):	Yes
	GSSAPI Kerberos 5 support:		No
	FEATURE_DEBUG (debug build, slow code):	No
	Runtime Instrumentation (slow code):	No

---
-----> Here is my rsyslog.conf from the offending box
---
[app-00-0001 ~]# cat /etc/rsyslog.conf
# rsyslog v3: load input modules
# If you do not load inputs, nothing happens!
# You may need to set the module load path if modules are not found.

# Debug Template
$template MyDebug,"Debug line with all properties:\nFROMHOST:  
'%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%',  
PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%',  
APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%', 
\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%', 
\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n"

$template Sys23Forward, "<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME 
% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%"


$ModLoad immark.so # provides --MARK-- message capability
$ModLoad imuxsock.so # provides support for local system logging (e.g.  
via logger command)
$ModLoad imklog.so # kernel logging (formerly provided by rklogd)
$ModLoad lmregexp.so

*.*							/var/log/all

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                -/var/log/ 
messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/ 
maillog


# Log cron stuff
cron.*                                                  -/var/log/cron

# Everybody gets emergency messages
*.emerg                                                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          -/var/log/ 
spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/ 
boot.log

# Remote Logging (we use TCP for reliable delivery)
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/spool/rsyslog/ # where to place spool files
$ActionQueueType LinkedList   # run asynchronously
$ActionQueueFileName remotelog # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as  
possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@(z9)10.238.0.1;RSYSLOG_SyslogProtocol23Format
*.* @@10.238.0.1;Sys23Forward


# ######### Receiving Messages from Remote Hosts ##########
# TCP Syslog Server:
# provides TCP syslog reception and GSS-API (if compiled to support it)
#$ModLoad imtcp.so  # load module
# Note: as of now, you need to use the -t command line option to
# enable TCP reception (e.g. -t514 to run a server at port 514/tcp)
# This will change in later v3 releases.

# UDP Syslog Server:
$ModLoad imudp.so  # provides UDP syslog reception
#$UDPServerRun 514 # start a UDP syslog server at standard port 514
$UDPServerRun 513 # start a UDP syslog server at standard port 514
#$UDPServerAddress



From rgerhards at hq.adiscon.com  Mon Aug  4 11:07:17 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 4 Aug 2008 11:07:17 +0200
Subject: [rsyslog] Denial of Service
In-Reply-To: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>
References: <F3E5EBA4-4D3B-4913-A6FC-6216C2DCD484@crm114.net>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF16@grfint2.intern.adiscon.com>

I think you are affected by this bug:

http://bugzilla.adiscon.com/show_bug.cgi?id=86

You need to upgrade to the recent v3-stable version.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Julian Tyler
> Sent: Monday, August 04, 2008 3:25 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Denial of Service
> 
> 
> I have a client/server setup of rsyslog, however if the server becomes
> unreachable
> and the client's memory buffers fill up, everything that needs to log
> just hangs/blocks,
> such as ssh.
> 
> I have enabled disk queueing however it never seems to queue anything
> to disk.
> 
> What am i doing wrong ?
> 
> ---
> ----> Rsyslog version
> ---
> [app-00-0001 ~]# rsyslogd -v
> rsyslogd 3.16.1, compiled with:
> 	FEATURE_REGEXP:				Yes
> 	FEATURE_LARGEFILE:			Yes
> 	FEATURE_NETZIP (message compression):	Yes
> 	GSSAPI Kerberos 5 support:		No
> 	FEATURE_DEBUG (debug build, slow code):	No
> 	Runtime Instrumentation (slow code):	No
> 
> ---
> -----> Here is my rsyslog.conf from the offending box
> ---
> [app-00-0001 ~]# cat /etc/rsyslog.conf
> # rsyslog v3: load input modules
> # If you do not load inputs, nothing happens!
> # You may need to set the module load path if modules are not found.
> 
> # Debug Template
> $template MyDebug,"Debug line with all properties:\nFROMHOST:
> '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%',
> PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%',
> APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',
> \nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',
> \nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg:
'%rawmsg%'\n\n"
> 
> $template Sys23Forward, "<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME
> % %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%"
> 
> 
> $ModLoad immark.so # provides --MARK-- message capability
> $ModLoad imuxsock.so # provides support for local system logging (e.g.
> via logger command)
> $ModLoad imklog.so # kernel logging (formerly provided by rklogd)
> $ModLoad lmregexp.so
> 
> *.*							/var/log/all
> 
> # Log all kernel messages to the console.
> # Logging much else clutters up the screen.
> #kern.*                                                 /dev/console
> 
> # Log anything (except mail) of level info or higher.
> # Don't log private authentication messages!
> *.info;mail.none;authpriv.none;cron.none                -/var/log/
> messages
> 
> # The authpriv file has restricted access.
> authpriv.*
/var/log/secure
> 
> # Log all the mail messages in one place.
> mail.*                                                  -/var/log/
> maillog
> 
> 
> # Log cron stuff
> cron.*                                                  -/var/log/cron
> 
> # Everybody gets emergency messages
> *.emerg                                                 *
> 
> # Save news errors of level crit and higher in a special file.
> uucp,news.crit                                          -/var/log/
> spooler
> 
> # Save boot messages also to boot.log
> local7.*                                                /var/log/
> boot.log
> 
> # Remote Logging (we use TCP for reliable delivery)
> # An on-disk queue is created for this action. If the remote host is
> # down, messages are spooled to disk and sent when it is up again.
> $WorkDirectory /var/spool/rsyslog/ # where to place spool files
> $ActionQueueType LinkedList   # run asynchronously
> $ActionQueueFileName remotelog # unique name prefix for spool files
> $ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as
> possible)
> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
> $ActionResumeRetryCount -1    # infinite retries if host is down
> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
> #*.* @@(z9)10.238.0.1;RSYSLOG_SyslogProtocol23Format
> *.* @@10.238.0.1;Sys23Forward
> 
> 
> # ######### Receiving Messages from Remote Hosts ##########
> # TCP Syslog Server:
> # provides TCP syslog reception and GSS-API (if compiled to support
it)
> #$ModLoad imtcp.so  # load module
> # Note: as of now, you need to use the -t command line option to
> # enable TCP reception (e.g. -t514 to run a server at port 514/tcp)
> # This will change in later v3 releases.
> 
> # UDP Syslog Server:
> $ModLoad imudp.so  # provides UDP syslog reception
> #$UDPServerRun 514 # start a UDP syslog server at standard port 514
> $UDPServerRun 513 # start a UDP syslog server at standard port 514
> #$UDPServerAddress
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From friedl at hq.adiscon.com  Mon Aug  4 17:40:28 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 4 Aug 2008 17:40:28 +0200
Subject: [rsyslog] rsyslog 3.21.2 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF2D@grfint2.intern.adiscon.com>

Hi all,

rsyslog 3.21.2, a member of the devel branch, has been released today.
It offers two improvements: Imuxsock now permits to specify an
alternative hostname to be used with messages originating from that
socket. This is useful for logging jail processes. Ommail now offers the
ability to specify multiple email recipients. Also, the troubleshooting
documentation has been enhanced and now contains information on debug
mode. There are no bug fixes in this release. This is a recommended
update for all devel branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-124.phtml

Changelog:

http://www.rsyslog.com/Article264.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .




From alorbach at ro1.adiscon.com  Tue Aug  5 16:25:04 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 5 Aug 2008 16:25:04 +0200
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>

Hi again, 

I want to inform you all that the search within phpLogCon has been
extended to support full and partial searches in Version 2.5.3,
available for download here: http://www.phplogcon.org/downloads

I first tests have shown, that filtering by hostname and other string
based fields performs much better on database sources now. 
You can use the submenu buttons on each field within the messages view
to perform such a search. If you want to do manual searches, it is very
simple. Here is a sample: 

Search for a full hostname: source:=SERVERNAME (Will only find if the
match is 100% = SERVERNAME)
Search for a partial hostname source:SERVER (Willmatch if the hostname
is SERVER or SERVERNAME).  

I hope this helps, 
Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Thursday, July 31, 2008 10:15 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> Hi,
> 
> the like query can indeed have quiet an impact on performance when
doing
> queries on large databases.
> But I think we can expand the syntax, so you can either search by part
> of a string (LIKE '%search%') or the whole string (= 'search'). This
> should be rather easy to implement. I will put this on my todolist, if
> it is as easy as I think, the next minor update of the devel branch
will
> contain this new feature.
> 
> Best regards,
> Andre Lorbach
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > Sent: Thursday, July 31, 2008 4:10 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] tips for managing data
> >
> > OK, so it seems that doing a query from the query line does a LIKE,
> > which can take significantly longer (sample query 8 seconds vs. 50
> msecs...)
> >
> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
> things
> > up quite a but, but I lose some flexibility. Is there some kind of
> > search syntax where I can differentiate between LIKE and =?
> >
> > If not, I'm thinking something like:
> >
> > source:foo.bar.com   # would be using =
> >
> > ~source:foo   # would be using LIKE
> >
> >
> >
> > Rory Toma wrote:
> > > So, my current mysql rsyslog drops about 20 million rows of data
per
> day.
> > >
> > > Over time, this gets slow as tables grow.
> > >
> > > I'm not a dba, so I was wondering if anyone had some suggestions
for
> > > keeping performance still on the order of seconds, and not minutes
> or hours.
> > >
> > > thx
> > >
> > > I did add a key for EventSource, as that is commonly searched.
> However,
> > > using PhpLogCon, it seems that if I search using the web interface
> (i.e.
> > > I click on a host entry and hit the available searches) it is
> relatively
> > > quick. However, changing the text field that is generated and
> hitting
> > > the "search" button is slow. Do these two methods use the same
> query, or
> > > is something else going on?
> > >
> > > thx
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From hks.private at gmail.com  Tue Aug  5 17:09:22 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 5 Aug 2008 11:09:22 -0400
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com> <48911EF0.1010004@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
Message-ID: <ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>

Will searching for NAME match SERVERNAME?

-HKS

On Tue, Aug 5, 2008 at 10:25 AM, Andre Lorbach <alorbach at ro1.adiscon.com> wrote:
> Hi again,
>
> I want to inform you all that the search within phpLogCon has been
> extended to support full and partial searches in Version 2.5.3,
> available for download here: http://www.phplogcon.org/downloads
>
> I first tests have shown, that filtering by hostname and other string
> based fields performs much better on database sources now.
> You can use the submenu buttons on each field within the messages view
> to perform such a search. If you want to do manual searches, it is very
> simple. Here is a sample:
>
> Search for a full hostname: source:=SERVERNAME (Will only find if the
> match is 100% = SERVERNAME)
> Search for a partial hostname source:SERVER (Willmatch if the hostname
> is SERVER or SERVERNAME).
>
> I hope this helps,
> Best regards,
> Andre Lorbach
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>> Sent: Thursday, July 31, 2008 10:15 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] tips for managing data
>>
>> Hi,
>>
>> the like query can indeed have quiet an impact on performance when
> doing
>> queries on large databases.
>> But I think we can expand the syntax, so you can either search by part
>> of a string (LIKE '%search%') or the whole string (= 'search'). This
>> should be rather easy to implement. I will put this on my todolist, if
>> it is as easy as I think, the next minor update of the devel branch
> will
>> contain this new feature.
>>
>> Best regards,
>> Andre Lorbach
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> > Sent: Thursday, July 31, 2008 4:10 AM
>> > To: rsyslog-users
>> > Subject: Re: [rsyslog] tips for managing data
>> >
>> > OK, so it seems that doing a query from the query line does a LIKE,
>> > which can take significantly longer (sample query 8 seconds vs. 50
>> msecs...)
>> >
>> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
>> things
>> > up quite a but, but I lose some flexibility. Is there some kind of
>> > search syntax where I can differentiate between LIKE and =?
>> >
>> > If not, I'm thinking something like:
>> >
>> > source:foo.bar.com   # would be using =
>> >
>> > ~source:foo   # would be using LIKE
>> >
>> >
>> >
>> > Rory Toma wrote:
>> > > So, my current mysql rsyslog drops about 20 million rows of data
> per
>> day.
>> > >
>> > > Over time, this gets slow as tables grow.
>> > >
>> > > I'm not a dba, so I was wondering if anyone had some suggestions
> for
>> > > keeping performance still on the order of seconds, and not minutes
>> or hours.
>> > >
>> > > thx
>> > >
>> > > I did add a key for EventSource, as that is commonly searched.
>> However,
>> > > using PhpLogCon, it seems that if I search using the web interface
>> (i.e.
>> > > I click on a host entry and hit the available searches) it is
>> relatively
>> > > quick. However, changing the text field that is generated and
>> hitting
>> > > the "search" button is slow. Do these two methods use the same
>> query, or
>> > > is something else going on?
>> > >
>> > > thx
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > >
>> >
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From alorbach at ro1.adiscon.com  Tue Aug  5 17:31:24 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 5 Aug 2008 17:31:24 +0200
Subject: [rsyslog] tips for managing data
In-Reply-To: <ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>
References: <4891042D.9040805@ooma.com>
	<48911EF0.1010004@ooma.com><F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
	<ac9ded160808050809j4f5ce8d1qddf94ed1a5cca3a6@mail.gmail.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D71@rmail.adiscon.com>

Yes if you use a partial search, it will match as well. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> Sent: Tuesday, August 05, 2008 5:09 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> Will searching for NAME match SERVERNAME?
> 
> -HKS
> 
> On Tue, Aug 5, 2008 at 10:25 AM, Andre Lorbach
<alorbach at ro1.adiscon.com>
> wrote:
> > Hi again,
> >
> > I want to inform you all that the search within phpLogCon has been
> > extended to support full and partial searches in Version 2.5.3,
> > available for download here: http://www.phplogcon.org/downloads
> >
> > I first tests have shown, that filtering by hostname and other
string
> > based fields performs much better on database sources now.
> > You can use the submenu buttons on each field within the messages
view
> > to perform such a search. If you want to do manual searches, it is
very
> > simple. Here is a sample:
> >
> > Search for a full hostname: source:=SERVERNAME (Will only find if
the
> > match is 100% = SERVERNAME)
> > Search for a partial hostname source:SERVER (Willmatch if the
hostname
> > is SERVER or SERVERNAME).
> >
> > I hope this helps,
> > Best regards,
> > Andre Lorbach
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> >> Sent: Thursday, July 31, 2008 10:15 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] tips for managing data
> >>
> >> Hi,
> >>
> >> the like query can indeed have quiet an impact on performance when
> > doing
> >> queries on large databases.
> >> But I think we can expand the syntax, so you can either search by
part
> >> of a string (LIKE '%search%') or the whole string (= 'search').
This
> >> should be rather easy to implement. I will put this on my todolist,
if
> >> it is as easy as I think, the next minor update of the devel branch
> > will
> >> contain this new feature.
> >>
> >> Best regards,
> >> Andre Lorbach
> >>
> >> > -----Original Message-----
> >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> >> > Sent: Thursday, July 31, 2008 4:10 AM
> >> > To: rsyslog-users
> >> > Subject: Re: [rsyslog] tips for managing data
> >> >
> >> > OK, so it seems that doing a query from the query line does a
LIKE,
> >> > which can take significantly longer (sample query 8 seconds vs.
50
> >> msecs...)
> >> >
> >> > So, replacing the LIKE % in logstreamdb.class.db with an = speeds
> >> things
> >> > up quite a but, but I lose some flexibility. Is there some kind
of
> >> > search syntax where I can differentiate between LIKE and =?
> >> >
> >> > If not, I'm thinking something like:
> >> >
> >> > source:foo.bar.com   # would be using =
> >> >
> >> > ~source:foo   # would be using LIKE
> >> >
> >> >
> >> >
> >> > Rory Toma wrote:
> >> > > So, my current mysql rsyslog drops about 20 million rows of
data
> > per
> >> day.
> >> > >
> >> > > Over time, this gets slow as tables grow.
> >> > >
> >> > > I'm not a dba, so I was wondering if anyone had some
suggestions
> > for
> >> > > keeping performance still on the order of seconds, and not
minutes
> >> or hours.
> >> > >
> >> > > thx
> >> > >
> >> > > I did add a key for EventSource, as that is commonly searched.
> >> However,
> >> > > using PhpLogCon, it seems that if I search using the web
interface
> >> (i.e.
> >> > > I click on a host entry and hit the available searches) it is
> >> relatively
> >> > > quick. However, changing the text field that is generated and
> >> hitting
> >> > > the "search" button is slow. Do these two methods use the same
> >> query, or
> >> > > is something else going on?
> >> > >
> >> > > thx
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > >
> >> >
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Tue Aug  5 20:45:31 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 11:45:31 -0700
Subject: [rsyslog] tips for managing data
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
References: <4891042D.9040805@ooma.com>
	<48911EF0.1010004@ooma.com>	<F1745A9F580ACD44BBA4BAC95A69F04F449D64@rmail.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D6F@rmail.adiscon.com>
Message-ID: <48989FCB.9030609@ooma.com>

Andre Lorbach wrote:
> Hi again, 
>
> I want to inform you all that the search within phpLogCon has been
> extended to support full and partial searches in Version 2.5.3,
> available for download here: http://www.phplogcon.org/downloads
>
> I first tests have shown, that filtering by hostname and other string
> based fields performs much better on database sources now. 
> You can use the submenu buttons on each field within the messages view
> to perform such a search. If you want to do manual searches, it is very
> simple. Here is a sample: 
>
> Search for a full hostname: source:=SERVERNAME (Will only find if the
> match is 100% = SERVERNAME)
> Search for a partial hostname source:SERVER (Willmatch if the hostname
> is SERVER or SERVERNAME).  
>   
Thanks for the quick change.

Now I have another question, which is more sql but I thought I'd ask it 
anyway.

What I'd like to do is keep 3 months worth of data, but rotate and merge 
my tables after a set amount of time to keep search times low. Tables 
more than 3 months old would be dropped.

If anyone has a nice, canned script for this, it would be appreciated. 
Otherwise, I'll read the mysql manuals. 8-)

thx


From rory at ooma.com  Tue Aug  5 23:29:40 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:29:40 -0700
Subject: [rsyslog] phplogcon UI question
Message-ID: <4898C644.5060906@ooma.com>

Is there a way to turn off the hover-exploding info windows when you 
move the mouse over the "message" field?

On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
away.

thx


From rgerhards at hq.adiscon.com  Tue Aug  5 23:31:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 5 Aug 2008 23:31:56 +0200
Subject: [rsyslog] phplogcon UI question
Message-ID: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>

There is a config setting. But I thought it would be off by default?

rainer

----- Urspr?ngliche Nachricht -----
Von: "Rory Toma" <rory at ooma.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 05.08.08 23:29
Betreff: [rsyslog] phplogcon UI question

Is there a way to turn off the hover-exploding info windows when you 
move the mouse over the "message" field?

On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
away.

thx
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Tue Aug  5 23:35:42 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:35:42 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <4898C7AE.7030800@ooma.com>

Rory Toma wrote:
> Is there a way to turn off the hover-exploding info windows when you 
> move the mouse over the "message" field?
>
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not go 
> away.
>
> thx
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   
Firefox 3 does not have this issue.

Neither does IE7.


From rory at ooma.com  Tue Aug  5 23:38:24 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 14:38:24 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
References: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
Message-ID: <4898C850.1000508@ooma.com>

Rainer Gerhards wrote:
> There is a config setting. But I thought it would be off by default?
>
> rainer
>
>   


ViewEnableDetailPopups

thx


From rory at ooma.com  Wed Aug  6 03:07:46 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 18:07:46 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if query returns no
	results
Message-ID: <4898F962.1030200@ooma.com>

Minor issue, but thought I'd report it.

If i do a query that returns no results, I get a bunch of empty rows 
displayed in phpLogCon.




From rgerhards at hq.adiscon.com  Wed Aug  6 07:43:06 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 07:43:06 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if query
	returns noresults
In-Reply-To: <4898F962.1030200@ooma.com>
References: <4898F962.1030200@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>

Which version are you using? There was an issue with postgres, but that
has been fixed in recent (devel) builds.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 3:08 AM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon displays lots of empty rows if query
> returns noresults
> 
> Minor issue, but thought I'd report it.
> 
> If i do a query that returns no results, I get a bunch of empty rows
> displayed in phpLogCon.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Wed Aug  6 07:47:14 2008
From: rory at ooma.com (Rory Toma)
Date: Tue, 05 Aug 2008 22:47:14 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if
 query	returns noresults
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
References: <4898F962.1030200@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
Message-ID: <48993AE2.7050607@ooma.com>

I'm running 2.5.3.


Rainer Gerhards wrote:
> Which version are you using? There was an issue with postgres, but that
> has been fixed in recent (devel) builds.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> Sent: Wednesday, August 06, 2008 3:08 AM
>> To: rsyslog-users
>> Subject: [rsyslog] phplogcon displays lots of empty rows if query
>> returns noresults
>>
>> Minor issue, but thought I'd report it.
>>
>> If i do a query that returns no results, I get a bunch of empty rows
>> displayed in phpLogCon.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   



From rgerhards at hq.adiscon.com  Wed Aug  6 08:01:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 08:01:19 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if
	query	returns noresults
In-Reply-To: <48993AE2.7050607@ooma.com>
References: <4898F962.1030200@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>
	<48993AE2.7050607@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>

Can you provide the config of the data source so that Andre can create a
testbed?

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 7:47 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon displays lots of empty rows if query
> returns noresults
> 
> I'm running 2.5.3.
> 
> 
> Rainer Gerhards wrote:
> > Which version are you using? There was an issue with postgres, but
> that
> > has been fixed in recent (devel) builds.
> >
> > Rainer
> >
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> >> Sent: Wednesday, August 06, 2008 3:08 AM
> >> To: rsyslog-users
> >> Subject: [rsyslog] phplogcon displays lots of empty rows if query
> >> returns noresults
> >>
> >> Minor issue, but thought I'd report it.
> >>
> >> If i do a query that returns no results, I get a bunch of empty
rows
> >> displayed in phpLogCon.
> >>
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Wed Aug  6 10:20:34 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 6 Aug 2008 10:20:34 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C7AE.7030800@ooma.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:36 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> Rory Toma wrote:
> > Is there a way to turn off the hover-exploding info windows when you
> > move the mouse over the "message" field?
> >
> > On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> > away.
> >
> > thx
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> Firefox 3 does not have this issue.
> 
> Neither does IE7.

I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
unfortanetly don't have a MacOS machine to test Safari Browser. 

Best regards,
Andre Lorbach


From alorbach at ro1.adiscon.com  Wed Aug  6 10:21:32 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 6 Aug 2008 10:21:32 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C850.1000508@ooma.com>
References: <001901c8f742$cc3ddd25$060013ac@intern.adiscon.com>
	<4898C850.1000508@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D76@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:38 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> Rainer Gerhards wrote:
> > There is a config setting. But I thought it would be off by default?
> >
> > rainer
> >
> >
> 
> ViewEnableDetailPopups

If you install phpLogCon with UserDB System, you can turn this feature
off and on within the Admin Center ;).

Best regards,
Andre Lorbach


From rory at ooma.com  Wed Aug  6 22:58:41 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 13:58:41 -0700
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
Message-ID: <489A1081.1090609@ooma.com>


> I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
> unfortanetly don't have a MacOS machine to test Safari Browser. 
>
>   
What about the Windows version of Safari?

http://www.apple.com/downloads/macosx/apple/windows/


From rory at ooma.com  Wed Aug  6 22:59:23 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 13:59:23 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows
 if	query	returns noresults
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>
References: <4898F962.1030200@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF4A@grfint2.intern.adiscon.com>	<48993AE2.7050607@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF4D@grfint2.intern.adiscon.com>
Message-ID: <489A10AB.30101@ooma.com>

Rainer Gerhards wrote:
> Can you provide the config of the data source so that Andre can create a
> testbed?
>
> Thanks,
> Rainer
>
>   
>
Exactly what info would you like? I am using the default syslog schema 
in SystemEvents.




From rgerhards at hq.adiscon.com  Wed Aug  6 23:04:59 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Aug 2008 23:04:59 +0200
Subject: [rsyslog] phplogcon displays lots of empty rows if	queryreturns
	noresults
Message-ID: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>

The definition of the data source in question - should be in xour config file...

----- Urspr?ngliche Nachricht -----
Von: "Rory Toma" <rory at ooma.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 06.08.08 22:59
Betreff: Re: [rsyslog] phplogcon displays lots of empty rows if	query	returns noresults

Rainer Gerhards wrote:
> Can you provide the config of the data source so that Andre can create a
> testbed?
>
> Thanks,
> Rainer
>
>   
>
Exactly what info would you like? I am using the default syslog schema 
in SystemEvents.


_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From rory at ooma.com  Wed Aug  6 23:09:40 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 14:09:40 -0700
Subject: [rsyslog] phplogcon displays lots of empty rows if	queryreturns
 noresults
In-Reply-To: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>
References: <001c01c8f808$29fcc2c6$060013ac@intern.adiscon.com>
Message-ID: <489A1314.3040909@ooma.com>

Rainer Gerhards wrote:
> The definition of the data source in question - should be in xour config file...
>
>
>   

$CFG['DefaultSourceID'] = 'Source1';

$CFG['Sources']['Source1']['ID'] = 'Source1';
$CFG['Sources']['Source1']['Name'] = 'rsyslog';
$CFG['Sources']['Source1']['ViewID'] = 'SYSLOG';
$CFG['Sources']['Source1']['SourceType'] = SOURCE_DB;
$CFG['Sources']['Source1']['DBTableType'] = 'monitorware';
$CFG['Sources']['Source1']['DBServer'] = 'rsyslog.corp.ooma.com';
$CFG['Sources']['Source1']['DBName'] = 'Syslog';
$CFG['Sources']['Source1']['DBUser'] = 'xxx';
$CFG['Sources']['Source1']['DBPassword'] = 'xxx';
$CFG['Sources']['Source1']['DBTableName'] = 'SystemEvents';
$CFG['Sources']['Source1']['DBEnableRowCounting'] = false;




From rory at ooma.com  Thu Aug  7 01:53:49 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 16:53:49 -0700
Subject: [rsyslog] ok, here's my sql rotation...
Message-ID: <489A398D.6010402@ooma.com>

OK, so here's what I ended up doing...

I created SystemEvents1 thru SystemEvents90, and create SystemEvents as 
a union of 90 thru 1. (1 being last on the union line so it's the 
"active" table)

In cron.daily, I run

mysql <blah blah blah> < script.mysql

where script.mysql is

use Syslog;

CREATE TABLE `SystemEvents0` ( <table def>) Engine=MyISAM;

RENAME TABLE SystemEvents90 to SystemEvents91,
SystemEvents89 to SystemEvents 90,
SystemEvents88 to SystemEvents 89,
........
SystemEvents1 to SystemEvents2,
SystemEvents0 to SystemEvents1;

DROP TABLE SystemEvents91;

This gives me 90 days of logs at a reasonable performance level.

I'm kinda new to MySQL, so I took the route of ease of implementation.


From rory at ooma.com  Thu Aug  7 02:12:13 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 06 Aug 2008 17:12:13 -0700
Subject: [rsyslog] see sql queries on phplogcon
Message-ID: <489A3DDD.4000205@ooma.com>

How do I see what queries are being done in phplogcon? I don't have sql 
logging turned on in my db.

the := syntax seems to be no longer working for me. I get an error 
telling me there were no syslog records found when I try it.


I've set up my SystemEvents table as a union and merged all the tables 
in with data. This shouldn't affect it, I would think. My sql queries by 
hand seem to work fine.




From mic at npgx.com.au  Thu Aug  7 04:23:29 2008
From: mic at npgx.com.au (Michael Mansour)
Date: Thu, 7 Aug 2008 13:23:29 +1100
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <489A1081.1090609@ooma.com>
References: <4898C644.5060906@ooma.com> <4898C7AE.7030800@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
	<489A1081.1090609@ooma.com>
Message-ID: <20080807022212.M77981@npgx.com.au>

Hi guys,

> > I usually test phpLogCon with the Browsers Opera, Firefox and IE7. But I
> > unfortanetly don't have a MacOS machine to test Safari Browser. 
> >
> >   
> What about the Windows version of Safari?
> 
> http://www.apple.com/downloads/macosx/apple/windows/

There are websites out there which will freely test your (public) web page on
a number of browsers of your choice, sending you the results after a period of
time.

A web search will be able to find these sites.

Michael.


From rgerhards at hq.adiscon.com  Thu Aug  7 08:06:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 08:06:35 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <489A3DDD.4000205@ooma.com>
References: <489A3DDD.4000205@ooma.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>

Andre,

I think this triggers a new feature request: I know you have this in
debugging (at least in times), but could you (easily enough) add an
option that displays the SQL statement, e.g. at the top of the page?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 07, 2008 2:12 AM
> To: rsyslog-users
> Subject: [rsyslog] see sql queries on phplogcon
> 
> How do I see what queries are being done in phplogcon? I don't have
sql
> logging turned on in my db.
> 
> the := syntax seems to be no longer working for me. I get an error
> telling me there were no syslog records found when I try it.
> 
> 
> I've set up my SystemEvents table as a union and merged all the tables
> in with data. This shouldn't affect it, I would think. My sql queries
> by
> hand seem to work fine.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:08:49 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:08:49 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <489A1081.1090609@ooma.com>
References: <4898C644.5060906@ooma.com>
	<4898C7AE.7030800@ooma.com><F1745A9F580ACD44BBA4BAC95A69F04F449D75@rmail.adiscon.com>
	<489A1081.1090609@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7B@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Wednesday, August 06, 2008 10:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] phplogcon UI question
> 
> 
> > I usually test phpLogCon with the Browsers Opera, Firefox and IE7.
But I
> > unfortanetly don't have a MacOS machine to test Safari Browser.
> >
> >
> What about the Windows version of Safari?
> 
> http://www.apple.com/downloads/macosx/apple/windows/

Oh thanks I wasn't aware Safari was available for Windows ;). I will
download and install this browser and see what I can do. 

Best regards,
Andre Lorbach


From alorbach at ro1.adiscon.com  Thu Aug  7 10:11:07 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:11:07 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
References: <489A3DDD.4000205@ooma.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>

I agree, I will add some debug output so you can see the SQL queries
which are performed in background.
Currently I could only tell where to uncomment some debug code in the
logstream classes, but I recommend to wait until I added this feature. 

Best regards,
Andre Lorbach


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 8:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] see sql queries on phplogcon
> 
> Andre,
> 
> I think this triggers a new feature request: I know you have this in
> debugging (at least in times), but could you (easily enough) add an
> option that displays the SQL statement, e.g. at the top of the page?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > Sent: Thursday, August 07, 2008 2:12 AM
> > To: rsyslog-users
> > Subject: [rsyslog] see sql queries on phplogcon
> >
> > How do I see what queries are being done in phplogcon? I don't have
> sql
> > logging turned on in my db.
> >
> > the := syntax seems to be no longer working for me. I get an error
> > telling me there were no syslog records found when I try it.
> >
> >
> > I've set up my SystemEvents table as a union and merged all the
tables
> > in with data. This shouldn't affect it, I would think. My sql
queries
> > by
> > hand seem to work fine.
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:14:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:14:25 +0200
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <489A3DDD.4000205@ooma.com>
References: <489A3DDD.4000205@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>

One question, are you searching for a single word or a phrase? 
If you are searching for a phrase, please use + instead of spaces. 
For example, you want to find a syslogtag like "mysql safe", then you
use this search: 
source:=mysql+safe

If you do not use the + instead of the space, the filter will be
splitted at the word safe, which will be considered as message filter
then. 

Maybe this helps - best regards,
Andre Lorbach



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 07, 2008 2:12 AM
> To: rsyslog-users
> Subject: [rsyslog] see sql queries on phplogcon
> 
> How do I see what queries are being done in phplogcon? I don't have
sql
> logging turned on in my db.
> 
> the := syntax seems to be no longer working for me. I get an error
> telling me there were no syslog records found when I try it.
> 
> 
> I've set up my SystemEvents table as a union and merged all the tables
> in with data. This shouldn't affect it, I would think. My sql queries
by
> hand seem to work fine.
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 10:26:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 10:26:44 +0200
Subject: [rsyslog] debug code in phpLogCon - was: see sql queries on
	phplogcon
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
References: <489A3DDD.4000205@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>

In general, let me recommend something that has been proven quite
valuable in rsyslog: Many debug facilities in rsyslog are not commented
out or removed, but can be activated during runtime by switch(es). This
is quite useful when it comes down to troubleshooting issues with a
user's installation. All that is needed is to activate a debug setting
and you see, in all its glory ;), what is going on in that installation.
It may even be worth a thought if phpLogCon should support creating
debug files, which can be sent to you for hard to find cases. Not sure
if that makes sense for phpLogCon, though...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Thursday, August 07, 2008 10:11 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] see sql queries on phplogcon
> 
> I agree, I will add some debug output so you can see the SQL queries
> which are performed in background.
> Currently I could only tell where to uncomment some debug code in the
> logstream classes, but I recommend to wait until I added this feature.
> 
> Best regards,
> Andre Lorbach
> 
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, August 07, 2008 8:07 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] see sql queries on phplogcon
> >
> > Andre,
> >
> > I think this triggers a new feature request: I know you have this in
> > debugging (at least in times), but could you (easily enough) add an
> > option that displays the SQL statement, e.g. at the top of the page?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > > Sent: Thursday, August 07, 2008 2:12 AM
> > > To: rsyslog-users
> > > Subject: [rsyslog] see sql queries on phplogcon
> > >
> > > How do I see what queries are being done in phplogcon? I don't
have
> > sql
> > > logging turned on in my db.
> > >
> > > the := syntax seems to be no longer working for me. I get an error
> > > telling me there were no syslog records found when I try it.
> > >
> > >
> > > I've set up my SystemEvents table as a union and merged all the
> tables
> > > in with data. This shouldn't affect it, I would think. My sql
> queries
> > > by
> > > hand seem to work fine.
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:32:39 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:32:39 +0200
Subject: [rsyslog] debug code in phpLogCon - was: see sql queries
	onphplogcon
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>
References: <489A3DDD.4000205@ooma.com><577465F99B41C842AAFBE9ED71E70ABA44EF55@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449D7C@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF5A@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7E@rmail.adiscon.com>

We have some kind of a debug facility already in phpLogCon, it just
needs to be enabled and extended a little bit.
I think this will be one of the next minor new features to do then ;)!

--
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 10:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] debug code in phpLogCon - was: see sql queries
> onphplogcon
> 
> In general, let me recommend something that has been proven quite
> valuable in rsyslog: Many debug facilities in rsyslog are not
commented
> out or removed, but can be activated during runtime by switch(es).
This
> is quite useful when it comes down to troubleshooting issues with a
> user's installation. All that is needed is to activate a debug setting
> and you see, in all its glory ;), what is going on in that
installation.
> It may even be worth a thought if phpLogCon should support creating
> debug files, which can be sent to you for hard to find cases. Not sure
> if that makes sense for phpLogCon, though...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > Sent: Thursday, August 07, 2008 10:11 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] see sql queries on phplogcon
> >
> > I agree, I will add some debug output so you can see the SQL queries
> > which are performed in background.
> > Currently I could only tell where to uncomment some debug code in
the
> > logstream classes, but I recommend to wait until I added this
feature.
> >
> > Best regards,
> > Andre Lorbach
> >
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > Sent: Thursday, August 07, 2008 8:07 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] see sql queries on phplogcon
> > >
> > > Andre,
> > >
> > > I think this triggers a new feature request: I know you have this
in
> > > debugging (at least in times), but could you (easily enough) add
an
> > > option that displays the SQL statement, e.g. at the top of the
page?
> > >
> > > Rainer
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rory Toma
> > > > Sent: Thursday, August 07, 2008 2:12 AM
> > > > To: rsyslog-users
> > > > Subject: [rsyslog] see sql queries on phplogcon
> > > >
> > > > How do I see what queries are being done in phplogcon? I don't
> have
> > > sql
> > > > logging turned on in my db.
> > > >
> > > > the := syntax seems to be no longer working for me. I get an
error
> > > > telling me there were no syslog records found when I try it.
> > > >
> > > >
> > > > I've set up my SystemEvents table as a union and merged all the
> > tables
> > > > in with data. This shouldn't affect it, I would think. My sql
> > queries
> > > > by
> > > > hand seem to work fine.
> > > >
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 10:36:12 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 10:36:12 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:30 PM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon UI question
> 
> Is there a way to turn off the hover-exploding info windows when you
> move the mouse over the "message" field?
> 
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> away.

Ok I have Safari installed now, and I see what you mean. 
Actually this is some kind of a redraw issue within Safari, which does
not redraw the content which was behind the popup window. If you resize
the window after some popup messages have been viewed, you will see that
the content is redrawn. 

However I think we are going to work on this popup window anyway, as it
is not perfect yet. 

Best regards,
Andre Lorbach



From ml at darville.vm.bytemark.co.uk  Thu Aug  7 11:12:35 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Thu, 7 Aug 2008 10:12:35 +0100
Subject: [rsyslog] Changing hostname field
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
References: <20080801102649.GA27628@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
Message-ID: <20080807091235.GA16356@darville.vm.bytemark.co.uk>

On Fri, Aug 01, 2008 at 03:06:43PM +0200, Rainer Gerhards wrote:

> > > It also looks like I was wrong ;) I have reviewed the case. If I
> > > understand correctly, you get you data from the different unix
> > sockets
> > > and then forward via a single rule. So you can not use different
> > > templates (which most often are the real solution to any such
> > problem,
> > > but are not obvious to most people not directly involved in rsyslog
> > ;)).
> > >
> > > Is my understanding correct now?
> > 
> > Yes, and then I'm splitting log messages on the dedicated log host,
> > based on
> > the hostname field.
> > 
> > We are keeping the rsyslogd process outside the jails, to to keep the
> > syslog
> > traffic on an isolated network segment.
> 
> OK, that was fairly easy to do. I have created a special version for you:
> 
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz
> 
> It contains a new config directive that allows you to specify the hostname, see last sample on
> 
> http://www.rsyslog.com/doc-imuxsock.html
> 
> In my lab it works, but I would appreciate if you could test and verify.

Thank you very much, that is extremely good service - it works perfectly.

I owe you a beer for that.......


-- David Darville


From rgerhards at hq.adiscon.com  Thu Aug  7 10:42:14 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 10:42:14 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
References: <4898C644.5060906@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>

> Ok I have Safari installed now, and I see what you mean.
> Actually this is some kind of a redraw issue within Safari, which does
> not redraw the content which was behind the popup window. If you
resize
> the window after some popup messages have been viewed, you will see
> that
> the content is redrawn.
> 
> However I think we are going to work on this popup window anyway, as
it
> is not perfect yet.

Let me express my personal opinion in the hope that others will also
provide some input: I do *not* think it is well spent time to enhance
this Window. The nice thing is that it can be turned off and that should
solve most problems ;) Actually, I (again personally!) find the window
in itself quite problematic: if it is turned on, it blocks me from doing
some things in the natural way, because it always pops up and blocks
view of parts of the web page *just* because I happen to have moved the
mouse over some part of the page. For me, this leads to such funny
things that I need to concentrate on moving the mouse from bottom to top
(and not in the opposite direction ;)) when I want to activate some
context menu items - because otherwise the popup would block me from
reaching that point.

Again, this is just my personal view, but I would like to bring that
topic up to a broader audience.

Rainer


From rgerhards at hq.adiscon.com  Thu Aug  7 12:22:19 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 12:22:19 +0200
Subject: [rsyslog] Changing hostname field
In-Reply-To: <20080807091235.GA16356@darville.vm.bytemark.co.uk>
References: <20080801102649.GA27628@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF0B@grfint2.intern.adiscon.com>
	<20080807091235.GA16356@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF67@grfint2.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Thursday, August 07, 2008 11:13 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Changing hostname field
> 
> On Fri, Aug 01, 2008 at 03:06:43PM +0200, Rainer Gerhards wrote:
> 
> > > > It also looks like I was wrong ;) I have reviewed the case. If I
> > > > understand correctly, you get you data from the different unix
> > > sockets
> > > > and then forward via a single rule. So you can not use different
> > > > templates (which most often are the real solution to any such
> > > problem,
> > > > but are not obvious to most people not directly involved in
> rsyslog
> > > ;)).
> > > >
> > > > Is my understanding correct now?
> > >
> > > Yes, and then I'm splitting log messages on the dedicated log
host,
> > > based on
> > > the hostname field.
> > >
> > > We are keeping the rsyslogd process outside the jails, to to keep
> the
> > > syslog
> > > traffic on an isolated network segment.
> >
> > OK, that was fairly easy to do. I have created a special version for
> you:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-3.21.2-Test1.tar.gz
> >
> > It contains a new config directive that allows you to specify the
> hostname, see last sample on
> >
> > http://www.rsyslog.com/doc-imuxsock.html
> >
> > In my lab it works, but I would appreciate if you could test and
> verify.
> 
> Thank you very much, that is extremely good service - it works
> perfectly.
> 

Excellent!

> I owe you a beer for that.......

Be careful: I extremely well remember some things ;)

Rainer
> 
> 
> -- David Darville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From ml at darville.vm.bytemark.co.uk  Thu Aug  7 14:05:52 2008
From: ml at darville.vm.bytemark.co.uk (David Darville)
Date: Thu, 7 Aug 2008 13:05:52 +0100
Subject: [rsyslog] Problems with spooling to disk
Message-ID: <20080807120552.GA16724@darville.vm.bytemark.co.uk>

Hello everybody

Now I'm trying to configure rsyslog to spool log messages to disk, while the
connection to the central log collector is down. But it does not work......

I am using rsyslog v 3.21.2

I have configured the client rsyslog like this:

$ModLoad imuxsock
$ModLoad omrelp
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName rsyslog-buffer
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
*.* :omrelp:172.16.0.43:2514

Which is similar to the reliable forwarding configuration from the
documentation, except that I prefer to use RELP.

But When I test it, by stopping rsyslog on the central log collector, and
running this small shell script on the client:

#! /bin/sh
for ((i=1;i<=10000;i+=1)); do
  logger Log line $i
done

The shell script does not complete until I restart rsyslog, on the central
log collector, and when I look at the timestamps in the logfile on the log
collector, I see a large time difference between log line 3303 and 3304.

My plan is to get rsyslog to spool all log entries locally, while the
connection the the central log collector is interrupted, and then flush all
the spooled log messages to the central log collector when the communication
is restored - while the rest of the client system continues with business as
usual.

I can send you a debug log is needed.


-- David Darville


From rgerhards at hq.adiscon.com  Thu Aug  7 14:19:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 14:19:04 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>

>From what I see, this should work. So please send the debug log, client
(more important) and server please :)

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Darville
> Sent: Thursday, August 07, 2008 2:06 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Problems with spooling to disk
> 
> Hello everybody
> 
> Now I'm trying to configure rsyslog to spool log messages to disk,
> while the
> connection to the central log collector is down. But it does not
> work......
> 
> I am using rsyslog v 3.21.2
> 
> I have configured the client rsyslog like this:
> 
> $ModLoad imuxsock
> $ModLoad omrelp
> $WorkDirectory /var/spool/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueFileName rsyslog-buffer
> $ActionResumeRetryCount -1
> $ActionQueueSaveOnShutdown on
> *.* :omrelp:172.16.0.43:2514
> 
> Which is similar to the reliable forwarding configuration from the
> documentation, except that I prefer to use RELP.
> 
> But When I test it, by stopping rsyslog on the central log collector,
> and
> running this small shell script on the client:
> 
> #! /bin/sh
> for ((i=1;i<=10000;i+=1)); do
>   logger Log line $i
> done
> 
> The shell script does not complete until I restart rsyslog, on the
> central
> log collector, and when I look at the timestamps in the logfile on the
> log
> collector, I see a large time difference between log line 3303 and
> 3304.
> 
> My plan is to get rsyslog to spool all log entries locally, while the
> connection the the central log collector is interrupted, and then
flush
> all
> the spooled log messages to the central log collector when the
> communication
> is restored - while the rest of the client system continues with
> business as
> usual.
> 
> I can send you a debug log is needed.
> 
> 
> -- David Darville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From alorbach at ro1.adiscon.com  Thu Aug  7 16:41:14 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 7 Aug 2008 16:41:14 +0200
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <4898C644.5060906@ooma.com>
References: <4898C644.5060906@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D86@rmail.adiscon.com>

A new Version of the beta branch has been released a few moments ago
(mailing list should be informed very soon).
I have rewritten and added some Javascript code in order to enhance the
detail popup window. It does work now equal and very well in Firefox,
Opera and Internet Explorer. It looks a little bit better in Safari, but
it still has redraw problems. But I would say these are browser problems
with css. 

You can download the new Version from here: 
http://www.phplogcon.org/downloads

If you are working with the devel branch, you will have to wait until
tomorrow. 

Best regards,
Andre Lorbach


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Tuesday, August 05, 2008 11:30 PM
> To: rsyslog-users
> Subject: [rsyslog] phplogcon UI question
> 
> Is there a way to turn off the hover-exploding info windows when you
> move the mouse over the "message" field?
> 
> On Safari 3 on MacOS 10.5, it has display issues. Old windows do not
go
> away.
> 
> thx
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 17:00:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 17:00:25 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><ac9ded160808010725p4afdf54ard3aaf04b15da8501@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF0D@grfint2.intern.adiscon.com><ac9ded160808010817h7f040a08k284f65e126e761cb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF0E@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF6F@grfint2.intern.adiscon.com>

Just to prove the point: I have now implemented it. As expected, it is a
very small code change (once you know where to apply it ;)). Have a look
at the primary patch:

http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=action.c;h=3a2584de0f
24fdf3f1af0c35748d29fe5e3a3845;hp=f72194059d41f011ae6daf6b6aa1a61d147255
3e;hb=1480263b026984a3d48a7f750a78911777464797;hpb=0a7f964436af73f2e7fbd
403b563f8d5a743f4a5

and note that most is comment. The amount of code actually executed each
time is rater limited and does neither bear a large memory footprint,
nor execution footprint. If the feature is not used, it is one simple
branch. Even if it is used, the performance effects are very limited.
Most importantly, a costly call to time() could be avoided by using the
value that was already present (but needed a bit of reordering).

I am going to this detail just to explain the fine difference between
what belongs into the core (even though it is not "real core
functionality") compared to what must not get into it.

For example, if I had implemented that via a plugin, I would have needed
at least one (indirectly addressed) procedure call branch, still an if,
plus a return branch. Not good for speculative execution. Also, the
plumbing would probably have required more code than the full patch
shown (what means bad from a maintenance point of view as well a bad
from a CPU memory cache point of view ;)).

It remains the argument that if the code would not have been introduced,
the core would be a little more slim. That's right ;) An easy solution
would have been conditional compilation, but I have not applied it as I
think the few extra bytes and CPU cycles really don't matter (that
much).

Just for you info...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, August 01, 2008 5:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Actually, in this case the (limited) functionality will go into the
> core. But that's not because we need it for alerting but it is
> something
> that was scheduled for the core engine at all (because there are other
> use cases besides alerting where you need it, e.g. start some
> corrective
> action only after the n-th error indication). Plus, it will be very
> limited code. There are some other things, namely the enhanced rate
> limiter, which will go to the core. I have some ways to do that via
> plug-ins too, but given the expected complexity of this functionality
> vs
> the expected complexity of pluginizing them, the choice to go to core
> is
> really obvious. Another point where one could debate is enhanced
> queuing. This, in the long term, is a candidate for being moved to a
> plugin because many installations do never use disk-based (or
assisted)
> queues.
> 
> Now comes the important difference: if a generic, any message let me
> know when it happens n times in the row filter would have been needed,
> that would NOT go into the core. Because it does not belong there. It
> is
> quite complex and even performance intense. For that, future version
> will have customer (RainerScript) functions which can be provided by
> library plugins. I have plans to implement such a beast (much later),
> but it will come as a function that you provide the message to and
that
> is only loaded on an as-needed basis.
> 
> I hope this clarifies.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of (private) HKS
> > Sent: Friday, August 01, 2008 5:17 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> >
> > Thanks for the link, that post makes a lot of sense. I interpreted
> > this discussion as moving towards adding alert functionality to the
> > rsyslogd core - but your stance of keeping the rsyslogd core lean
and
> > efficient while plugins provide a full suite of event handling
> > processes is reassuring.
> >
> > I'm looking forward to seeing where you take this plugin
> architecture.
> >
> > -HKS
> >
> > On Fri, Aug 1, 2008 at 11:02 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> > > That's not a silly one ;) And it is coming up every now and then.
> The
> > > last time it came up, I was smart enough to write a blogpost:
> > > http://blog.gerhards.net/2008/03/on-rsyslog-design-philosophy-
> > plugins.ht
> > > ml
> > >
> > > In short, and to this questions: there are different schools of
> > thought.
> > > If you think about a plain ole syslogd shuffling data to disk
> files,
> > you
> > > do not need that. My vision of the syslogd (actually the "event
> > logging
> > > and alerting") subsystem is much broader. IMHO, it should support
> > > anything that is needed to gather, process and persistently store
> > > events. Also note that I say "events" for a reason - syslog
> messages
> > are
> > > just a subset of the potential set of events.
> > >
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> bounces at lists.adiscon.com] On Behalf Of (private) HKS
> > >> Sent: Friday, August 01, 2008 4:25 PM
> > >> To: rsyslog-users
> > >> Subject: Re: [rsyslog] Alert when multiple repeated lines are
> found
> > >>
> > >> This may be a silly question, but is the syslog daemon the proper
> > >> place for something like this?
> > >>
> > >> -HKS
> > >>
> > >> On Fri, Aug 1, 2008 at 6:13 AM, Julian Yap
<julianokyap at gmail.com>
> > >> wrote:
> > >> > Roger that Rainer.
> > >> >
> > >> > Thanks,
> > >> > Julian
> > >> >
> > >> > On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > >> > <rgerhards at hq.adiscon.com> wrote:
> > >> >> OK, that greatly simplifies things. Actually, it now boils
down
> > to
> > >> >> "execute an action only on the n-the time the filter evaluates
> to
> > >> true".
> > >> >> I think this is quite easy to implement, but I must verify
> > that...
> > >> >>
> > >> >> Rainer
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug  7 17:08:20 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Aug 2008 17:08:20 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>

Julian,

as you have probably seen in my other post, I have implemented the
functionality. I have now also created a test tarball. I'd appreciate if
you could obtain it from

http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz

and give it a try. Read ./doc/rsyslog_conf.html in regard to
$ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
what you intend to do, this should work:

$ActionExecOnlyEveryNthTime 3
*.* ..your action..

You don't need the timeout, but I have included it for completeness.
Well, actually if I were you I'd think if you really don't need it. Is
it really OK that "three in a row" means one each day?

Please provide feedback on this feature.

Thanks,
Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 01, 2008 12:14 PM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> Roger that Rainer.
> 
> Thanks,
> Julian
> 
> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > OK, that greatly simplifies things. Actually, it now boils down to
> > "execute an action only on the n-the time the filter evaluates to
> true".
> > I think this is quite easy to implement, but I must verify that...
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> Sent: Friday, August 01, 2008 11:03 AM
> >> To: Rainer Gerhards
> >> Cc: rsyslog at lists.adiscon.com
> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > found
> >>
> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >> <rgerhards at hq.adiscon.com> wrote:
> >> > Just one more re-confirmation:
> >> >
> >> >> What I'm looking for:
> >> >> if $msg == 'This is really bad' happens 3 times in a row then
> >> >> :ommail:;mailBody
> >> >>
> >> >> This would be nice but is not required since the 'This is really
> >> bad'
> >> >> message in my case is very unique:
> >> >> if ($msg == 'This is really bad' and $server == 'server' and
> >> $program
> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
> >> >
> >> > So you would actually use such a rule. If "this other thing is
> > really
> >> > bad" happened three times, the rule shall not trigger. Is this
> > right?
> >>
> >> Yes, I would use such a rule.  It would make what is already an
> >> awesome application even more awesome. :P  I am also willing to
test
> >> it out and run the latest development version...  Which I'm doing
> >> anyway.
> >>
> >> And yes, what you just wrote is correct.
> >>
> >> - Julian
> >


From friedl at hq.adiscon.com  Thu Aug  7 17:28:30 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Thu, 7 Aug 2008 17:28:30 +0200
Subject: [rsyslog] rsyslog 2.0.6 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF73@grfint2.intern.adiscon.com>

Hi all,

we have just released rsyslog 2.0.6, a version of the ultra-conservative
v2-stable branch. There is a single bugfix in this release: IPv6
addresses could not be specified in forwarding actions, because they
contain colons and the colon character was already used for some other
purpose. IPv6 addresses can now be specified inside of square brackets.
This is a recommended update for all v2-stable branch users.


Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-125.phtml

Changelog:

http://www.rsyslog.com/Article266.phtml


As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From julianokyap at gmail.com  Fri Aug  8 00:38:07 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 7 Aug 2008 12:38:07 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
Message-ID: <d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>

Rainer,

Initial testing looks fine.  I'll try some more to see if I can break it.

- Julian

On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Julian,
>
> as you have probably seen in my other post, I have implemented the
> functionality. I have now also created a test tarball. I'd appreciate if
> you could obtain it from
>
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>
> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
> what you intend to do, this should work:
>
> $ActionExecOnlyEveryNthTime 3
> *.* ..your action..
>
> You don't need the timeout, but I have included it for completeness.
> Well, actually if I were you I'd think if you really don't need it. Is
> it really OK that "three in a row" means one each day?
>
> Please provide feedback on this feature.
>
> Thanks,
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 01, 2008 12:14 PM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> Roger that Rainer.
>>
>> Thanks,
>> Julian
>>
>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>> > OK, that greatly simplifies things. Actually, it now boils down to
>> > "execute an action only on the n-the time the filter evaluates to
>> true".
>> > I think this is quite easy to implement, but I must verify that...
>> >
>> > Rainer
>> >
>> >> -----Original Message-----
>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >> Sent: Friday, August 01, 2008 11:03 AM
>> >> To: Rainer Gerhards
>> >> Cc: rsyslog at lists.adiscon.com
>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> > found
>> >>
>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> >> <rgerhards at hq.adiscon.com> wrote:
>> >> > Just one more re-confirmation:
>> >> >
>> >> >> What I'm looking for:
>> >> >> if $msg == 'This is really bad' happens 3 times in a row then
>> >> >> :ommail:;mailBody
>> >> >>
>> >> >> This would be nice but is not required since the 'This is really
>> >> bad'
>> >> >> message in my case is very unique:
>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>> >> $program
>> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>> >> >
>> >> > So you would actually use such a rule. If "this other thing is
>> > really
>> >> > bad" happened three times, the rule shall not trigger. Is this
>> > right?
>> >>
>> >> Yes, I would use such a rule.  It would make what is already an
>> >> awesome application even more awesome. :P  I am also willing to
> test
>> >> it out and run the latest development version...  Which I'm doing
>> >> anyway.
>> >>
>> >> And yes, what you just wrote is correct.
>> >>
>> >> - Julian
>> >
>


From julianokyap at gmail.com  Fri Aug  8 01:55:03 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 7 Aug 2008 13:55:03 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
Message-ID: <d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>

Yep, after further testing this works great!  Thanks Rainer.

On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com> wrote:
> Rainer,
>
> Initial testing looks fine.  I'll try some more to see if I can break it.
>
> - Julian
>
> On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> Julian,
>>
>> as you have probably seen in my other post, I have implemented the
>> functionality. I have now also created a test tarball. I'd appreciate if
>> you could obtain it from
>>
>> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>>
>> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout. For
>> what you intend to do, this should work:
>>
>> $ActionExecOnlyEveryNthTime 3
>> *.* ..your action..
>>
>> You don't need the timeout, but I have included it for completeness.
>> Well, actually if I were you I'd think if you really don't need it. Is
>> it really OK that "three in a row" means one each day?
>>
>> Please provide feedback on this feature.
>>
>> Thanks,
>> Rainer
>>
>>> -----Original Message-----
>>> From: Julian Yap [mailto:julianokyap at gmail.com]
>>> Sent: Friday, August 01, 2008 12:14 PM
>>> To: Rainer Gerhards
>>> Cc: rsyslog at lists.adiscon.com
>>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> found
>>>
>>> Roger that Rainer.
>>>
>>> Thanks,
>>> Julian
>>>
>>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>>> <rgerhards at hq.adiscon.com> wrote:
>>> > OK, that greatly simplifies things. Actually, it now boils down to
>>> > "execute an action only on the n-the time the filter evaluates to
>>> true".
>>> > I think this is quite easy to implement, but I must verify that...
>>> >
>>> > Rainer
>>> >
>>> >> -----Original Message-----
>>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>>> >> Sent: Friday, August 01, 2008 11:03 AM
>>> >> To: Rainer Gerhards
>>> >> Cc: rsyslog at lists.adiscon.com
>>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>>> > found
>>> >>
>>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>>> >> <rgerhards at hq.adiscon.com> wrote:
>>> >> > Just one more re-confirmation:
>>> >> >
>>> >> >> What I'm looking for:
>>> >> >> if $msg == 'This is really bad' happens 3 times in a row then
>>> >> >> :ommail:;mailBody
>>> >> >>
>>> >> >> This would be nice but is not required since the 'This is really
>>> >> bad'
>>> >> >> message in my case is very unique:
>>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>>> >> $program
>>> >> >> == 'program') happens 3 times in a row then :ommail:;mailBody
>>> >> >
>>> >> > So you would actually use such a rule. If "this other thing is
>>> > really
>>> >> > bad" happened three times, the rule shall not trigger. Is this
>>> > right?
>>> >>
>>> >> Yes, I would use such a rule.  It would make what is already an
>>> >> awesome application even more awesome. :P  I am also willing to
>> test
>>> >> it out and run the latest development version...  Which I'm doing
>>> >> anyway.
>>> >>
>>> >> And yes, what you just wrote is correct.
>>> >>
>>> >> - Julian
>>> >
>>
>


From mic at npgx.com.au  Fri Aug  8 03:58:18 2008
From: mic at npgx.com.au (Michael Mansour)
Date: Fri, 8 Aug 2008 12:58:18 +1100
Subject: [rsyslog] phplogcon UI question
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>
References: <4898C644.5060906@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7F@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF5C@grfint2.intern.adiscon.com>
Message-ID: <20080808014639.M81869@npgx.com.au>

Hi,

> > Ok I have Safari installed now, and I see what you mean.
> > Actually this is some kind of a redraw issue within Safari, which does
> > not redraw the content which was behind the popup window. If you
> resize
> > the window after some popup messages have been viewed, you will see
> > that
> > the content is redrawn.
> > 
> > However I think we are going to work on this popup window anyway, as
> it
> > is not perfect yet.
> 
> Let me express my personal opinion in the hope that others will also
> provide some input: I do *not* think it is well spent time to enhance
> this Window. The nice thing is that it can be turned off and that should
> solve most problems ;) Actually, I (again personally!) find the 
> window in itself quite problematic: if it is turned on, it blocks me 
> from doing some things in the natural way, because it always pops up 
> and blocks view of parts of the web page *just* because I happen to 
> have moved the mouse over some part of the page. For me, this leads 
> to such funny things that I need to concentrate on moving the mouse 
> from bottom to top
> (and not in the opposite direction ;)) when I want to activate some 
> context menu items - because otherwise the popup would block me from 
> reaching that point.
> 
> Again, this is just my personal view, but I would like to bring that
> topic up to a broader audience.

I can only describe my opinion by giving an example of a firefox extension I
like to use. 

I use an extension (or Add-on in FF3 terminology) called Cooliris Previews.
This extension is nifty in that it allows you to open a sub window (pop up
window) for URL's on web pages. From there you can "lock" the sub window down
and mouse over other URL's and it updates the sub windows.

I find this very handy when going through helpdesk cases in our helpdesk
system or emails in my webmial system.

The problem is, before bringing up the sub window, each URL you mouse over
brings up the cooliris previews icon to the right of the link (location
configurable). By default, mousing over the icon brings up the sub window.
People that use cooliris would quickly find this very annoying because sub
windows will constantly open up by accident. So after changing the default to
only allow opening of a sub window by clicking the cooliris icon, it's more
pleasant to use.

After a while though even this becomes annoying, because 90% of my browsing
time I don't use/need cooliris previews, and many times I accidentally click
the icon and bring up sub windows by accident again (you'll realise how easy
it is to make this mistake when using it).

So now I have the bottom right toolbar which enables/disables the extension,
so I no longer see the cooliris icon to the right of every URL I mouse over.

Sometimes though (when I have multiple FF Windows open) that toolbar icon
turns itself on automatically, and does so in every tab that's open in every
window. It's not perfect (and maybe the developers concentrate more on having
this feature enabled in a FF browser than not) but I have come to the point of
considering removing the extension because the pop ups can be annoying when
browsing.

Regards,

Michael.

> Rainer



From rgerhards at hq.adiscon.com  Fri Aug  8 10:39:14 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 10:39:14 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280807311650j689f5854o3a61c0386c520550@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEF8@grfint2.intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>

Thanks for the feedback, it will now be part of the next devel release,
I think some time next week :)

Rainer

> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Friday, August 08, 2008 1:55 AM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> Yep, after further testing this works great!  Thanks Rainer.
> 
> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
> wrote:
> > Rainer,
> >
> > Initial testing looks fine.  I'll try some more to see if I can
break
> it.
> >
> > - Julian
> >
> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >> Julian,
> >>
> >> as you have probably seen in my other post, I have implemented the
> >> functionality. I have now also created a test tarball. I'd
> appreciate if
> >> you could obtain it from
> >>
> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> >>
> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> >> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout.
> For
> >> what you intend to do, this should work:
> >>
> >> $ActionExecOnlyEveryNthTime 3
> >> *.* ..your action..
> >>
> >> You don't need the timeout, but I have included it for
completeness.
> >> Well, actually if I were you I'd think if you really don't need it.
> Is
> >> it really OK that "three in a row" means one each day?
> >>
> >> Please provide feedback on this feature.
> >>
> >> Thanks,
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> >>> Sent: Friday, August 01, 2008 12:14 PM
> >>> To: Rainer Gerhards
> >>> Cc: rsyslog at lists.adiscon.com
> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> >> found
> >>>
> >>> Roger that Rainer.
> >>>
> >>> Thanks,
> >>> Julian
> >>>
> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >>> <rgerhards at hq.adiscon.com> wrote:
> >>> > OK, that greatly simplifies things. Actually, it now boils down
> to
> >>> > "execute an action only on the n-the time the filter evaluates
to
> >>> true".
> >>> > I think this is quite easy to implement, but I must verify
> that...
> >>> >
> >>> > Rainer
> >>> >
> >>> >> -----Original Message-----
> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> >>> >> To: Rainer Gerhards
> >>> >> Cc: rsyslog at lists.adiscon.com
> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> are
> >>> > found
> >>> >>
> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >>> >> <rgerhards at hq.adiscon.com> wrote:
> >>> >> > Just one more re-confirmation:
> >>> >> >
> >>> >> >> What I'm looking for:
> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
then
> >>> >> >> :ommail:;mailBody
> >>> >> >>
> >>> >> >> This would be nice but is not required since the 'This is
> really
> >>> >> bad'
> >>> >> >> message in my case is very unique:
> >>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
> >>> >> $program
> >>> >> >> == 'program') happens 3 times in a row then
:ommail:;mailBody
> >>> >> >
> >>> >> > So you would actually use such a rule. If "this other thing
is
> >>> > really
> >>> >> > bad" happened three times, the rule shall not trigger. Is
this
> >>> > right?
> >>> >>
> >>> >> Yes, I would use such a rule.  It would make what is already an
> >>> >> awesome application even more awesome. :P  I am also willing to
> >> test
> >>> >> it out and run the latest development version...  Which I'm
> doing
> >>> >> anyway.
> >>> >>
> >>> >> And yes, what you just wrote is correct.
> >>> >>
> >>> >> - Julian
> >>> >
> >>
> >


From rgerhards at hq.adiscon.com  Fri Aug  8 12:28:11 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 12:28:11 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>

Thanks, I now received the log files. And, of course, they made
immediately clear what is going on ;) What you experience is flow
control kicking in. Let me first elaborate a bit:

Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
the local socket) and those who can not (like UDP). The idea is that we
can slow down senders where this does not result in loss of messages,
leaving room inside the queue on busy systems for those where we can not
do that. We had cases where not delaying caused a lot of grief.

There are also two watermarks, the low and high delay mark. If the low
delay mark is reached, a delayable source is delayed for one second. It
is at 70% of the (in-memory) queue capacity. Again, this is all done for
some very good reasons.

However, when I introduced these features, I did not make them
configurable. Also, I did classify the local unix socket as one who can
be delayed. In your scenario, this does not stop the system, but once
the local log socket is filled up, it needs to wait one second for each
message before it is pulled.

Again, the rate limiting logic is there for good reasons. So what I now
see is that I need to add a few config statements to allow to control it
according to your needs. I am also unsure if I should re-classify
imuxsock as an input which cannot be delayed - but that also can have
some drawbacks. Maybe the best thing to do is make its state
configurable and use "not to be delayed" as default.

Anyone with a thought on that?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, August 07, 2008 2:19 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> >From what I see, this should work. So please send the debug log,
> client
> (more important) and server please :)
> 
> Thanks,
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of David Darville
> > Sent: Thursday, August 07, 2008 2:06 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Problems with spooling to disk
> >
> > Hello everybody
> >
> > Now I'm trying to configure rsyslog to spool log messages to disk,
> > while the
> > connection to the central log collector is down. But it does not
> > work......
> >
> > I am using rsyslog v 3.21.2
> >
> > I have configured the client rsyslog like this:
> >
> > $ModLoad imuxsock
> > $ModLoad omrelp
> > $WorkDirectory /var/spool/rsyslog
> > $ActionQueueType LinkedList
> > $ActionQueueFileName rsyslog-buffer
> > $ActionResumeRetryCount -1
> > $ActionQueueSaveOnShutdown on
> > *.* :omrelp:172.16.0.43:2514
> >
> > Which is similar to the reliable forwarding configuration from the
> > documentation, except that I prefer to use RELP.
> >
> > But When I test it, by stopping rsyslog on the central log
collector,
> > and
> > running this small shell script on the client:
> >
> > #! /bin/sh
> > for ((i=1;i<=10000;i+=1)); do
> >   logger Log line $i
> > done
> >
> > The shell script does not complete until I restart rsyslog, on the
> > central
> > log collector, and when I look at the timestamps in the logfile on
> the
> > log
> > collector, I see a large time difference between log line 3303 and
> > 3304.
> >
> > My plan is to get rsyslog to spool all log entries locally, while
the
> > connection the the central log collector is interrupted, and then
> flush
> > all
> > the spooled log messages to the central log collector when the
> > communication
> > is restored - while the rest of the client system continues with
> > business as
> > usual.
> >
> > I can send you a debug log is needed.
> >
> >
> > -- David Darville
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Fri Aug  8 12:38:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Aug 2008 12:38:56 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF81@grfint2.intern.adiscon.com>

David,

I have done a quick and dirty (not to stay) fix, which makes imuxsock a
NO_DELAY input:

http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test4.tar.gz

I'd appreciate if you could give it a try and report back.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, August 08, 2008 12:28 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> Thanks, I now received the log files. And, of course, they made
> immediately clear what is going on ;) What you experience is flow
> control kicking in. Let me first elaborate a bit:
> 
> Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> the local socket) and those who can not (like UDP). The idea is that
we
> can slow down senders where this does not result in loss of messages,
> leaving room inside the queue on busy systems for those where we can
> not
> do that. We had cases where not delaying caused a lot of grief.
> 
> There are also two watermarks, the low and high delay mark. If the low
> delay mark is reached, a delayable source is delayed for one second.
It
> is at 70% of the (in-memory) queue capacity. Again, this is all done
> for
> some very good reasons.
> 
> However, when I introduced these features, I did not make them
> configurable. Also, I did classify the local unix socket as one who
can
> be delayed. In your scenario, this does not stop the system, but once
> the local log socket is filled up, it needs to wait one second for
each
> message before it is pulled.
> 
> Again, the rate limiting logic is there for good reasons. So what I
now
> see is that I need to add a few config statements to allow to control
> it
> according to your needs. I am also unsure if I should re-classify
> imuxsock as an input which cannot be delayed - but that also can have
> some drawbacks. Maybe the best thing to do is make its state
> configurable and use "not to be delayed" as default.
> 
> Anyone with a thought on that?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, August 07, 2008 2:19 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Problems with spooling to disk
> >
> > >From what I see, this should work. So please send the debug log,
> > client
> > (more important) and server please :)
> >
> > Thanks,
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of David Darville
> > > Sent: Thursday, August 07, 2008 2:06 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] Problems with spooling to disk
> > >
> > > Hello everybody
> > >
> > > Now I'm trying to configure rsyslog to spool log messages to disk,
> > > while the
> > > connection to the central log collector is down. But it does not
> > > work......
> > >
> > > I am using rsyslog v 3.21.2
> > >
> > > I have configured the client rsyslog like this:
> > >
> > > $ModLoad imuxsock
> > > $ModLoad omrelp
> > > $WorkDirectory /var/spool/rsyslog
> > > $ActionQueueType LinkedList
> > > $ActionQueueFileName rsyslog-buffer
> > > $ActionResumeRetryCount -1
> > > $ActionQueueSaveOnShutdown on
> > > *.* :omrelp:172.16.0.43:2514
> > >
> > > Which is similar to the reliable forwarding configuration from the
> > > documentation, except that I prefer to use RELP.
> > >
> > > But When I test it, by stopping rsyslog on the central log
> collector,
> > > and
> > > running this small shell script on the client:
> > >
> > > #! /bin/sh
> > > for ((i=1;i<=10000;i+=1)); do
> > >   logger Log line $i
> > > done
> > >
> > > The shell script does not complete until I restart rsyslog, on the
> > > central
> > > log collector, and when I look at the timestamps in the logfile on
> > the
> > > log
> > > collector, I see a large time difference between log line 3303 and
> > > 3304.
> > >
> > > My plan is to get rsyslog to spool all log entries locally, while
> the
> > > connection the the central log collector is interrupted, and then
> > flush
> > > all
> > > the spooled log messages to the central log collector when the
> > > communication
> > > is restored - while the rest of the client system continues with
> > > business as
> > > usual.
> > >
> > > I can send you a debug log is needed.
> > >
> > >
> > > -- David Darville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From friedl at hq.adiscon.com  Fri Aug  8 14:18:56 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 8 Aug 2008 14:18:56 +0200
Subject: [rsyslog] rsyslog 3.18.2 released (stable)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EF85@grfint2.intern.adiscon.com>

Hi all,

we have just released rsyslog 3.18.2, a version of the v3-stable branch.
There is a single bugfix in this release: IPv6 addresses could not be
specified in forwarding actions, because they contain colons and the
colon character was already used for some other purpose. IPv6 addresses
can now be specified inside of square brackets. This is the same fix
that was released for v2-stable yesterday. This is a recommended update
for all v3-stable branch users.


Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-126.phtml

Changelog:

http://www.rsyslog.com/Article268.phtml


As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From rory at ooma.com  Fri Aug  8 23:32:48 2008
From: rory at ooma.com (Rory Toma)
Date: Fri, 08 Aug 2008 14:32:48 -0700
Subject: [rsyslog] see sql queries on phplogcon
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>
References: <489A3DDD.4000205@ooma.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449D7D@rmail.adiscon.com>
Message-ID: <489CBB80.10507@ooma.com>

Actually, I ended up doing a "LIKE" search on hostname, looking at the 
query via full processlist and running the query by hand, assuming it 
was the same, just without the "LIKE" syntax.

Turns out I had a horked index in a table that MAX(id) exposed. Fixed it 
and moved on.

thx

Andre Lorbach wrote:
> One question, are you searching for a single word or a phrase? 
> If you are searching for a phrase, please use + instead of spaces. 
> For example, you want to find a syslogtag like "mysql safe", then you
> use this search: 
> source:=mysql+safe
>
> If you do not use the + instead of the space, the filter will be
> splitted at the word safe, which will be considered as message filter
> then. 
>
> Maybe this helps - best regards,
> Andre Lorbach
>
>
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rory Toma
>> Sent: Thursday, August 07, 2008 2:12 AM
>> To: rsyslog-users
>> Subject: [rsyslog] see sql queries on phplogcon
>>
>> How do I see what queries are being done in phplogcon? I don't have
>>     
> sql
>   
>> logging turned on in my db.
>>
>> the := syntax seems to be no longer working for me. I get an error
>> telling me there were no syslog records found when I try it.
>>
>>
>> I've set up my SystemEvents table as a union and merged all the tables
>> in with data. This shouldn't affect it, I would think. My sql queries
>>     
> by
>   
>> hand seem to work fine.
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>   



From mmcgrath at redhat.com  Tue Aug 12 17:37:35 2008
From: mmcgrath at redhat.com (Mike McGrath)
Date: Tue, 12 Aug 2008 10:37:35 -0500 (CDT)
Subject: [rsyslog] rsyslog dropping logs
In-Reply-To: <1216896024.7184.189.camel@rgf9dev.intern.adiscon.com>
References: <alpine.LFD.1.10.0807231413270.14749@laptop.mmcgrath.net>
	<1216896024.7184.189.camel@rgf9dev.intern.adiscon.com>
Message-ID: <alpine.LFD.1.10.0808121034550.5157@laptop.mmcgrath.net>

On Thu, 24 Jul 2008, Rainer Gerhards wrote:

> (I am not commenting on v2 vs. v3 as this is already done)
>
> First of all, we need to keep in mind that UDP is inherently lossy. Even
> when a frame is seen received by the local stack, it does not mean that
> it will eventually be forwarded to the application. If message bursts
> come in very quickly and the OS scheduler does not schedule the app fast
> enough to receive this messages (or the app is too slow in itself! ;))
> new frames may overwrite frames inside the stack's receive buffers.
>
> So it is always a good idea to avoid UDP if that's possible.
>
> HOWEVER, I, too, find it somewhat unusual that around 90% of all
> incoming frames are discarded before the rsyslog receiver could process
> them. One explanation I have is that you have bursts (or volume in
> general) that outperforms the configured actions. Having seen the config
> file, and seeing it does not include any database writer, it is hard to
> imagine this should happen, assuming reasonable hardware sizing is used.
> A cause could be excessive synchronous writes. Many rules do not put a
> dash in front of the file name and without it (in v2), every write is
> immediately synced. This is very costly. But still, I have never seen
> that this alone outperforms a system.
>
> To dig deeper into what is happening, a debug log would be most useful,
> together with the information which frames have been seen in tcpdump but
> NOT in one of the log files. You can enable debug mode via -dn command
> line switch and is recommended to run rsyslog interactively while doing
> so. Then, you can simply capture its output via stdout redirection.
> Please note that debug mode generates considerable output, and requires
> considerable additional processing time. In any case, though, it should
> show us where the bottleneck is. Please note that I need a consistent
> excerpt from the debug log that shows how things began and how it worked
> during the fault conditions. Usually, this means I need everything ;)
> Debug logs may also reveal sensitive information, even passwords, so you
> should be careful in what you do. I am used to log files around the size
> of 1GB. With reasonable compression, the transfer is usually not a
> problem (but I suggest you place them on a server for me to download).
> Download links and/or smaller logs you can email me privately at
> rgerhards at gmail.com (please NOT at my primary, adiscon, email address).
>
> I hope this helps and I am looking forward for the additional
> information.

So after a long hiatus and a new config the problems went away but only
temporarily.  I think because of a rebooted box.  They have returned.  I'm
going to mail the logs your way.  I can send you more.  Its very easy to
reproduce.

	-Mike









============== Snip ===================
>
> On Wed, 2008-07-23 at 14:21 -0500, Mike McGrath wrote:
> > I've got a RHEL5.2 host with rsyslog-2.0.0-11 installed as a central
> > logging server.  When running tcpdump I'm seeing all the udp packets
> > coming in but many of them are not getting logged.  And we're talking
> > like 10% or so getting logged (maybe less) and the rest are just lost.
> > I've attached my config file.
> >
> > (side note, if I'm doing something stupid in the config please correct me)
> >
> > 	-Mike
> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>



From julianokyap at gmail.com  Wed Aug 13 12:32:00 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Wed, 13 Aug 2008 00:32:00 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
Message-ID: <d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>

It's kind of strange but I've had this running for about a week now...
 I seem to have had 2 false alerts for no apparent reason.


On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Thanks for the feedback, it will now be part of the next devel release,
> I think some time next week :)
>
> Rainer
>
>> -----Original Message-----
>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> Sent: Friday, August 08, 2008 1:55 AM
>> To: Rainer Gerhards
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
>>
>> Yep, after further testing this works great!  Thanks Rainer.
>>
>> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
>> wrote:
>> > Rainer,
>> >
>> > Initial testing looks fine.  I'll try some more to see if I can
> break
>> it.
>> >
>> > - Julian
>> >
>> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> >> Julian,
>> >>
>> >> as you have probably seen in my other post, I have implemented the
>> >> functionality. I have now also created a test tarball. I'd
>> appreciate if
>> >> you could obtain it from
>> >>
>> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>> >>
>> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> >> $ActionExecOnlyEveryNthTime and $ActionExecOnlyEveryNthTimeTimeout.
>> For
>> >> what you intend to do, this should work:
>> >>
>> >> $ActionExecOnlyEveryNthTime 3
>> >> *.* ..your action..
>> >>
>> >> You don't need the timeout, but I have included it for
> completeness.
>> >> Well, actually if I were you I'd think if you really don't need it.
>> Is
>> >> it really OK that "three in a row" means one each day?
>> >>
>> >> Please provide feedback on this feature.
>> >>
>> >> Thanks,
>> >> Rainer
>> >>
>> >>> -----Original Message-----
>> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >>> Sent: Friday, August 01, 2008 12:14 PM
>> >>> To: Rainer Gerhards
>> >>> Cc: rsyslog at lists.adiscon.com
>> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> >> found
>> >>>
>> >>> Roger that Rainer.
>> >>>
>> >>> Thanks,
>> >>> Julian
>> >>>
>> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> >>> <rgerhards at hq.adiscon.com> wrote:
>> >>> > OK, that greatly simplifies things. Actually, it now boils down
>> to
>> >>> > "execute an action only on the n-the time the filter evaluates
> to
>> >>> true".
>> >>> > I think this is quite easy to implement, but I must verify
>> that...
>> >>> >
>> >>> > Rainer
>> >>> >
>> >>> >> -----Original Message-----
>> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> >>> >> Sent: Friday, August 01, 2008 11:03 AM
>> >>> >> To: Rainer Gerhards
>> >>> >> Cc: rsyslog at lists.adiscon.com
>> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
>> are
>> >>> > found
>> >>> >>
>> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> >>> >> <rgerhards at hq.adiscon.com> wrote:
>> >>> >> > Just one more re-confirmation:
>> >>> >> >
>> >>> >> >> What I'm looking for:
>> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
> then
>> >>> >> >> :ommail:;mailBody
>> >>> >> >>
>> >>> >> >> This would be nice but is not required since the 'This is
>> really
>> >>> >> bad'
>> >>> >> >> message in my case is very unique:
>> >>> >> >> if ($msg == 'This is really bad' and $server == 'server' and
>> >>> >> $program
>> >>> >> >> == 'program') happens 3 times in a row then
> :ommail:;mailBody
>> >>> >> >
>> >>> >> > So you would actually use such a rule. If "this other thing
> is
>> >>> > really
>> >>> >> > bad" happened three times, the rule shall not trigger. Is
> this
>> >>> > right?
>> >>> >>
>> >>> >> Yes, I would use such a rule.  It would make what is already an
>> >>> >> awesome application even more awesome. :P  I am also willing to
>> >> test
>> >>> >> it out and run the latest development version...  Which I'm
>> doing
>> >>> >> anyway.
>> >>> >>
>> >>> >> And yes, what you just wrote is correct.
>> >>> >>
>> >>> >> - Julian
>> >>> >
>> >>
>> >
>


From rgerhards at hq.adiscon.com  Wed Aug 13 12:47:36 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 12:47:36 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com>
	<d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
	<d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>

Do you use $ActionExecOnlyEveryNthTimeTimeout?

Rainer


> -----Original Message-----
> From: Julian Yap [mailto:julianokyap at gmail.com]
> Sent: Wednesday, August 13, 2008 12:32 PM
> To: Rainer Gerhards
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
found
> 
> It's kind of strange but I've had this running for about a week now...
>  I seem to have had 2 false alerts for no apparent reason.
> 
> 
> On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Thanks for the feedback, it will now be part of the next devel
> release,
> > I think some time next week :)
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> Sent: Friday, August 08, 2008 1:55 AM
> >> To: Rainer Gerhards
> >> Cc: rsyslog at lists.adiscon.com
> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > found
> >>
> >> Yep, after further testing this works great!  Thanks Rainer.
> >>
> >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap <julianokyap at gmail.com>
> >> wrote:
> >> > Rainer,
> >> >
> >> > Initial testing looks fine.  I'll try some more to see if I can
> > break
> >> it.
> >> >
> >> > - Julian
> >> >
> >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> >> > <rgerhards at hq.adiscon.com> wrote:
> >> >> Julian,
> >> >>
> >> >> as you have probably seen in my other post, I have implemented
> the
> >> >> functionality. I have now also created a test tarball. I'd
> >> appreciate if
> >> >> you could obtain it from
> >> >>
> >> >> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> >> >>
> >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> >> >> $ActionExecOnlyEveryNthTime and
> $ActionExecOnlyEveryNthTimeTimeout.
> >> For
> >> >> what you intend to do, this should work:
> >> >>
> >> >> $ActionExecOnlyEveryNthTime 3
> >> >> *.* ..your action..
> >> >>
> >> >> You don't need the timeout, but I have included it for
> > completeness.
> >> >> Well, actually if I were you I'd think if you really don't need
> it.
> >> Is
> >> >> it really OK that "three in a row" means one each day?
> >> >>
> >> >> Please provide feedback on this feature.
> >> >>
> >> >> Thanks,
> >> >> Rainer
> >> >>
> >> >>> -----Original Message-----
> >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> >>> Sent: Friday, August 01, 2008 12:14 PM
> >> >>> To: Rainer Gerhards
> >> >>> Cc: rsyslog at lists.adiscon.com
> >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> are
> >> >> found
> >> >>>
> >> >>> Roger that Rainer.
> >> >>>
> >> >>> Thanks,
> >> >>> Julian
> >> >>>
> >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> >> >>> <rgerhards at hq.adiscon.com> wrote:
> >> >>> > OK, that greatly simplifies things. Actually, it now boils
> down
> >> to
> >> >>> > "execute an action only on the n-the time the filter
evaluates
> > to
> >> >>> true".
> >> >>> > I think this is quite easy to implement, but I must verify
> >> that...
> >> >>> >
> >> >>> > Rainer
> >> >>> >
> >> >>> >> -----Original Message-----
> >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> >> >>> >> To: Rainer Gerhards
> >> >>> >> Cc: rsyslog at lists.adiscon.com
> >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
lines
> >> are
> >> >>> > found
> >> >>> >>
> >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> >> >>> >> <rgerhards at hq.adiscon.com> wrote:
> >> >>> >> > Just one more re-confirmation:
> >> >>> >> >
> >> >>> >> >> What I'm looking for:
> >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a row
> > then
> >> >>> >> >> :ommail:;mailBody
> >> >>> >> >>
> >> >>> >> >> This would be nice but is not required since the 'This is
> >> really
> >> >>> >> bad'
> >> >>> >> >> message in my case is very unique:
> >> >>> >> >> if ($msg == 'This is really bad' and $server == 'server'
> and
> >> >>> >> $program
> >> >>> >> >> == 'program') happens 3 times in a row then
> > :ommail:;mailBody
> >> >>> >> >
> >> >>> >> > So you would actually use such a rule. If "this other
thing
> > is
> >> >>> > really
> >> >>> >> > bad" happened three times, the rule shall not trigger. Is
> > this
> >> >>> > right?
> >> >>> >>
> >> >>> >> Yes, I would use such a rule.  It would make what is already
> an
> >> >>> >> awesome application even more awesome. :P  I am also willing
> to
> >> >> test
> >> >>> >> it out and run the latest development version...  Which I'm
> >> doing
> >> >>> >> anyway.
> >> >>> >>
> >> >>> >> And yes, what you just wrote is correct.
> >> >>> >>
> >> >>> >> - Julian
> >> >>> >
> >> >>
> >> >
> >


From r.bhatia at ipax.at  Wed Aug 13 13:04:07 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Aug 2008 13:04:07 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
Message-ID: <48A2BFA7.8000408@ipax.at>

hi,

Rainer Gerhards wrote:
> Thanks, I now received the log files. And, of course, they made
> immediately clear what is going on ;) What you experience is flow
> control kicking in. Let me first elaborate a bit:
> 
> Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> the local socket) and those who can not (like UDP). The idea is that we
> can slow down senders where this does not result in loss of messages,
> leaving room inside the queue on busy systems for those where we can not
> do that. We had cases where not delaying caused a lot of grief.
> 
> There are also two watermarks, the low and high delay mark. If the low
> delay mark is reached, a delayable source is delayed for one second. It
> is at 70% of the (in-memory) queue capacity. Again, this is all done for
> some very good reasons.
> 
> However, when I introduced these features, I did not make them
> configurable. Also, I did classify the local unix socket as one who can
> be delayed. In your scenario, this does not stop the system, but once
> the local log socket is filled up, it needs to wait one second for each
> message before it is pulled.
> 
> Again, the rate limiting logic is there for good reasons. So what I now
> see is that I need to add a few config statements to allow to control it
> according to your needs. I am also unsure if I should re-classify
> imuxsock as an input which cannot be delayed - but that also can have
> some drawbacks. Maybe the best thing to do is make its state
> configurable and use "not to be delayed" as default.

ok, after reading through your explanation, i (almost) understand the
delay mechanism and that at present it is not configurable.

almost, because i do not get the part where you talk about the "70% of
the (in-memory) queue capacity".

anyways, how does this relate to the original problem, that the
following code stalls the logging process.

>>> #! /bin/sh
>>> for ((i=1;i<=10000;i+=1)); do
>>>   logger Log line $i
>>> done

is this a capacity problem? is this another problem?

i really consider this a critical bug, as i yesterday ran out of
diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
rsyslog-client, which logs locally and to the remote destination,
stalled completely.

cheers,
raoul
ps: i am on 3.16.1-1~bpo40+1
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From rgerhards at hq.adiscon.com  Wed Aug 13 14:33:00 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 14:33:00 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <48A2BFA7.8000408@ipax.at>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk>
	<577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com>
	<48A2BFA7.8000408@ipax.at>
Message-ID: <1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>

On Wed, 2008-08-13 at 13:04 +0200, Raoul Bhatia [IPAX] wrote:
> hi,
> 
> Rainer Gerhards wrote:
> > Thanks, I now received the log files. And, of course, they made
> > immediately clear what is going on ;) What you experience is flow
> > control kicking in. Let me first elaborate a bit:
> > 
> > Rsyslog knows inputs which can be delayed a bit (like tcp receiver AND
> > the local socket) and those who can not (like UDP). The idea is that we
> > can slow down senders where this does not result in loss of messages,
> > leaving room inside the queue on busy systems for those where we can not
> > do that. We had cases where not delaying caused a lot of grief.
> > 
> > There are also two watermarks, the low and high delay mark. If the low
> > delay mark is reached, a delayable source is delayed for one second. It
> > is at 70% of the (in-memory) queue capacity. Again, this is all done for
> > some very good reasons.
> > 
> > However, when I introduced these features, I did not make them
> > configurable. Also, I did classify the local unix socket as one who can
> > be delayed. In your scenario, this does not stop the system, but once
> > the local log socket is filled up, it needs to wait one second for each
> > message before it is pulled.
> > 
> > Again, the rate limiting logic is there for good reasons. So what I now
> > see is that I need to add a few config statements to allow to control it
> > according to your needs. I am also unsure if I should re-classify
> > imuxsock as an input which cannot be delayed - but that also can have
> > some drawbacks. Maybe the best thing to do is make its state
> > configurable and use "not to be delayed" as default.
> 
> ok, after reading through your explanation, i (almost) understand the
> delay mechanism and that at present it is not configurable.
> 
> almost, because i do not get the part where you talk about the "70% of
> the (in-memory) queue capacity".

An example: if the configured queue capacity is 10,000 (the default
value for the main message queue), than no flow control at all happens
until 7,000 messages are inside the queue. Once they are (and only as
long as they are), flow control is applied to those inputs that permit
it (imuxsock permits it).

It doesn't matter here if the queue is disk-assisted or not. Even with
disk-assisted queues, this setting is very important to prevent an input
(e.g. imfile!) to send massive amounts of data to the queue when there
is no need to (because the input can be flow-controlled).

> 
> anyways, how does this relate to the original problem, that the
> following code stalls the logging process.
> 
> >>> #! /bin/sh
> >>> for ((i=1;i<=10000;i+=1)); do
> >>>   logger Log line $i
> >>> done
> 
> is this a capacity problem? is this another problem?

It is going over the flow control watermark, flow control kicks in and
delays imuxsock writes by one second each as long as there are more than
7,000 messages. No problem, intended behavior, but obviously
questionable. In the imuxsock case, there are also some bad cases.

> 
> i really consider this a critical bug, as i yesterday ran out of
> diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
> rsyslog-client, which logs locally and to the remote destination,
> stalled completely.

We'll - not completely, one message per second ;) But that's probably
complete enough ;)

As a quick fix, I think I'll make imuxsock a non-flow-controllable input
in the v3-stable (3.18.x). The devel will have a switch to make it flow
controlled if one desires. I'll also add some other knobs to fine-tune
the flow control algo, but that probably needs to wait until after my
summer break. These tweaks could easily break things and I don't like
the idea of doing them under time pressure ;)

HTH
Rainer
> 
> cheers,
> raoul
> ps: i am on 3.16.1-1~bpo40+1



From rgerhards at hq.adiscon.com  Wed Aug 13 14:39:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Aug 2008 14:39:44 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com><48A2BFA7.8000408@ipax.at>
	<1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>

This is the v3-stable fix:

http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=plugins/imuxsock/imux
sock.c;h=60ccaffbc526a03011be3ce5869e40bc3e32c319;hp=f8798039c5d53e58fc7
fa17807547862e08c999e;hb=8eb888d049da12e1294a7688432b6325794ade32;hpb=02
f768c37dac9dde424bbd31e378482750fc276c

You can also easily apply it to 3.16 - I just won't do it that version
has been superseded.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, August 13, 2008 2:33 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with spooling to disk
> 
> On Wed, 2008-08-13 at 13:04 +0200, Raoul Bhatia [IPAX] wrote:
> > hi,
> >
> > Rainer Gerhards wrote:
> > > Thanks, I now received the log files. And, of course, they made
> > > immediately clear what is going on ;) What you experience is flow
> > > control kicking in. Let me first elaborate a bit:
> > >
> > > Rsyslog knows inputs which can be delayed a bit (like tcp receiver
> AND
> > > the local socket) and those who can not (like UDP). The idea is
> that we
> > > can slow down senders where this does not result in loss of
> messages,
> > > leaving room inside the queue on busy systems for those where we
> can not
> > > do that. We had cases where not delaying caused a lot of grief.
> > >
> > > There are also two watermarks, the low and high delay mark. If the
> low
> > > delay mark is reached, a delayable source is delayed for one
> second. It
> > > is at 70% of the (in-memory) queue capacity. Again, this is all
> done for
> > > some very good reasons.
> > >
> > > However, when I introduced these features, I did not make them
> > > configurable. Also, I did classify the local unix socket as one
who
> can
> > > be delayed. In your scenario, this does not stop the system, but
> once
> > > the local log socket is filled up, it needs to wait one second for
> each
> > > message before it is pulled.
> > >
> > > Again, the rate limiting logic is there for good reasons. So what
I
> now
> > > see is that I need to add a few config statements to allow to
> control it
> > > according to your needs. I am also unsure if I should re-classify
> > > imuxsock as an input which cannot be delayed - but that also can
> have
> > > some drawbacks. Maybe the best thing to do is make its state
> > > configurable and use "not to be delayed" as default.
> >
> > ok, after reading through your explanation, i (almost) understand
the
> > delay mechanism and that at present it is not configurable.
> >
> > almost, because i do not get the part where you talk about the "70%
> of
> > the (in-memory) queue capacity".
> 
> An example: if the configured queue capacity is 10,000 (the default
> value for the main message queue), than no flow control at all happens
> until 7,000 messages are inside the queue. Once they are (and only as
> long as they are), flow control is applied to those inputs that permit
> it (imuxsock permits it).
> 
> It doesn't matter here if the queue is disk-assisted or not. Even with
> disk-assisted queues, this setting is very important to prevent an
> input
> (e.g. imfile!) to send massive amounts of data to the queue when there
> is no need to (because the input can be flow-controlled).
> 
> >
> > anyways, how does this relate to the original problem, that the
> > following code stalls the logging process.
> >
> > >>> #! /bin/sh
> > >>> for ((i=1;i<=10000;i+=1)); do
> > >>>   logger Log line $i
> > >>> done
> >
> > is this a capacity problem? is this another problem?
> 
> It is going over the flow control watermark, flow control kicks in and
> delays imuxsock writes by one second each as long as there are more
> than
> 7,000 messages. No problem, intended behavior, but obviously
> questionable. In the imuxsock case, there are also some bad cases.
> 
> >
> > i really consider this a critical bug, as i yesterday ran out of
> > diskspace on my remote-rsyslog-and-mysql-on-one-host-testbed and the
> > rsyslog-client, which logs locally and to the remote destination,
> > stalled completely.
> 
> We'll - not completely, one message per second ;) But that's probably
> complete enough ;)
> 
> As a quick fix, I think I'll make imuxsock a non-flow-controllable
> input
> in the v3-stable (3.18.x). The devel will have a switch to make it
flow
> controlled if one desires. I'll also add some other knobs to fine-tune
> the flow control algo, but that probably needs to wait until after my
> summer break. These tweaks could easily break things and I don't like
> the idea of doing them under time pressure ;)
> 
> HTH
> Rainer
> >
> > cheers,
> > raoul
> > ps: i am on 3.16.1-1~bpo40+1
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From r.bhatia at ipax.at  Wed Aug 13 15:13:48 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Aug 2008 15:13:48 +0200
Subject: [rsyslog] Problems with spooling to disk
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>
References: <20080807120552.GA16724@darville.vm.bytemark.co.uk><577465F99B41C842AAFBE9ED71E70ABA44EF6B@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44EF80@grfint2.intern.adiscon.com><48A2BFA7.8000408@ipax.at>	<1218630780.7184.368.camel@rgf9dev.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC8@grfint2.intern.adiscon.com>
Message-ID: <48A2DE0C.6010502@ipax.at>

Rainer Gerhards wrote:
> This is the v3-stable fix:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=blobdiff;f=plugins/imuxsock/imux
> sock.c;h=60ccaffbc526a03011be3ce5869e40bc3e32c319;hp=f8798039c5d53e58fc7
> fa17807547862e08c999e;hb=8eb888d049da12e1294a7688432b6325794ade32;hpb=02
> f768c37dac9dde424bbd31e378482750fc276c

thank you for your prompt fix. i will try to simulate the failure again.

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From rgerhards at hq.adiscon.com  Thu Aug 14 11:31:03 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Aug 2008 11:31:03 +0200
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com><d66707280808010047l174d5a51jbffe08124a1ba548@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EEFE@grfint2.intern.adiscon.com><d66707280808010202u64442478mccf478a53e1f3d8@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com><d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com><d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com><d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com><d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>

Just to keep everyone update. I got a private mail with some details not
to be posted publically. However, Julian stated that he does not use
$ActionExecOnlyEveryNthTimeTimeout.

With that (and the non-public) info, I think that the missing
$ActionExecOnlyEveryNthTimeTimeout is the culprit. I have not asked how
often the receiving syslogd is restarted or huped. Julian, please let me
know. If it is sufficiently seldom, I think it could be an "old"
message.

I think about this scenario: a message comes it at, eg., 10am. That's
count 1. Then, nothing happens for the next 10 hours. At 8pm, the next
one occurs. That's count 2. Another 5 hours later, the next message
occurs, bringing the total count to 3. Thus, this message now triggers
the rule.

The question is if this is desired behavior? Or should the rule only be
triggered if the messages occur within an e.g. 20 minute window? If the
later is the case, you need a

$ActionExecOnlyEveryNthTimeTimeout 1200

This directive will timeout previous messages seen if they are older
than 20 minutes. In the example above, the count would now be always 1
and consequently no rule would ever be triggered (and that's the reason
why this directive is present;)).

Does this make sense? Does it match what you see?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, August 13, 2008 12:48 PM
> To: Julian Yap
> Cc: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
> 
> Do you use $ActionExecOnlyEveryNthTimeTimeout?
> 
> Rainer
> 
> 
> > -----Original Message-----
> > From: Julian Yap [mailto:julianokyap at gmail.com]
> > Sent: Wednesday, August 13, 2008 12:32 PM
> > To: Rainer Gerhards
> > Cc: rsyslog at lists.adiscon.com
> > Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> found
> >
> > It's kind of strange but I've had this running for about a week
> now...
> >  I seem to have had 2 false alerts for no apparent reason.
> >
> >
> > On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> > > Thanks for the feedback, it will now be part of the next devel
> > release,
> > > I think some time next week :)
> > >
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> Sent: Friday, August 08, 2008 1:55 AM
> > >> To: Rainer Gerhards
> > >> Cc: rsyslog at lists.adiscon.com
> > >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
> > > found
> > >>
> > >> Yep, after further testing this works great!  Thanks Rainer.
> > >>
> > >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap
> <julianokyap at gmail.com>
> > >> wrote:
> > >> > Rainer,
> > >> >
> > >> > Initial testing looks fine.  I'll try some more to see if I can
> > > break
> > >> it.
> > >> >
> > >> > - Julian
> > >> >
> > >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
> > >> > <rgerhards at hq.adiscon.com> wrote:
> > >> >> Julian,
> > >> >>
> > >> >> as you have probably seen in my other post, I have implemented
> > the
> > >> >> functionality. I have now also created a test tarball. I'd
> > >> appreciate if
> > >> >> you could obtain it from
> > >> >>
> > >> >>
http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
> > >> >>
> > >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
> > >> >> $ActionExecOnlyEveryNthTime and
> > $ActionExecOnlyEveryNthTimeTimeout.
> > >> For
> > >> >> what you intend to do, this should work:
> > >> >>
> > >> >> $ActionExecOnlyEveryNthTime 3
> > >> >> *.* ..your action..
> > >> >>
> > >> >> You don't need the timeout, but I have included it for
> > > completeness.
> > >> >> Well, actually if I were you I'd think if you really don't
need
> > it.
> > >> Is
> > >> >> it really OK that "three in a row" means one each day?
> > >> >>
> > >> >> Please provide feedback on this feature.
> > >> >>
> > >> >> Thanks,
> > >> >> Rainer
> > >> >>
> > >> >>> -----Original Message-----
> > >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> >>> Sent: Friday, August 01, 2008 12:14 PM
> > >> >>> To: Rainer Gerhards
> > >> >>> Cc: rsyslog at lists.adiscon.com
> > >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
> > are
> > >> >> found
> > >> >>>
> > >> >>> Roger that Rainer.
> > >> >>>
> > >> >>> Thanks,
> > >> >>> Julian
> > >> >>>
> > >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
> > >> >>> <rgerhards at hq.adiscon.com> wrote:
> > >> >>> > OK, that greatly simplifies things. Actually, it now boils
> > down
> > >> to
> > >> >>> > "execute an action only on the n-the time the filter
> evaluates
> > > to
> > >> >>> true".
> > >> >>> > I think this is quite easy to implement, but I must verify
> > >> that...
> > >> >>> >
> > >> >>> > Rainer
> > >> >>> >
> > >> >>> >> -----Original Message-----
> > >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
> > >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
> > >> >>> >> To: Rainer Gerhards
> > >> >>> >> Cc: rsyslog at lists.adiscon.com
> > >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
> lines
> > >> are
> > >> >>> > found
> > >> >>> >>
> > >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
> > >> >>> >> <rgerhards at hq.adiscon.com> wrote:
> > >> >>> >> > Just one more re-confirmation:
> > >> >>> >> >
> > >> >>> >> >> What I'm looking for:
> > >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a
row
> > > then
> > >> >>> >> >> :ommail:;mailBody
> > >> >>> >> >>
> > >> >>> >> >> This would be nice but is not required since the 'This
> is
> > >> really
> > >> >>> >> bad'
> > >> >>> >> >> message in my case is very unique:
> > >> >>> >> >> if ($msg == 'This is really bad' and $server ==
'server'
> > and
> > >> >>> >> $program
> > >> >>> >> >> == 'program') happens 3 times in a row then
> > > :ommail:;mailBody
> > >> >>> >> >
> > >> >>> >> > So you would actually use such a rule. If "this other
> thing
> > > is
> > >> >>> > really
> > >> >>> >> > bad" happened three times, the rule shall not trigger.
Is
> > > this
> > >> >>> > right?
> > >> >>> >>
> > >> >>> >> Yes, I would use such a rule.  It would make what is
> already
> > an
> > >> >>> >> awesome application even more awesome. :P  I am also
> willing
> > to
> > >> >> test
> > >> >>> >> it out and run the latest development version...  Which
I'm
> > >> doing
> > >> >>> >> anyway.
> > >> >>> >>
> > >> >>> >> And yes, what you just wrote is correct.
> > >> >>> >>
> > >> >>> >> - Julian
> > >> >>> >
> > >> >>
> > >> >
> > >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From julianokyap at gmail.com  Thu Aug 14 12:30:02 2008
From: julianokyap at gmail.com (Julian Yap)
Date: Thu, 14 Aug 2008 00:30:02 -1000
Subject: [rsyslog] Alert when multiple repeated lines are found
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>
References: <001501c8f34d$7b4db75c$060013ac@intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF00@grfint2.intern.adiscon.com>
	<d66707280808010313v9414316v50cdb34638eed750@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF70@grfint2.intern.adiscon.com>
	<d66707280808071538y949972fm4e60fb927e6f6e59@mail.gmail.com>
	<d66707280808071655t1f9af0a3gb243a8ef50b682b9@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EF79@grfint2.intern.adiscon.com>
	<d66707280808130332h728124b3o1dc0279bf3691efb@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFC3@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44EFE1@grfint2.intern.adiscon.com>
Message-ID: <d66707280808140330i77592c8fk56da76125736da63@mail.gmail.com>

Rainer,

Yes, after looking at it some more, I guess I do need to have the
$ActionExecOnlyEveryNthTimeTimeout setting.  Your explanation below
clarifies things a lot.

I guess I was confused with my existing
$ActionExecOnlyOnceEveryInterval setting and thinking that was
similar.

That should fix my issue.

- Julian

On Wed, Aug 13, 2008 at 11:31 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Just to keep everyone update. I got a private mail with some details not
> to be posted publically. However, Julian stated that he does not use
> $ActionExecOnlyEveryNthTimeTimeout.
>
> With that (and the non-public) info, I think that the missing
> $ActionExecOnlyEveryNthTimeTimeout is the culprit. I have not asked how
> often the receiving syslogd is restarted or huped. Julian, please let me
> know. If it is sufficiently seldom, I think it could be an "old"
> message.
>
> I think about this scenario: a message comes it at, eg., 10am. That's
> count 1. Then, nothing happens for the next 10 hours. At 8pm, the next
> one occurs. That's count 2. Another 5 hours later, the next message
> occurs, bringing the total count to 3. Thus, this message now triggers
> the rule.
>
> The question is if this is desired behavior? Or should the rule only be
> triggered if the messages occur within an e.g. 20 minute window? If the
> later is the case, you need a
>
> $ActionExecOnlyEveryNthTimeTimeout 1200
>
> This directive will timeout previous messages seen if they are older
> than 20 minutes. In the example above, the count would now be always 1
> and consequently no rule would ever be triggered (and that's the reason
> why this directive is present;)).
>
> Does this make sense? Does it match what you see?
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Wednesday, August 13, 2008 12:48 PM
>> To: Julian Yap
>> Cc: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] Alert when multiple repeated lines are found
>>
>> Do you use $ActionExecOnlyEveryNthTimeTimeout?
>>
>> Rainer
>>
>>
>> > -----Original Message-----
>> > From: Julian Yap [mailto:julianokyap at gmail.com]
>> > Sent: Wednesday, August 13, 2008 12:32 PM
>> > To: Rainer Gerhards
>> > Cc: rsyslog at lists.adiscon.com
>> > Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> found
>> >
>> > It's kind of strange but I've had this running for about a week
>> now...
>> >  I seem to have had 2 false alerts for no apparent reason.
>> >
>> >
>> > On Thu, Aug 7, 2008 at 10:39 PM, Rainer Gerhards
>> > <rgerhards at hq.adiscon.com> wrote:
>> > > Thanks for the feedback, it will now be part of the next devel
>> > release,
>> > > I think some time next week :)
>> > >
>> > > Rainer
>> > >
>> > >> -----Original Message-----
>> > >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> Sent: Friday, August 08, 2008 1:55 AM
>> > >> To: Rainer Gerhards
>> > >> Cc: rsyslog at lists.adiscon.com
>> > >> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines are
>> > > found
>> > >>
>> > >> Yep, after further testing this works great!  Thanks Rainer.
>> > >>
>> > >> On Thu, Aug 7, 2008 at 12:38 PM, Julian Yap
>> <julianokyap at gmail.com>
>> > >> wrote:
>> > >> > Rainer,
>> > >> >
>> > >> > Initial testing looks fine.  I'll try some more to see if I can
>> > > break
>> > >> it.
>> > >> >
>> > >> > - Julian
>> > >> >
>> > >> > On Thu, Aug 7, 2008 at 5:08 AM, Rainer Gerhards
>> > >> > <rgerhards at hq.adiscon.com> wrote:
>> > >> >> Julian,
>> > >> >>
>> > >> >> as you have probably seen in my other post, I have implemented
>> > the
>> > >> >> functionality. I have now also created a test tarball. I'd
>> > >> appreciate if
>> > >> >> you could obtain it from
>> > >> >>
>> > >> >>
> http://download.rsyslog.com/rsyslog/rsyslog-3.21.3-Test3.tar.gz
>> > >> >>
>> > >> >> and give it a try. Read ./doc/rsyslog_conf.html in regard to
>> > >> >> $ActionExecOnlyEveryNthTime and
>> > $ActionExecOnlyEveryNthTimeTimeout.
>> > >> For
>> > >> >> what you intend to do, this should work:
>> > >> >>
>> > >> >> $ActionExecOnlyEveryNthTime 3
>> > >> >> *.* ..your action..
>> > >> >>
>> > >> >> You don't need the timeout, but I have included it for
>> > > completeness.
>> > >> >> Well, actually if I were you I'd think if you really don't
> need
>> > it.
>> > >> Is
>> > >> >> it really OK that "three in a row" means one each day?
>> > >> >>
>> > >> >> Please provide feedback on this feature.
>> > >> >>
>> > >> >> Thanks,
>> > >> >> Rainer
>> > >> >>
>> > >> >>> -----Original Message-----
>> > >> >>> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> >>> Sent: Friday, August 01, 2008 12:14 PM
>> > >> >>> To: Rainer Gerhards
>> > >> >>> Cc: rsyslog at lists.adiscon.com
>> > >> >>> Subject: Re: Re: [rsyslog] Alert when multiple repeated lines
>> > are
>> > >> >> found
>> > >> >>>
>> > >> >>> Roger that Rainer.
>> > >> >>>
>> > >> >>> Thanks,
>> > >> >>> Julian
>> > >> >>>
>> > >> >>> On Thu, Jul 31, 2008 at 11:58 PM, Rainer Gerhards
>> > >> >>> <rgerhards at hq.adiscon.com> wrote:
>> > >> >>> > OK, that greatly simplifies things. Actually, it now boils
>> > down
>> > >> to
>> > >> >>> > "execute an action only on the n-the time the filter
>> evaluates
>> > > to
>> > >> >>> true".
>> > >> >>> > I think this is quite easy to implement, but I must verify
>> > >> that...
>> > >> >>> >
>> > >> >>> > Rainer
>> > >> >>> >
>> > >> >>> >> -----Original Message-----
>> > >> >>> >> From: Julian Yap [mailto:julianokyap at gmail.com]
>> > >> >>> >> Sent: Friday, August 01, 2008 11:03 AM
>> > >> >>> >> To: Rainer Gerhards
>> > >> >>> >> Cc: rsyslog at lists.adiscon.com
>> > >> >>> >> Subject: Re: Re: [rsyslog] Alert when multiple repeated
>> lines
>> > >> are
>> > >> >>> > found
>> > >> >>> >>
>> > >> >>> >> On Thu, Jul 31, 2008 at 10:18 PM, Rainer Gerhards
>> > >> >>> >> <rgerhards at hq.adiscon.com> wrote:
>> > >> >>> >> > Just one more re-confirmation:
>> > >> >>> >> >
>> > >> >>> >> >> What I'm looking for:
>> > >> >>> >> >> if $msg == 'This is really bad' happens 3 times in a
> row
>> > > then
>> > >> >>> >> >> :ommail:;mailBody
>> > >> >>> >> >>
>> > >> >>> >> >> This would be nice but is not required since the 'This
>> is
>> > >> really
>> > >> >>> >> bad'
>> > >> >>> >> >> message in my case is very unique:
>> > >> >>> >> >> if ($msg == 'This is really bad' and $server ==
> 'server'
>> > and
>> > >> >>> >> $program
>> > >> >>> >> >> == 'program') happens 3 times in a row then
>> > > :ommail:;mailBody
>> > >> >>> >> >
>> > >> >>> >> > So you would actually use such a rule. If "this other
>> thing
>> > > is
>> > >> >>> > really
>> > >> >>> >> > bad" happened three times, the rule shall not trigger.
> Is
>> > > this
>> > >> >>> > right?
>> > >> >>> >>
>> > >> >>> >> Yes, I would use such a rule.  It would make what is
>> already
>> > an
>> > >> >>> >> awesome application even more awesome. :P  I am also
>> willing
>> > to
>> > >> >> test
>> > >> >>> >> it out and run the latest development version...  Which
> I'm
>> > >> doing
>> > >> >>> >> anyway.
>> > >> >>> >>
>> > >> >>> >> And yes, what you just wrote is correct.
>> > >> >>> >>
>> > >> >>> >> - Julian
>> > >> >>> >
>> > >> >>
>> > >> >
>> > >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From klas.lindfors at it.su.se  Fri Aug 15 09:37:14 2008
From: klas.lindfors at it.su.se (Klas Lindfors)
Date: Fri, 15 Aug 2008 09:37:14 +0200
Subject: [rsyslog] patch adding option for testing configuration
Message-ID: <48A5322A.9080400@it.su.se>

Hello,

I've attached a patch to this message adding a config option (-T) to run 
rsyslog with configuration testing, immediatly exiting with 0 if the 
conf is valid. For us this is an important feature to make sure a 
configuration is valid before starting the daemon.

The style of the patch is probably not acceptable for inclusion, but it 
does it's job, so maybe a variant of it can be included.

Regards,

Klas Lindfors
Stockholm University

From rgerhards at hq.adiscon.com  Fri Aug 15 11:37:15 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Aug 2008 11:37:15 +0200
Subject: [rsyslog] patch adding option for testing configuration
In-Reply-To: <48A5322A.9080400@it.su.se>
References: <48A5322A.9080400@it.su.se>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFF0@grfint2.intern.adiscon.com>

Hi Klas,

the patch did not make it to the mailing list (the list processor is
very picky to prevent spreading spam and malware). However, from you
description it sounds like the -N option which has recently (3.21.1)
been introduced. Please see

http://www.rsyslog.com/doc-troubleshoot.html

under "configuration problems". I don't know out of my head if the exit
code works in the way you describe, but that should be an easy add. I
suggest you file an enhancement request if in doubt
(http://bugzilla.adiscon.com). I'll be soon away for summer break, so
that needs to wait until after it ;)

Please let me know if -N goes into the right direction.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Klas Lindfors
> Sent: Friday, August 15, 2008 9:37 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] patch adding option for testing configuration
> 
> Hello,
> 
> I've attached a patch to this message adding a config option (-T) to
> run
> rsyslog with configuration testing, immediatly exiting with 0 if the
> conf is valid. For us this is an important feature to make sure a
> configuration is valid before starting the daemon.
> 
> The style of the patch is probably not acceptable for inclusion, but
it
> does it's job, so maybe a variant of it can be included.
> 
> Regards,
> 
> Klas Lindfors
> Stockholm University


From friedl at hq.adiscon.com  Fri Aug 15 14:07:34 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Fri, 15 Aug 2008 14:07:34 +0200
Subject: [rsyslog] rsyslog 3.21.3 (devel) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44EFFF@grfint2.intern.adiscon.com>

Hi all,

I have just released 3.21.3, a devel branch version. It provides support
for specifying custom MySQL server ports and adds the ability to execute
an action only ever n-th time. Also, flow control for imuxsock messages
can now be turned on or off. There are also some bug fixes, among others
one to fully support IPv6 hosts in forwarding actions. This is a
recommended update for all devel branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-127.phtml

Changelog:

http://www.rsyslog.com/Article270.phtml

As always, feedback is appreciated.

Florian Riedl


From friedl at hq.adiscon.com  Mon Aug 18 17:16:49 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 18 Aug 2008 17:16:49 +0200
Subject: [rsyslog] rsyslog 3.18.3 released (stable)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F006@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 3.18.3, a member of the v3-stable branch,
today. This is primarily a bug-fixing release. It fixes a problem that
causes imfile to segfault on termination, disables flow control in
imuxsock which could lead to system hang conditions and corrects a
misspelled configuration directive. There is a small enhancement which
now permits to use MySQL custom server ports. This is a recommended
update for all v3-stable branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-128.phtml

Changelog:

http://www.rsyslog.com/Article271.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .





From stefan.wold at it.su.se  Wed Aug 20 07:40:19 2008
From: stefan.wold at it.su.se (Stefan Wold)
Date: Wed, 20 Aug 2008 07:40:19 +0200
Subject: [rsyslog] rsyslog with RELP resolving issue
Message-ID: <1219210819.4967.18.camel@localhost>

Hi,

I've been testing rsyslog versions 3.18.2 and 3.21.3 with RELP. My
problem here is that no matter what I try I can't get the receiving
rsyslog to do hostname resolving. If I disable RELP and go back to
either tcp or udp it will resolve my hosts properly. On the receiving
side I have a template that create one log file per host with the
%HOSTNAME% variable, if I try using %fromhost% and RELP I end up with an
unresolved ip-address.

Is there a hidden option I'm not aware of or is resolving just not
working properly with RELP yet?

Secondly I wonder if adding a option that force rsyslog to use FQDN for
everything including hostnames within the same domain as the logserver
would be plausible? 

-- 
Sincerely,
Stefan Wold
IT Services, Stockholm University, Sweden



From satoru.satoh at gmail.com  Wed Aug 20 17:31:19 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 00:31:19 +0900
Subject: [rsyslog] [PATCH 00/07] man-i18n patches
Message-ID: <20080820153118.GC20566@localhost.localdomain>

Hello / Guten tag,


Rio-san (rfujita at redhat.com) previously told about his idea of
documentation i18n / l10n for rsyslog. I'm one of his colleagues and
originally told him about related topics.


A series of patches I'll post right after this mail are to implement
this i18n / l10n workflow for man pages into normal build process.

These patches enable to 

 * generate original English man pages from DocBook XML files
 * generate translated non-English man pages from translated DocBook
   XML files, which can be generated from PO files (PO is much suitable
   format for translation maintainace than XML, I guess)
 * create POT (PO template)

And this automated man-pages-generation process is only performed when
the configure option "--enable-regenerate-man" is specified. So, side
effects of these patches should be kept to a minimum, I think.

- satoru


From satoru.satoh at gmail.com  Wed Aug 20 19:07:40 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 02:07:40 +0900
Subject: [rsyslog] [PATCH] man-i18n patches summary
Message-ID: <20080820170735.GB20932@localhost.localdomain>

Hello,


I posted the first patch but it seems the patch was too big to post. I
list them here at once.

Could you please take some time to review these and merge them?


I splitted origianl big patch into minimum units. However, to complete
to man-i18n support, the following patches must be applied in this
order.

All of the patches below are available from
http://www.gnome.gr.jp/~ss/t/rsyslog/.


1. 0001-man-i18n-move-man-files-from-tools-to-man-C.patch

This patch just moves man files from tools/ to man/C/ (new dir to hold
man related stuff) and update related Makefile.am files.


2. 0002-man-i18n-add-man-ja.patch

This patch creates new dir, man/ja/ and Makefile.am in it.

There are no man files in it and this should not have harmful effects
for build process at all, I guess.


3. 0003-man-i18n-add-man-ja.patch

This patch adds man files in man/ja and update Makefile.am in it to
install these.


4. 0004-man-i18n-add-docbook-xml-sources-for-man-files.patch

This patch adds docbook XML sources for man pages.


5. 0005-man-i18n-add-m4-macros-in-m4-and-make-aclocal-look.patch

This patch adds m4 macros in m4/ and make aclocal look for this dir.

These m4 files are not evaluated in actual and this should not have
adverse effect at all, I guess.


6. 0006-man-i18n-add-build-process-of-man-pages-in-Makefile.patch

This patch adds transformation process of man pages and po files in
Makefile.am.

Added part are in "if ENABLE_REGENERATE_MAN ... endif" and
ENABLE_REGENERATE_MAN is not defined at this point, so this also should
not have side effects.


7. 0007-man-i18n-add-enable-regenerate-man-option-for-con.patch

This is the last one. This patch adds "--enable-regenerate-man"
configure option to enable man build process.

This option is disabled by default and should not affect normal build
process, I think.


How to build:

To enable the man build process, configure have to be regenerated:

  $ (autoreconf -vfi && \
        ./configure --enable-regenerate-man [...other options] && \
        make && make install


Translator's workflow:

 a. initial translation:

   0. arrange lang dir, that is, cp man/ja to man/YOUR_LANG/ and edit
      Makefile.am; s/ja/YOUR_LANG/.
   1. create POT: make -C man/YOUR_LANG/ create-pot
   2. copy POT to PO and translate it
   3. run make

 b. maintanance:

   1. update PO: make -C man/YOUR_LANG update-po
   2. update translations
   3. just run make


Requirements for man regeneration / translation:
 
   xmllint, xsltproc, xml2po in gnome-doc-utils, DocBook DTD and DocBook
   xslt stylesheet for man pages are required.

   If xmlcatalog (e.g. /etc/xml/catalog) is configured appropriately,
   both xmllint and xsltproc will use local DTD and stylsheet files
   instead of these from remote site
   (http://www.oasis-open.org/docbook/...).


- satoru


From satoru.satoh at gmail.com  Wed Aug 20 19:25:24 2008
From: satoru.satoh at gmail.com (Satoru SATOH)
Date: Thu, 21 Aug 2008 02:25:24 +0900
Subject: [rsyslog] [PATCH] man-i18n patches summary
In-Reply-To: <20080820170735.GB20932@localhost.localdomain>
References: <20080820170735.GB20932@localhost.localdomain>
Message-ID: <20080820172520.GC20932@localhost.localdomain>

Hello,


On Thu, Aug 21, 2008 at 02:07:40AM +0900, Satoru SATOH wrote:
> Could you please take some time to review these and merge them?

Of course, my patches change many and don't think easy to merge nor am
not sure accetable enhancement in the first place.

Please let me know if you have any issues on them, I'll work to fix
them as far as I can.

- satoru


From rgerhards at hq.adiscon.com  Wed Aug 20 21:25:42 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Aug 2008 21:25:42 +0200
Subject: [rsyslog] [PATCH 00/07] man-i18n patches
In-Reply-To: <20080820153118.GC20566@localhost.localdomain>
References: <20080820153118.GC20566@localhost.localdomain>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F018@grfint2.intern.adiscon.com>

Hi Satoru-san,

Your contribution is greatly appreciated. But I am currently on summer
vacation end will return end of month. Then I'll have a look at the
patch :)

Thanks,
Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Satoru SATOH
> Sent: Wednesday, August 20, 2008 5:31 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] [PATCH 00/07] man-i18n patches
> 
> Hello / Guten tag,
> 
> 
> Rio-san (rfujita at redhat.com) previously told about his idea of
> documentation i18n / l10n for rsyslog. I'm one of his colleagues and
> originally told him about related topics.
> 
> 
> A series of patches I'll post right after this mail are to implement
> this i18n / l10n workflow for man pages into normal build process.
> 
> These patches enable to 
> 
>  * generate original English man pages from DocBook XML files
>  * generate translated non-English man pages from translated DocBook
>    XML files, which can be generated from PO files (PO is 
> much suitable
>    format for translation maintainace than XML, I guess)
>  * create POT (PO template)
> 
> And this automated man-pages-generation process is only performed when
> the configure option "--enable-regenerate-man" is specified. So, side
> effects of these patches should be kept to a minimum, I think.
> 
> - satoru
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> 


From jeffschroed at gmail.com  Thu Aug 21 19:33:51 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Thu, 21 Aug 2008 10:33:51 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
Message-ID: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>

Rsyslog seems to be mangling messages sent from our in-house applications.
We are trying to get the same format as we did with syslog-ng previously and
not having a lot of luck.

The logs are in the format something like this:
TAG APPLICATION: MSG

In syslog-ng, it looks like this locally and when sent to a remote
syslog-ng server:
Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
ops-192.168.101.251:65261 offset changed 0.0004593 seconds!

rsyslog with the TraditionalFileFormat makes it look like this
locally, stripping the hostname:
Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
0.0004593 seconds!

and like this when sent to a remote syslog-ng server. Notice how the
ADMIN tag was removed:
Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
offset changed 0.0004593 seconds!

We have realtime logscraping software that looks at all message tags
and does custom reporting on them.
It keys off the TAG property that rsyslog is stripping off. What can
we do to enable this? We've been playing
with custom $templates in the rsyslog.conf to no real avail.

Thanks!

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From hks.private at gmail.com  Thu Aug 21 19:46:43 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 21 Aug 2008 13:46:43 -0400
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
Message-ID: <ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>

I'm not familiar with syslog-ng, but I suspect this is just a
difference in the formatting/interpretation of the raw packet. Try
capturing a message or two from syslog-ng with tcpdump and compare to
section 4 of RFC 3164.

You can see how rsyslog interprets it with the following template:
$template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
PROGRAM: %programname% MSG: %rawmsg%\n"

This should give you some idea of how to create a template that will
log in the format you need.

-HKS



On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> Rsyslog seems to be mangling messages sent from our in-house applications.
> We are trying to get the same format as we did with syslog-ng previously and
> not having a lot of luck.
>
> The logs are in the format something like this:
> TAG APPLICATION: MSG
>
> In syslog-ng, it looks like this locally and when sent to a remote
> syslog-ng server:
> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>
> rsyslog with the TraditionalFileFormat makes it look like this
> locally, stripping the hostname:
> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
> 0.0004593 seconds!
>
> and like this when sent to a remote syslog-ng server. Notice how the
> ADMIN tag was removed:
> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
> offset changed 0.0004593 seconds!
>
> We have realtime logscraping software that looks at all message tags
> and does custom reporting on them.
> It keys off the TAG property that rsyslog is stripping off. What can
> we do to enable this? We've been playing
> with custom $templates in the rsyslog.conf to no real avail.
>
> Thanks!
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>


From rgerhards at hq.adiscon.com  Thu Aug 21 19:53:39 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 21 Aug 2008 19:53:39 +0200
Subject: [rsyslog] Problems migrating from syslog-ng
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>

I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

Rainer


-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com on behalf of (private) HKS
Sent: Thu 8/21/2008 7:46 PM
To: rsyslog-users
Subject: Re: [rsyslog] Problems migrating from syslog-ng
 
I'm not familiar with syslog-ng, but I suspect this is just a
difference in the formatting/interpretation of the raw packet. Try
capturing a message or two from syslog-ng with tcpdump and compare to
section 4 of RFC 3164.

You can see how rsyslog interprets it with the following template:
$template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
PROGRAM: %programname% MSG: %rawmsg%\n"

This should give you some idea of how to create a template that will
log in the format you need.

-HKS



On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> Rsyslog seems to be mangling messages sent from our in-house applications.
> We are trying to get the same format as we did with syslog-ng previously and
> not having a lot of luck.
>
> The logs are in the format something like this:
> TAG APPLICATION: MSG
>
> In syslog-ng, it looks like this locally and when sent to a remote
> syslog-ng server:
> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>
> rsyslog with the TraditionalFileFormat makes it look like this
> locally, stripping the hostname:
> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
> 0.0004593 seconds!
>
> and like this when sent to a remote syslog-ng server. Notice how the
> ADMIN tag was removed:
> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
> offset changed 0.0004593 seconds!
>
> We have realtime logscraping software that looks at all message tags
> and does custom reporting on them.
> It keys off the TAG property that rsyslog is stripping off. What can
> we do to enable this? We've been playing
> with custom $templates in the rsyslog.conf to no real avail.
>
> Thanks!
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From jeffschroed at gmail.com  Thu Aug 21 22:44:16 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Thu, 21 Aug 2008 13:44:16 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
Message-ID: <c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>

On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
work with -c3 and I'd
rather not run in compatibility mode.

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com on behalf of (private) HKS
> Sent: Thu 8/21/2008 7:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems migrating from syslog-ng
>
> I'm not familiar with syslog-ng, but I suspect this is just a
> difference in the formatting/interpretation of the raw packet. Try
> capturing a message or two from syslog-ng with tcpdump and compare to
> section 4 of RFC 3164.
>
> You can see how rsyslog interprets it with the following template:
> $template test, "TIME: %timestamp% HOST: %hostname% TAG: %syslogtag%
> PROGRAM: %programname% MSG: %rawmsg%\n"
>
> This should give you some idea of how to create a template that will
> log in the format you need.
>
> -HKS
>
>
>
> On Thu, Aug 21, 2008 at 1:33 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
>> Rsyslog seems to be mangling messages sent from our in-house applications.
>> We are trying to get the same format as we did with syslog-ng previously and
>> not having a lot of luck.
>>
>> The logs are in the format something like this:
>> TAG APPLICATION: MSG
>>
>> In syslog-ng, it looks like this locally and when sent to a remote
>> syslog-ng server:
>> Aug 21 00:00:00 ops051.nyc03.int ADMIN JAVA-EVENT:
>> ops-192.168.101.251:65261 offset changed 0.0004593 seconds!
>>
>> rsyslog with the TraditionalFileFormat makes it look like this
>> locally, stripping the hostname:
>> Aug 21 00:00:00 JAVA-EVENT: ops-192.168.101.251:65261 offset changed
>> 0.0004593 seconds!
>>
>> and like this when sent to a remote syslog-ng server. Notice how the
>> ADMIN tag was removed:
>> Aug 21 00:00:00 ops051.nyc03.int JAVA-EVENT: ops-192.168.101.251:65261
>> offset changed 0.0004593 seconds!
>>
>> We have realtime logscraping software that looks at all message tags
>> and does custom reporting on them.
>> It keys off the TAG property that rsyslog is stripping off. What can
>> we do to enable this? We've been playing
>> with custom $templates in the rsyslog.conf to no real avail.
>>
>> Thanks!

-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From hks.private at gmail.com  Thu Aug 21 23:00:35 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 21 Aug 2008 17:00:35 -0400
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
	<c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
Message-ID: <ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>

On Thu, Aug 21, 2008 at 4:44 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
> On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.
>
> Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
> work with -c3 and I'd
> rather not run in compatibility mode.


I don't think so.

-HKS


From jeffschroed at gmail.com  Fri Aug 22 15:21:01 2008
From: jeffschroed at gmail.com (Jeff Schroeder)
Date: Fri, 22 Aug 2008 06:21:01 -0700
Subject: [rsyslog] Problems migrating from syslog-ng
In-Reply-To: <ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>
References: <c24352ca0808211033u3e1a810dsb5ad7f11c594bfe6@mail.gmail.com>
	<ac9ded160808211046h4eb276e2t9653c5e7169008f5@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA0108D9@grfint2.intern.adiscon.com>
	<c24352ca0808211344n221aff68ib6e143c287680b48@mail.gmail.com>
	<ac9ded160808211400w2da90ce3k17a307a214aa939f@mail.gmail.com>
Message-ID: <c24352ca0808220621s5419322fi3f0820e1dea79da9@mail.gmail.com>

On Thu, Aug 21, 2008 at 2:00 PM, (private) HKS <hks.private at gmail.com> wrote:
> On Thu, Aug 21, 2008 at 4:44 PM, Jeff Schroeder <jeffschroed at gmail.com> wrote:
>> On Thu, Aug 21, 2008 at 10:53 AM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>>> I have only been able to have a brief look, but it looks like the message is incorrectly formatted. rsyslog is smart enough to detect that the hostname is missing if the tag is followed by a character not valid in hostnames. But if the tag even looks like a hostname, it has no chance of detecting that it isn't one. As suggested, see RFC 3164 for what the format should look like. I think the -x option (or some other) enables to strip hostname detection, but I am not sure. You can "solve" this by misusing some fields. E.g. FROMHOST probably has what actually is the tag. HKS suggestion will help you find a suitable format.

You were right Rainer. It looks like the java code which injects the
message is sending malformed
syslog requests. syslog-ng still sends it through and does the correct
things. Is there a way to make
rsyslog a bit less strict about it? Running rsyslog with -c0 defeats
the purpose of using rsyslog.

Until our application has been fixed and rolled out accross our
clusters worldwide, we rolled back to syslog-ng.


>>
>> Is there an equivalent of "-x" with "-c 3" enabled? It doesn't seem to
>> work with -c3 and I'd
>> rather not run in compatibility mode.
>
>
> I don't think so.
>
> -HKS
>



-- 
Jeff Schroeder

Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com


From aoz.syn at gmail.com  Sat Aug 23 09:45:41 2008
From: aoz.syn at gmail.com (RB)
Date: Sat, 23 Aug 2008 01:45:41 -0600
Subject: [rsyslog] Running hashes
Message-ID: <4255c2570808230045kb625e11u795435a8a6c8d457@mail.gmail.com>

Acting on an old comment from a friend, I decided to look into what
it'd take to get rsyslog to perform running hashes of logs.
Conceptually, it's pretty simple - every Nth message inject one
message containing the hash of the previous N messages (including the
previous hash message).  It also gave me an excuse to start digging
into the rsyslog code.

At first I thought I could do it with a property replacer, but that
seems a wash since those are wholly message-based and don't [seem to]
give the opportunity to store information (even a running hash) of
prior messages.  A plugin was my next hope, but there doesn't seem to
be a good mechanism to pipeline those together - AFAICT they're
expected to be single ingress/egress points, with no interstitial
stages.  I see the code for loading other objects as Rainer mentioned
in April, but that seems more for central functionality than for
chaining modules together.

This all brings me back to one of my original questions for rsyslog -
is module chaining something that is even on your radar?  I'm thinking
normalization, hashing, encryption, etc.  Almost feels like there
should be another layer here, maybe a "mangle" plugin interface that
could stack in after im* and before om*?


RB


From aoz.syn at gmail.com  Sun Aug 24 08:01:38 2008
From: aoz.syn at gmail.com (RB)
Date: Sun, 24 Aug 2008 00:01:38 -0600
Subject: [rsyslog] broken size_nbr handling
Message-ID: <4255c2570808232301n5bab0c30w549df07c8e5b6c1f@mail.gmail.com>

In implementing my own 'human-readable' size parser (would be nice to
have one as a part of one of the usable objects), I noticed there
seemed to be order-of-magnitude issues on the base-10 size definitions
in cfsysline.c.  Although the base-2 operations properly increase 2^10
(1024) for each SI prefix, the base-10 ones only increase by one order
of magnitude (10) instead of the proper 10^3 per prefix, ending up
with EB only being 10^8 instead of 10^18.

Following is a simple patch that should put things right:
======================================
diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c
index 18643ba..1989247 100644
--- a/runtime/cfsysline.c
+++ b/runtime/cfsysline.c
@@ -215,11 +215,11 @@ static rsRetVal doGetSize(uchar **pp, rsRetVal
(*pSetHdlr)(void*, uid_t), void *
                case 'e': i *= (int64) 1024 * 1024 * 1024 * 1024 *
1024 * 1024; ++(*pp); break; /* exa */
                /* and now the "new" 1000-based definitions */
                case 'K': i *= 1000; ++(*pp); break;
-               case 'M': i *= 10000; ++(*pp); break;
-               case 'G': i *= 100000; ++(*pp); break;
-               case 'T': i *= 1000000; ++(*pp); break; /* tera */
-               case 'P': i *= 10000000; ++(*pp); break; /* peta */
-               case 'E': i *= 100000000; ++(*pp); break; /* exa */
+               case 'M': i *= 1000000; ++(*pp); break;
+               case 'G': i *= 1000000000; ++(*pp); break;
+               case 'T': i *= 1000000000000; ++(*pp); break; /* tera */
+               case 'P': i *= 1000000000000000; ++(*pp); break; /* peta */
+               case 'E': i *= 1000000000000000000; ++(*pp); break; /* exa */
        }

        /* done */
======================================


From friedl at hq.adiscon.com  Mon Aug 25 17:47:31 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 25 Aug 2008 17:47:31 +0200
Subject: [rsyslog] rsyslog 3.19.11 released (beta)
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F039@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 3.19.11, a member of the beta branch, today.
This is a refresh of the current beta, bringing it up to date. It
includes a number of bugfixes from the v3-stable branch. It is a
recommended update for all beta branch users.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-129.phtml

Changelog:

http://www.rsyslog.com/Article273.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .




From mattjhell at gmail.com  Tue Aug 26 00:07:37 2008
From: mattjhell at gmail.com (Matt Hellman)
Date: Mon, 25 Aug 2008 17:07:37 -0500
Subject: [rsyslog] gnutls support in 3.18.3?
Message-ID: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>

I don't see the --enable-gnutls option in 3.18.3.  Is this only in dev
branch currently?


From mbiebl at gmail.com  Tue Aug 26 00:39:07 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 26 Aug 2008 00:39:07 +0200
Subject: [rsyslog] gnutls support in 3.18.3?
In-Reply-To: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>
References: <a92cc2440808251507p5fb8aa18g2a593c6cb9145dc5@mail.gmail.com>
Message-ID: <aa76cdac0808251539o5f5508e9qf45c13af2edfcf83@mail.gmail.com>

2008/8/26 Matt Hellman <mattjhell at gmail.com>:
> I don't see the --enable-gnutls option in 3.18.3.  Is this only in dev
> branch currently?

That is correct.
The TLS support is not available in the v3-stable branch.
You should use the beta branch (latest released version is 3.19.11) or
the master aka devel branch (latest released version is 3.21.2)

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From danson at rackspace.com  Wed Aug 27 23:18:40 2008
From: danson at rackspace.com (Daniel Anson)
Date: Wed, 27 Aug 2008 16:18:40 -0500
Subject: [rsyslog] test
Message-ID: <19205_1219871984_m7RLJaDk013323_96AF20FDF4301D419B33CCE8E3A0132B08A296C2@SAT4MX07.RACKSPACE.CORP>

Test.  New to group.

Daniel M. Anson
Linux Systems Engineer
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From rory at ooma.com  Thu Aug 28 00:20:16 2008
From: rory at ooma.com (Rory Toma)
Date: Wed, 27 Aug 2008 15:20:16 -0700
Subject: [rsyslog] phpLogCon bug?
Message-ID: <48B5D320.80405@ooma.com>

It seems that if I execute a query, and then close the browser window 
(safari and firefox), the query remains running on the sql server and 
only goes away after it times out, degrading performance.

I'm using 2.5.3 of phpLogCon


From alorbach at ro1.adiscon.com  Thu Aug 28 09:50:39 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Thu, 28 Aug 2008 09:50:39 +0200
Subject: [rsyslog] phpLogCon bug?
In-Reply-To: <48B5D320.80405@ooma.com>
References: <48B5D320.80405@ooma.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449D9D@rmail.adiscon.com>

Hi,

I think this behavior comes from the design of php. Closing the Browser
Window tells the webserver to quit the current connection, but if the
php processor hangs within a working sql query, he has to wait until
this query is done. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 28, 2008 12:20 AM
> To: rsyslog-users
> Subject: [rsyslog] phpLogCon bug?
> 
> It seems that if I execute a query, and then close the browser window
> (safari and firefox), the query remains running on the sql server and
> only goes away after it times out, degrading performance.
> 
> I'm using 2.5.3 of phpLogCon
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog


From rgerhards at hq.adiscon.com  Thu Aug 28 10:20:46 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Aug 2008 10:20:46 +0200
Subject: [rsyslog] phpLogCon bug?
Message-ID: <000c01c908e7$0d9b6f72$060013ac@intern.adiscon.com>

I agree with andre, thats http behaviour. The srver doesnt even know the client is dead until it tries to send data (which it can only after completing the query).
Hth
rainer

----- Urspr?ngliche Nachricht -----
Von: "Andre Lorbach" <alorbach at ro1.adiscon.com>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 28.08.08 09:50
Betreff: Re: [rsyslog] phpLogCon bug?

Hi,

I think this behavior comes from the design of php. Closing the Browser
Window tells the webserver to quit the current connection, but if the
php processor hangs within a working sql query, he has to wait until
this query is done. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, August 28, 2008 12:20 AM
> To: rsyslog-users
> Subject: [rsyslog] phpLogCon bug?
> 
> It seems that if I execute a query, and then close the browser window
> (safari and firefox), the query remains running on the sql server and
> only goes away after it times out, degrading performance.
> 
> I'm using 2.5.3 of phpLogCon
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog


From danson at rackspace.com  Thu Aug 28 16:54:59 2008
From: danson at rackspace.com (Daniel Anson)
Date: Thu, 28 Aug 2008 09:54:59 -0500
Subject: [rsyslog] Memory vanishes
Message-ID: <28252_1219935370_m7SEu5H1031505_96AF20FDF4301D419B33CCE8E3A0132B08A29886@SAT4MX07.RACKSPACE.CORP>

For a bit of background, I have a RHEL5 server acting as an aggregate
point to write rsyslog/syslog info to:
rsyslog.i386                             2.0.0-11.el5
installed       
rsyslog-mysql.i386                       2.0.0-11.el5
installed

I am manipulating this data received with a threaded C program.  So, in
the spirit of testing, I wrote this small code to produce logs so I can
test how many messages that can be processed.  Here is the code for that
program:

#include <syslog.h>

int
main(int argc, char **argv) {
  int num_syslogs = atoi(argv[1]), i;
  openlog("syslog_writer", LOG_CONS | LOG_PID, LOG_USER);
  for(i=0; i < num_syslogs; i++) {
    syslog(LOG_INFO, "syslog_writer: log number %d", i);
  }
  closelog();
  return 1;
}

I run it thusly:
./syslogwriter 300000

After the running of this program, I find that ~30MB of memory just
disappear and never reappear.  I understand that MySQL and other
programs can hold some memory but do end up releasing it.  This is not
the case, this memory is not freed.  I know that I am not using the most
current version but I am using a STABLE release.  Has anyone seen this
behavior before or have an idea why this is happening.  I am at a loss.
Have I merely reached a limitation point in the number of consecutive
logs that can be handled?

Thx,

Daniel M. Anson
Linux Systems Engineer III
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From rgerhards at hq.adiscon.com  Thu Aug 28 21:47:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Aug 2008 21:47:56 +0200
Subject: [rsyslog] Memory vanishes
Message-ID: <000e01c90947$0ce8c017$060013ac@intern.adiscon.com>

2.0.0 is quite old. I think there are some mem leak fixes in the more recent v2-stable builds (see changelog). I suggest updating (from source) to the current v2-stable (2.0.6 or so, not sure out of my head).

Rainer

----- Urspr?ngliche Nachricht -----
Von: "Daniel Anson" <danson at rackspace.com>
An: "rsyslog at lists.adiscon.com" <rsyslog at lists.adiscon.com>
Gesendet: 28.08.08 16:56
Betreff: [rsyslog] Memory vanishes

For a bit of background, I have a RHEL5 server acting as an aggregate
point to write rsyslog/syslog info to:
rsyslog.i386                             2.0.0-11.el5
installed       
rsyslog-mysql.i386                       2.0.0-11.el5
installed

I am manipulating this data received with a threaded C program.  So, in
the spirit of testing, I wrote this small code to produce logs so I can
test how many messages that can be processed.  Here is the code for that
program:

#include <syslog.h>

int
main(int argc, char **argv) {
  int num_syslogs = atoi(argv[1]), i;
  openlog("syslog_writer", LOG_CONS | LOG_PID, LOG_USER);
  for(i=0; i < num_syslogs; i++) {
    syslog(LOG_INFO, "syslog_writer: log number %d", i);
  }
  closelog();
  return 1;
}

I run it thusly:
./syslogwriter 300000

After the running of this program, I find that ~30MB of memory just
disappear and never reappear.  I understand that MySQL and other
programs can hold some memory but do end up releasing it.  This is not
the case, this memory is not freed.  I know that I am not using the most
current version but I am using a STABLE release.  Has anyone seen this
behavior before or have an idea why this is happening.  I am at a loss.
Have I merely reached a limitation point in the number of consecutive
logs that can be handled?

Thx,

Daniel M. Anson
Linux Systems Engineer III
Rackspace
danson at rackspace.com
Office: (210)312-5114




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.

_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog