From corey at sequestered.net  Fri Jul  1 01:37:31 2011
From: corey at sequestered.net (Corey Quinn)
Date: Thu, 30 Jun 2011 16:37:31 -0700
Subject: [rsyslog] Active/Active Config
Message-ID: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>

I have two rsyslog servers that I'm trying to set up in an active/active configuration.  I want log1 to log all traffic that it receives, and then send it to og2, avie versa, but I don't want them looping the same message 5000 times each.  What's the "best" way to do this?

-- Corey

From taotetek at gmail.com  Fri Jul  1 02:15:05 2011
From: taotetek at gmail.com (Brian Knox)
Date: Thu, 30 Jun 2011 20:15:05 -0400
Subject: [rsyslog] Active/Active Config
In-Reply-To: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>
References: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>
Message-ID: <BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>

I do this using a forwarding rule that basically says "send everything to
the other server that did not originate from the other server", using a
!isequal -

:fromhost-ip, !isequal, "[other host ip] " @@[other host ip]


On Thu, Jun 30, 2011 at 7:37 PM, Corey Quinn <corey at sequestered.net> wrote:

> I have two rsyslog servers that I'm trying to set up in an active/active
> configuration.  I want log1 to log all traffic that it receives, and then
> send it to og2, avie versa, but I don't want them looping the same message
> 5000 times each.  What's the "best" way to do this?
>
> -- Corey
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From corey at sequestered.net  Fri Jul  1 07:49:14 2011
From: corey at sequestered.net (Corey Quinn)
Date: Thu, 30 Jun 2011 22:49:14 -0700
Subject: [rsyslog] Active/Active Config
In-Reply-To: <BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>
References: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>
	<BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>
Message-ID: <F65B6C76-08BD-453A-967C-A58A772C61FB@sequestered.net>


On Jun 30, 2011, at 5:15 PM, Brian Knox wrote:

> I do this using a forwarding rule that basically says "send everything to
> the other server that did not originate from the other server", using a
> !isequal -
> 
> :fromhost-ip, !isequal, "[other host ip] " @@[other host ip]

You're a gentleman and a scholar; thank you.  This worked flawlessly.

-- Corey / KB1JWQ

From conloos at googlemail.com  Fri Jul  1 10:52:20 2011
From: conloos at googlemail.com (Frank Dornheim)
Date: Fri, 1 Jul 2011 10:52:20 +0200
Subject: [rsyslog] rsyslog and GSSAPI
Message-ID: <BANLkTi=+Ta0QCKKy+pCwQeyPGPdCS8S3pw@mail.gmail.com>

Hi,

i made a post in the forum, but i got no awnser, so i try the list.

I worked through a lot of post, e.g.:

* http://blog.gmane.org/gmane.comp.sysutils.rsyslog/month=20071101
* http://www.rsyslog.com/doc/gssapi.html
* http://www.rsyslog.com/doc/imgssapi.html

and it seams to me that the cliendside didn't use a keytab.
A user have to get the ticket every time?

Is that right?
Have anybody a running config?


Thanks Con

From rgerhards at hq.adiscon.com  Fri Jul  1 11:23:19 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 1 Jul 2011 11:23:19 +0200
Subject: [rsyslog] rsyslog and GSSAPI
In-Reply-To: <BANLkTi=+Ta0QCKKy+pCwQeyPGPdCS8S3pw@mail.gmail.com>
References: <BANLkTi=+Ta0QCKKy+pCwQeyPGPdCS8S3pw@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F51@GRFEXC.intern.adiscon.com>

Just so that you get an answer from me: the whole GSSAPI is contributed (by
Red Hat), I have no idea about it ;)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Frank Dornheim
> Sent: Friday, July 01, 2011 10:52 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] rsyslog and GSSAPI
> 
> Hi,
> 
> i made a post in the forum, but i got no awnser, so i try the list.
> 
> I worked through a lot of post, e.g.:
> 
> * http://blog.gmane.org/gmane.comp.sysutils.rsyslog/month=20071101
> * http://www.rsyslog.com/doc/gssapi.html
> * http://www.rsyslog.com/doc/imgssapi.html
> 
> and it seams to me that the cliendside didn't use a keytab.
> A user have to get the ticket every time?
> 
> Is that right?
> Have anybody a running config?
> 
> 
> Thanks Con
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From taotetek at gmail.com  Fri Jul  1 13:00:12 2011
From: taotetek at gmail.com (Brian Knox)
Date: Fri, 1 Jul 2011 07:00:12 -0400
Subject: [rsyslog] Active/Active Config
In-Reply-To: <F65B6C76-08BD-453A-967C-A58A772C61FB@sequestered.net>
References: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>
	<BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>
	<F65B6C76-08BD-453A-967C-A58A772C61FB@sequestered.net>
Message-ID: <BANLkTik0KnSbU1r=3OqSE=AEdFKwbNP7jw@mail.gmail.com>

You're welcome!

Brian

On Fri, Jul 1, 2011 at 1:49 AM, Corey Quinn <corey at sequestered.net> wrote:

>
> On Jun 30, 2011, at 5:15 PM, Brian Knox wrote:
>
> > I do this using a forwarding rule that basically says "send everything to
> > the other server that did not originate from the other server", using a
> > !isequal -
> >
> > :fromhost-ip, !isequal, "[other host ip] " @@[other host ip]
>
> You're a gentleman and a scholar; thank you.  This worked flawlessly.
>
> -- Corey / KB1JWQ
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From corey at sequestered.net  Fri Jul  1 20:11:02 2011
From: corey at sequestered.net (Corey Quinn)
Date: Fri, 1 Jul 2011 11:11:02 -0700
Subject: [rsyslog] Active/Active Config
In-Reply-To: <BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>
References: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>
	<BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>
Message-ID: <E2BD3ED0-7318-4FCB-92C2-346815881998@sequestered.net>


On Jun 30, 2011, at 5:15 PM, Brian Knox wrote:

> I do this using a forwarding rule that basically says "send everything to
> the other server that did not originate from the other server", using a
> !isequal -
> 
> :fromhost-ip, !isequal, "[other host ip] " @@[other host ip]
> 

Hmm, looking at this in daylight, it seems that log1 receives a message, sends it to log2.  log2 then logs it, sends the message BACK to log1, where it's dropped.  Is there a way to avoid that return hop before being dropped?

-- Corey / KB1JWQ

From rgerhards at hq.adiscon.com  Sat Jul  2 13:05:45 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 2 Jul 2011 13:05:45 +0200
Subject: [rsyslog] imudp activation failure - was: Remote syslogging through
	a (broken) VPN
In-Reply-To: <4E0C895A.7010809@mejor.pl>
References: <20110627205918.0cc9ce80@mistral.stie>	<4E09E1B5.7040704@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7280F1D@GRFEXC.intern.adiscon.com>	<4E0C6228.4000203@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7280F47@GRFEXC.intern.adiscon.com>
	<4E0C895A.7010809@mejor.pl>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F5C@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw
> Sent: Thursday, June 30, 2011 4:34 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Remote syslogging through a (broken) VPN

Thanks for the debug log, I just reviewed it.

> P.S. I've got messages:
> rsyslogd3: activation of module imudp.so failed [try
> http://www.rsyslog.com/e/-3 ]
> rsyslogd3: activation of module imudp.so failed [try
> http://www.rsyslog.com/e/-3 ]

Did you run rsyslogd as root? As far as I can see, no listeners could be
activated. Obviously, error reporting needs to be improved, will check that
out.

> rsyslogd-2040: fatal error on disk queue 'action 1 queue[DA]',
> emergency
> switch to direct mode [try http://www.rsyslog.com/e/2
> 040 ]

As I guessed, the queue files are not OK. The .qi file requests buffer file
1251, which seems not to be present (or not accessible to the account you run
rsyslog under). This can lead to the rest of the problem you see.

Rainer
> 
> probably i missed changes of configuration format...
> 
> Regards,
> Marcin
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Sat Jul  2 13:09:56 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 2 Jul 2011 13:09:56 +0200
Subject: [rsyslog] imudp activation failure - was: Remote syslogging
	througha (broken) VPN
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280F5C@GRFEXC.intern.adiscon.com>
References: <20110627205918.0cc9ce80@mistral.stie>	<4E09E1B5.7040704@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7280F1D@GRFEXC.intern.adiscon.com>	<4E0C6228.4000203@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7280F47@GRFEXC.intern.adiscon.com><4E0C895A.7010809@mejor.pl>
	<9B6E2A8877C38245BFB15CC491A11DA7280F5C@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F5D@GRFEXC.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Saturday, July 02, 2011 1:06 PM
> To: rsyslog-users
> Subject: [rsyslog] imudp activation failure - was: Remote syslogging
> througha (broken) VPN


> > P.S. I've got messages:
> > rsyslogd3: activation of module imudp.so failed [try
> > http://www.rsyslog.com/e/-3 ]
> > rsyslogd3: activation of module imudp.so failed [try
> > http://www.rsyslog.com/e/-3 ]
> 
> Did you run rsyslogd as root? As far as I can see, no listeners could
> be
> activated. Obviously, error reporting needs to be improved, will check
> that
> out.

Sorry, I should have checked that first. The debug log indicates that the
error message

"imudp: no listeners could be started, input not activated"

was generated right in front of the message you quote. Wasn't that present
inside your log?

Rainer

From rgerhards at hq.adiscon.com  Sat Jul  2 13:17:07 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 2 Jul 2011 13:17:07 +0200
Subject: [rsyslog] imfile feature request
In-Reply-To: <alpine.DEB.2.02.1106291344380.24638@asgard.lang.hm>
References: <4E0AC2C3.8010205@cs.bgu.ac.il><9B6E2A8877C38245BFB15CC491A11DA7280F28@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1106291344380.24638@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F5E@GRFEXC.intern.adiscon.com>

David,

that's an interesting idea and I agree that it would be useful in a couple of
cases (maybe the majority). I am hesitant do touch the code though for two
reasons:

a) this doesn't seem to be ultra-critical and as a refactoring is going on,
I'd really like to save some time and wait for that to complete

b) using that feature would more than clumsy with the current config format.
Let's finish something decent here, where we can build on and introduce the
feature after that (when hopefully a) has been finished as well).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, June 29, 2011 10:50 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] imfile feature request
> 
> Rainer,
>    one of the things that I think needs to be done with imfile is to
> have
> it run through the 'normal' parser process. If you do that, then
> $InputFileTag essentially becomes a custom parser module, but if it's
> not
> provided, you would fall through to the traditional parser, which would
> handle files containing syslog-type data (as opposed to the message-
> only
> data that you are expecting now)
> 
> I think this would probably simplify imfile as well.
> 
> this would also handle the issue of escaping control characters in the
> message as a side effect.
> 
> David Lang
> 
>   On Wed, 29 Jun 2011, Rainer Gerhards wrote:
> 
> > There currently is a refactor/rewrite of imfile underway. Please see
> here:
> >
> > http://kb.monitorware.com/imfile-refactor-t10622.html
> >
> > I am hesitant to add non-trivial things (like this is) until this is
> > finished. I suggest to add that feature request to the thread (note
> that I am
> > NOT the author of the refactoring, so I don't know if the author has
> seen
> > your list posting).
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Piavlo
> >> Sent: Wednesday, June 29, 2011 8:14 AM
> >> To: rsyslog at lists.adiscon.com
> >> Subject: [rsyslog] imfile feature request
> >>
> >>   Hi,
> >>
> >> Currently the $InputFileTag option is mandatory.
> >> I have a case there the messages in $InputFileName already have the
> >> ident field (programname) as the first field in the messages - I'd
> like
> >> imfile to get the messages as-is without adding the $InputFileTag to
> >> the
> >> messages - so that I could use the original first field as
> programname
> >> to automatically split the messages to files  on remote system.
> >>
> >> Is there chance such feature can be implemented.
> >>
> >> Otherwise I need to do some ugly things with property replacer which
> on
> >> the my first trials does seems to have enough features to do that I
> >> need.
> >>
> >> Thanks
> >> Alex
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From marcin at mejor.pl  Sat Jul  2 18:52:25 2011
From: marcin at mejor.pl (=?ISO-8859-2?Q?Marcin_Miros=B3aw?=)
Date: Sat, 02 Jul 2011 18:52:25 +0200
Subject: [rsyslog] imudp activation failure - was: Remote syslogging
 through a (broken) VPN
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280F5C@GRFEXC.intern.adiscon.com>
References: <20110627205918.0cc9ce80@mistral.stie>	<4E09E1B5.7040704@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7280F1D@GRFEXC.intern.adiscon.com>	<4E0C6228.4000203@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7280F47@GRFEXC.intern.adiscon.com>
	<4E0C895A.7010809@mejor.pl>
	<9B6E2A8877C38245BFB15CC491A11DA7280F5C@GRFEXC.intern.adiscon.com>
Message-ID: <4E0F4CC9.8070602@mejor.pl>

W dniu 2011-07-02 13:05, Rainer Gerhards pisze:
> Did you run rsyslogd as root? As far as I can see, no listeners could be
> activated. Obviously, error reporting needs to be improved, will check that
> out.

Yes, i run rsyslogd as root. It was started by the same init script as
earlier version. I don't know is it important, i'm using hardened kernel
(grsecurity patch).

>> rsyslogd-2040: fatal error on disk queue 'action 1 queue[DA]',
>> emergency
>> switch to direct mode [try http://www.rsyslog.com/e/2
>> 040 ]
> 
> As I guessed, the queue files are not OK. The .qi file requests buffer file
> 1251, which seems not to be present (or not accessible to the account you run
> rsyslog under). This can lead to the rest of the problem you see.

I've often such problem. When OS crashed or i have to stop rsyslogd
using SIGKILL instead SIGTERM. But with version 5.6.5 it has only one
side effect, rsyglogd didn't send queue to another rsyslogd. V5.6.5
quits cleanly after SIGTERM.

[from later email]
> Sorry, I should have checked that first. The debug log indicates that
> the
> error message
> "imudp: no listeners could be started, input not activated"
> was generated right in front of the message you quote. Wasn't that
> present
> inside your log?

Indeed, it was present. (don't trust users, they always skip something
from log...).

This should be all messages generated by git-version:

2011-06-30T15:26:12.613744+02:00 hermes rsyslogd: [origin
software="rsyslogd" swVersion="6.3.1" x-pid="29536"
x-info="http://www.rsyslog.com"] start
2011-06-30T15:26:12.578971+02:00 hermes rsyslogd: imklog 6.3.1, log
source = /proc/kmsg started.
2011-06-30T15:26:12.579117+02:00 hermes rsyslogd: imudp: no listeners
could be started, input not activated.
: No such file or directory
2011-06-30T15:26:12.579124+02:00 hermes rsyslogd3: activation of module
imudp.so failed [try http://www.rsyslog.com/e/-3 ]
2011-06-30T15:26:12.579385+02:00 hermes rsyslogd3: activation of module
imtcp.so failed [try http://www.rsyslog.com/e/-3 ]
2011-06-30T15:26:12.615118+02:00 hermes rsyslogd-2040: fatal error on
disk queue 'action 1 queue[DA]', emergency switch to direct mode [try
http://www.rsyslog
.com/e/2040 ]

Thanks for help.

From josu.lazkano at barcelonamedia.org  Mon Jul  4 12:54:47 2011
From: josu.lazkano at barcelonamedia.org (Josu Lazkano)
Date: Mon, 4 Jul 2011 12:54:47 +0200
Subject: [rsyslog] delete local logs
Message-ID: <A55193C141A0254B8D28E8E7C9D3952C26263769CE@FBMEC01.corp.barcelonamedia.org>

Hello list, I am trying to configure the log system on my LAN. I want to send my logs to a remote server this way:

auth.*,authpriv.*       @logserver

It sends well all auth logs, but I want to just send and not log on /var/log/auth.log locally

Is this possible?

Thanks for all your help.

Best regards.

From rgerhards at hq.adiscon.com  Mon Jul  4 12:59:44 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 4 Jul 2011 12:59:44 +0200
Subject: [rsyslog] delete local logs
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F64@GRFEXC.intern.adiscon.com>

auth.*,authpriv.*       @logserver
& ~

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Josu Lazkano
> Sent: Monday, July 04, 2011 12:55 PM
> To: rsyslog-users
> Subject: [rsyslog] delete local logs
> 
> Hello list, I am trying to configure the log system on my LAN. I want to
send
> my logs to a remote server this way:
> 
> auth.*,authpriv.*       @logserver
> 
> It sends well all auth logs, but I want to just send and not log on
> /var/log/auth.log locally
> 
> Is this possible?
> 
> Thanks for all your help.
> 
> Best regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From josu.lazkano at barcelonamedia.org  Mon Jul  4 13:18:44 2011
From: josu.lazkano at barcelonamedia.org (Josu Lazkano)
Date: Mon, 4 Jul 2011 13:18:44 +0200
Subject: [rsyslog] delete local logs
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280F64@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280F64@GRFEXC.intern.adiscon.com>
Message-ID: <A55193C141A0254B8D28E8E7C9D3952C26263769CF@FBMEC01.corp.barcelonamedia.org>

Thanks Rainer, I have another question: if the server is not accessible (shutdown, network problem...), what happens with the logs?

It will be lost?

Thanks again and kind regards.


-----Mensaje original-----
De: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] En nombre de Rainer Gerhards
Enviado el: lunes, 04 de julio de 2011 13:00
Para: rsyslog-users
Asunto: Re: [rsyslog] delete local logs

auth.*,authpriv.*       @logserver
& ~

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Josu Lazkano
> Sent: Monday, July 04, 2011 12:55 PM
> To: rsyslog-users
> Subject: [rsyslog] delete local logs
> 
> Hello list, I am trying to configure the log system on my LAN. I want to
send
> my logs to a remote server this way:
> 
> auth.*,authpriv.*       @logserver
> 
> It sends well all auth logs, but I want to just send and not log on
> /var/log/auth.log locally
> 
> Is this possible?
> 
> Thanks for all your help.
> 
> Best regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From friedl at hq.adiscon.com  Mon Jul  4 14:29:04 2011
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 4 Jul 2011 14:29:04 +0200
Subject: [rsyslog] delete local logs
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F65@GRFEXC.intern.adiscon.com>

Hi,

Many thanks for contacting Adiscon support. We value your attention, but need
to tell you that Adiscon support for open source projects is available as a
paid option, only. This does not mean there is no free support: please visit
the project's homepage to find all support options, which usually consist of
the project mailing list and the public forum. Often, questions are answered
there very quickly as well.

If you need private support, or some support guarantee (e.g. as a corporate
requirement), we invite you to explore our low-priced professional support
options available at

http://www.rsyslog.com/professional-services/

Best regards,

The Adiscon Team

-----Urspr?ngliche Nachricht-----
Von: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Josu Lazkano
Gesendet: Montag, 4. Juli 2011 13:19
An: rsyslog-users
Betreff: Re: [rsyslog] delete local logs

Thanks Rainer, I have another question: if the server is not accessible
(shutdown, network problem...), what happens with the logs?

It will be lost?

Thanks again and kind regards.


-----Mensaje original-----
De: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] En nombre de Rainer Gerhards
Enviado el: lunes, 04 de julio de 2011 13:00
Para: rsyslog-users
Asunto: Re: [rsyslog] delete local logs

auth.*,authpriv.*       @logserver
& ~

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- 
> bounces at lists.adiscon.com] On Behalf Of Josu Lazkano
> Sent: Monday, July 04, 2011 12:55 PM
> To: rsyslog-users
> Subject: [rsyslog] delete local logs
> 
> Hello list, I am trying to configure the log system on my LAN. I want 
> to
send
> my logs to a remote server this way:
> 
> auth.*,authpriv.*       @logserver
> 
> It sends well all auth logs, but I want to just send and not log on 
> /var/log/auth.log locally
> 
> Is this possible?
> 
> Thanks for all your help.
> 
> Best regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From friedl at hq.adiscon.com  Mon Jul  4 14:30:53 2011
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 4 Jul 2011 14:30:53 +0200
Subject: [rsyslog] delete local logs
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F66@GRFEXC.intern.adiscon.com>

Sorry, this email should have gone elsewhere ;)

Correct link is coming in a second.

Florian

-----Urspr?ngliche Nachricht-----
Von: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Florian Riedl
Gesendet: Montag, 4. Juli 2011 14:29
An: rsyslog-users
Betreff: Re: [rsyslog] delete local logs

Hi,

Many thanks for contacting Adiscon support. We value your attention, but need
to tell you that Adiscon support for open source projects is available as a
paid option, only. This does not mean there is no free support: please visit
the project's homepage to find all support options, which usually consist of
the project mailing list and the public forum. Often, questions are answered
there very quickly as well.

If you need private support, or some support guarantee (e.g. as a corporate
requirement), we invite you to explore our low-priced professional support
options available at

http://www.rsyslog.com/professional-services/

Best regards,

The Adiscon Team

-----Urspr?ngliche Nachricht-----
Von: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Josu Lazkano
Gesendet: Montag, 4. Juli 2011 13:19
An: rsyslog-users
Betreff: Re: [rsyslog] delete local logs

Thanks Rainer, I have another question: if the server is not accessible
(shutdown, network problem...), what happens with the logs?

It will be lost?

Thanks again and kind regards.


-----Mensaje original-----
De: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] En nombre de Rainer Gerhards
Enviado el: lunes, 04 de julio de 2011 13:00
Para: rsyslog-users
Asunto: Re: [rsyslog] delete local logs

auth.*,authpriv.*       @logserver
& ~

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- 
> bounces at lists.adiscon.com] On Behalf Of Josu Lazkano
> Sent: Monday, July 04, 2011 12:55 PM
> To: rsyslog-users
> Subject: [rsyslog] delete local logs
> 
> Hello list, I am trying to configure the log system on my LAN. I want 
> to
send
> my logs to a remote server this way:
> 
> auth.*,authpriv.*       @logserver
> 
> It sends well all auth logs, but I want to just send and not log on 
> /var/log/auth.log locally
> 
> Is this possible?
> 
> Thanks for all your help.
> 
> Best regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From friedl at hq.adiscon.com  Mon Jul  4 14:39:33 2011
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Mon, 4 Jul 2011 14:39:33 +0200
Subject: [rsyslog] delete local logs
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F67@GRFEXC.intern.adiscon.com>

Hi,

so here comes the answer to your question.

Currently, you are using UDP transmission of messages. If your central server
is not available, the messages will be dropped and thus are lost. Easiest way
to resolve this would be to use TCP instead. Here is an article from the
rsyslog documentation about reliable forwarding with rsyslog. Using it to
make a reliable setup is of course suggested.

http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html

Best regards,

Florian

-----Urspr?ngliche Nachricht-----
Von: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Josu Lazkano
Gesendet: Montag, 4. Juli 2011 13:19
An: rsyslog-users
Betreff: Re: [rsyslog] delete local logs

Thanks Rainer, I have another question: if the server is not accessible
(shutdown, network problem...), what happens with the logs?

It will be lost?

Thanks again and kind regards.


-----Mensaje original-----
De: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] En nombre de Rainer Gerhards
Enviado el: lunes, 04 de julio de 2011 13:00
Para: rsyslog-users
Asunto: Re: [rsyslog] delete local logs

auth.*,authpriv.*       @logserver
& ~

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- 
> bounces at lists.adiscon.com] On Behalf Of Josu Lazkano
> Sent: Monday, July 04, 2011 12:55 PM
> To: rsyslog-users
> Subject: [rsyslog] delete local logs
> 
> Hello list, I am trying to configure the log system on my LAN. I want 
> to
send
> my logs to a remote server this way:
> 
> auth.*,authpriv.*       @logserver
> 
> It sends well all auth logs, but I want to just send and not log on 
> /var/log/auth.log locally
> 
> Is this possible?
> 
> Thanks for all your help.
> 
> Best regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Wed Jul  6 09:00:22 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 6 Jul 2011 09:00:22 +0200
Subject: [rsyslog] small but important change in config format
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F81@GRFEXC.intern.adiscon.com>

Hi all,

the omusrmsg action format is quite problematic and I cleaned this up today.
Please have a look at this blog post:

http://blog.gerhards.net/2011/07/why-omusrmsg-is-evil-and-how-it-is.html

I'll do the necessary releases soon and let you know when done.

Please note that the old-style format will still be supported, but removed
some time in the future (probably with v7).

Thanks,
Rainer

From tbergfeld at hq.adiscon.com  Wed Jul  6 10:01:24 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Wed, 6 Jul 2011 10:01:24 +0200
Subject: [rsyslog] rsyslog 6.3.2 (devel) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F84@GRFEXC.intern.adiscon.com>

This release offers better support for systemd and better timestamps for
local log messages as well as an important syntax enhancement for omusrmsg.
It also contains some bug fixes.

It is a recommended update for all v6-devel users.

ChangeLog:

http://www.rsyslog.com/changelog-for-6-3-2-v6-devel/

Download:

http://www.rsyslog.com/rsyslog-6-3-2-devel/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From josu.lazkano at barcelonamedia.org  Wed Jul  6 11:29:38 2011
From: josu.lazkano at barcelonamedia.org (Josu Lazkano)
Date: Wed, 6 Jul 2011 11:29:38 +0200
Subject: [rsyslog] delete local logs
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280F67@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280F67@GRFEXC.intern.adiscon.com>
Message-ID: <A55193C141A0254B8D28E8E7C9D3952C26263769EA@FBMEC01.corp.barcelonamedia.org>

Thanks for your help, it is working great.

Best regards.

-----Mensaje original-----
De: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] En nombre de Florian Riedl
Enviado el: lunes, 04 de julio de 2011 14:40
Para: rsyslog-users
Asunto: Re: [rsyslog] delete local logs

Hi,

so here comes the answer to your question.

Currently, you are using UDP transmission of messages. If your central server
is not available, the messages will be dropped and thus are lost. Easiest way
to resolve this would be to use TCP instead. Here is an article from the
rsyslog documentation about reliable forwarding with rsyslog. Using it to
make a reliable setup is of course suggested.

http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html

Best regards,

Florian

-----Urspr?ngliche Nachricht-----
Von: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Josu Lazkano
Gesendet: Montag, 4. Juli 2011 13:19
An: rsyslog-users
Betreff: Re: [rsyslog] delete local logs

Thanks Rainer, I have another question: if the server is not accessible
(shutdown, network problem...), what happens with the logs?

It will be lost?

Thanks again and kind regards.


-----Mensaje original-----
De: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] En nombre de Rainer Gerhards
Enviado el: lunes, 04 de julio de 2011 13:00
Para: rsyslog-users
Asunto: Re: [rsyslog] delete local logs

auth.*,authpriv.*       @logserver
& ~

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- 
> bounces at lists.adiscon.com] On Behalf Of Josu Lazkano
> Sent: Monday, July 04, 2011 12:55 PM
> To: rsyslog-users
> Subject: [rsyslog] delete local logs
> 
> Hello list, I am trying to configure the log system on my LAN. I want 
> to
send
> my logs to a remote server this way:
> 
> auth.*,authpriv.*       @logserver
> 
> It sends well all auth logs, but I want to just send and not log on 
> /var/log/auth.log locally
> 
> Is this possible?
> 
> Thanks for all your help.
> 
> Best regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From josu.lazkano at barcelonamedia.org  Wed Jul  6 11:30:05 2011
From: josu.lazkano at barcelonamedia.org (Josu Lazkano)
Date: Wed, 6 Jul 2011 11:30:05 +0200
Subject: [rsyslog] Send Apache vhost logs
Message-ID: <A55193C141A0254B8D28E8E7C9D3952C26263769EB@FBMEC01.corp.barcelonamedia.org>

Hello list, I just configured my log system. I have some servers with rsyslog as client and a log server with syslog-ng.

On the client-side I send all system logs this way:

$ cat /etc/rsyslog.conf
$ModLoad imuxsock
$ModLoad imklog
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$RepeatedMsgReduction on


$KLogPath /proc/kmsg
$WorkDirectory /rsyslog/work
$ActionQueueType LinkedList
$ActionQueueFileName srvrfwd1
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
*.*       @@logserver

With this configuration, I have this logs:

$ tree /var/log/extern/host1/
/var/log/extern/host1/
??? 2011-07
    ??? auth
    ??? authpriv
    ??? cron
    ??? daemon
    ??? syslog
    ??? user

I want to add Apache logs, this is how I configure the Apache logs:

ErrorLog /var/www/domain1/log/error.log
LogLevel warn
SetEnvIf Remote_Addr "x\.x\.x\.x" dontlog
SetEnvIf Remote_Addr "y\.y\.y\.y" dontlog
CustomLog /var/www/domain1/log/access.log common env=!dontlog

On the same machine I have lots of vhost logs:

/var/www/domain1/log/error.log
/var/www/domain1/log/access.log
/var/www/domain2/log/error.log
/var/www/domain2/log/access.log
/var/www/domain3/log/error.log
/var/www/domain3/log/access.log
...

Is possible to send all vhost logs to the log server?

Thanks for all your great help, kind regards.


From piavka at cs.bgu.ac.il  Wed Jul  6 21:50:47 2011
From: piavka at cs.bgu.ac.il (Piavlo)
Date: Wed, 06 Jul 2011 22:50:47 +0300
Subject: [rsyslog] howto get rid off unneeded buffer files under rsyslog
	workdir
Message-ID: <4E14BC97.3010603@cs.bgu.ac.il>


Hi,

For ActionQueueType rsyslog creates buffer logs then remote destination 
is unreachable.
Then remote is back again all buffered logs are senk ok to remote - but 
the latest log file always remain - even that log entries in it have 
been sent.

Is there safe way to get rid of these buffers automatically?
This is useful for monitoring - where I want to check that mp buffer 
files exist.

Thanks
Alex


From rgerhards at hq.adiscon.com  Thu Jul  7 07:12:38 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 7 Jul 2011 07:12:38 +0200
Subject: [rsyslog] howto get rid off unneeded buffer files under
	rsyslogworkdir
In-Reply-To: <4E14BC97.3010603@cs.bgu.ac.il>
References: <4E14BC97.3010603@cs.bgu.ac.il>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280F8C@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Piavlo
> Sent: Wednesday, July 06, 2011 9:51 PM
> To: rsyslog-users
> Subject: [rsyslog] howto get rid off unneeded buffer files under
> rsyslogworkdir
> 
> 
> Hi,
> 
> For ActionQueueType rsyslog creates buffer logs then remote destination
> is unreachable.
> Then remote is back again all buffered logs are senk ok to remote - but
> the latest log file always remain - even that log entries in it have
> been sent.
> 
> Is there safe way to get rid of these buffers automatically?

Unfortunately not. The reason is that once the DA worker has started, it will
never terminate. In v4, it terminated early if there were no work to do and
that resulted in quite complex logic and required a lot of performance. So we
keep them active. We anticipate if the DA queue has been used once, it is
highly likely it will be reused again.
Rainer
> This is useful for monitoring - where I want to check that mp buffer
> files exist.
> 
> Thanks
> Alex
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From corey at sequestered.net  Fri Jul  8 00:52:41 2011
From: corey at sequestered.net (Corey Quinn)
Date: Thu, 7 Jul 2011 15:52:41 -0700
Subject: [rsyslog] Active/Active Config
In-Reply-To: <E2BD3ED0-7318-4FCB-92C2-346815881998@sequestered.net>
References: <9E43B8AD-B02C-46C3-88D3-2089965F3835@sequestered.net>
	<BANLkTimdYJkkoG2DyQENOe-rJOn+P8zWNQ@mail.gmail.com>
	<E2BD3ED0-7318-4FCB-92C2-346815881998@sequestered.net>
Message-ID: <4EC77469-604C-4980-B58C-1A4152724CCB@sequestered.net>

> 
> On Jun 30, 2011, at 5:15 PM, Brian Knox wrote:
> 
>> I do this using a forwarding rule that basically says "send everything to
>> the other server that did not originate from the other server", using a
>> !isequal -
>> 
>> :fromhost-ip, !isequal, "[other host ip] " @@[other host ip]
>> 
> 


I've returned to pick at this problem a bit.  

The exact line I have in place is:

:fromhost-ip, !isequal, "10.1.1.1" :omrelp:log1.example.com:2514

The problem I'm seeing is that I wind up with massive log storms between the two boxes-- a message shows up from a random host (webserver.example.com), shows up in log1's log, gets thrown to log2 and is logged, then winds up starting this cycle over again.  

Does fromhost / fromhost-ip break with relp?

-- Corey / KB1JWQ

From jason at jasonantman.com  Fri Jul  8 19:14:33 2011
From: jason at jasonantman.com (Jason Antman)
Date: Fri, 08 Jul 2011 13:14:33 -0400
Subject: [rsyslog] Send log lines to persistent program
Message-ID: <4E173AF9.3080806@jasonantman.com>

Hello,

A colleague just finished migrating the last bits from our syslog-ng log 
centralized server to our new rsyslog server.

In one case (actually a relatively important one, which has turned into 
a big problem), we were using the syslog-ng program() log destination. 
The syslog-ng docs describe it thusly:

"This driver executes the specified program with the specified arguments 
and sends messages to the standard input (stdin) of the child. The 
program() driver has a single required parameter, specifying a program 
name to start. The program is executed with the help of the current 
shell, so the command may include both file patterns and I/O 
redirection, they will be processed. [...] Version 1.6 of syslog-ng 
executed the program once at startup, and kept it running until SIGHUP 
or exit. The reason was to prevent starting up a large number of 
programs for messages, which would have enabled an easy DoS attack. 
Versions 2.0 and later restart the program if it exits for reliability 
reasons. However it is not recommended to launch programs for single 
messages as that might easily cause a DoS for the system."

Attention is drawn to the last few sentences. syslog-ng passes all 
matching log lines to STDIN of the program, and *leaves* the program 
running indefinitely (well, until syslog-ng gets a HUP or exits). It 
also restarts the child if it has died.

$coworker translated this to a "^" action in rsyslog - the implications 
of this are pretty obvious, and even more troublesome if you know that 
the program being executed is a PHP script which does a whole bunch of 
string parsing and then sends the result to a MS SQL database.

So... I haven't found anything in the docs, but I just wanted to confirm 
that rsyslog doesn't have an action or output module that replicates 
this functionality. Assuming that is the case (no directly comparable 
feature), what are your suggestions for an interim fix (our MS SQL DBA 
is on vacation for a week, so I need to hold off on doing 
omlibdbi/Ms-Sql until he gets back)? Try to get the PHP script running 
as a daemon, log to named pipe, have script read named pipe?

Thanks for any advice/suggestions,
Jason

From rgerhards at hq.adiscon.com  Fri Jul  8 20:29:05 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 8 Jul 2011 20:29:05 +0200
Subject: [rsyslog] Send log lines to persistent program
Message-ID: <001b01cc3d9c$f2f4a1c2$100013ac@intern.adiscon.com>

omprog is what you need

>From phone
rainer

----- Urspr?ngliche Nachricht -----
Von: Jason Antman <jason at jasonantman.com>
Gesendet: Freitag, 8. Juli 2011 19:21
An: rsyslog-users <rsyslog at lists.adiscon.com>
Betreff: [rsyslog] Send log lines to persistent program

Hello,

A colleague just finished migrating the last bits from our syslog-ng log 
centralized server to our new rsyslog server.

In one case (actually a relatively important one, which has turned into 
a big problem), we were using the syslog-ng program() log destination. 
The syslog-ng docs describe it thusly:

"This driver executes the specified program with the specified arguments 
and sends messages to the standard input (stdin) of the child. The 
program() driver has a single required parameter, specifying a program 
name to start. The program is executed with the help of the current 
shell, so the command may include both file patterns and I/O 
redirection, they will be processed. [...] Version 1.6 of syslog-ng 
executed the program once at startup, and kept it running until SIGHUP 
or exit. The reason was to prevent starting up a large number of 
programs for messages, which would have enabled an easy DoS attack. 
Versions 2.0 and later restart the program if it exits for reliability 
reasons. However it is not recommended to launch programs for single 
messages as that might easily cause a DoS for the system."

Attention is drawn to the last few sentences. syslog-ng passes all 
matching log lines to STDIN of the program, and *leaves* the program 
running indefinitely (well, until syslog-ng gets a HUP or exits). It 
also restarts the child if it has died.

$coworker translated this to a "^" action in rsyslog - the implications 
of this are pretty obvious, and even more troublesome if you know that 
the program being executed is a PHP script which does a whole bunch of 
string parsing and then sends the result to a MS SQL database.

So... I haven't found anything in the docs, but I just wanted to confirm 
that rsyslog doesn't have an action or output module that replicates 
this functionality. Assuming that is the case (no directly comparable 
feature), what are your suggestions for an interim fix (our MS SQL DBA 
is on vacation for a week, so I need to hold off on doing 
omlibdbi/Ms-Sql until he gets back)? Try to get the PHP script running 
as a daemon, log to named pipe, have script read named pipe?

Thanks for any advice/suggestions,
Jason
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From jason at jasonantman.com  Fri Jul  8 21:17:42 2011
From: jason at jasonantman.com (Jason Antman)
Date: Fri, 08 Jul 2011 15:17:42 -0400
Subject: [rsyslog] Send log lines to persistent program
In-Reply-To: <001b01cc3d9c$f2f4a1c2$100013ac@intern.adiscon.com>
References: <001b01cc3d9c$f2f4a1c2$100013ac@intern.adiscon.com>
Message-ID: <4E1757D6.8040206@jasonantman.com>

Hey, thanks!

Rainer Gerhards wrote:
> omprog is what you need
>
> From phone
> rainer
>
> ----- Urspr?ngliche Nachricht -----
> Von: Jason Antman <jason at jasonantman.com>
> Gesendet: Freitag, 8. Juli 2011 19:21
> An: rsyslog-users <rsyslog at lists.adiscon.com>
> Betreff: [rsyslog] Send log lines to persistent program
>
> Hello,
>
> A colleague just finished migrating the last bits from our syslog-ng log 
> centralized server to our new rsyslog server.
>
> In one case (actually a relatively important one, which has turned into 
> a big problem), we were using the syslog-ng program() log destination. 
> The syslog-ng docs describe it thusly:
>
> "This driver executes the specified program with the specified arguments 
> and sends messages to the standard input (stdin) of the child. The 
> program() driver has a single required parameter, specifying a program 
> name to start. The program is executed with the help of the current 
> shell, so the command may include both file patterns and I/O 
> redirection, they will be processed. [...] Version 1.6 of syslog-ng 
> executed the program once at startup, and kept it running until SIGHUP 
> or exit. The reason was to prevent starting up a large number of 
> programs for messages, which would have enabled an easy DoS attack. 
> Versions 2.0 and later restart the program if it exits for reliability 
> reasons. However it is not recommended to launch programs for single 
> messages as that might easily cause a DoS for the system."
>
> Attention is drawn to the last few sentences. syslog-ng passes all 
> matching log lines to STDIN of the program, and *leaves* the program 
> running indefinitely (well, until syslog-ng gets a HUP or exits). It 
> also restarts the child if it has died.
>
> $coworker translated this to a "^" action in rsyslog - the implications 
> of this are pretty obvious, and even more troublesome if you know that 
> the program being executed is a PHP script which does a whole bunch of 
> string parsing and then sends the result to a MS SQL database.
>
> So... I haven't found anything in the docs, but I just wanted to confirm 
> that rsyslog doesn't have an action or output module that replicates 
> this functionality. Assuming that is the case (no directly comparable 
> feature), what are your suggestions for an interim fix (our MS SQL DBA 
> is on vacation for a week, so I need to hold off on doing 
> omlibdbi/Ms-Sql until he gets back)? Try to get the PHP script running 
> as a daemon, log to named pipe, have script read named pipe?
>
> Thanks for any advice/suggestions,
> Jason
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
>   


From silas.silva at ufabc.edu.br  Fri Jul  8 22:47:17 2011
From: silas.silva at ufabc.edu.br (Silas Silva)
Date: Fri, 8 Jul 2011 17:47:17 -0300
Subject: [rsyslog] Rsyslog logging sshd. Strange behaviour: duplicated sshd
	log.
Message-ID: <20110708204717.GH13176@auron.ufabc.int.br>

Hello all!

I'm using rsyslog in a cluster system to make a central log.  In a
machine, I have the following rsyslog.conf snippet:

    auth.*                          @@rsyslog-server
    authpriv.*                      @@rsyslog-server
    daemon.*                        @@rsyslog-server
    kern.*                          @@rsyslog-server
    security.*                      @@rsyslog-server
    user.*                          @@rsyslog-server

The problem is that, if I have security.* uncommented, sshd output
(connect, disconnect, etc.) is duplicated.  If I just let it commmented,
it logs just one line.  I'm pretty sure my /etc/ssh/sshd_config has
SyslogFacility AUTH and LogLevel INFO.

rsyslog-server also has the same configuration snippet, so if I let
security.* uncommented, it duplicates sshd messages, 2 (because they are
already duplicated) turns 4.

Any help on this?

Thank you very much.

-- 
Silas Silva

From david at lang.hm  Fri Jul  8 23:48:25 2011
From: david at lang.hm (david at lang.hm)
Date: Fri, 8 Jul 2011 14:48:25 -0700 (PDT)
Subject: [rsyslog] Rsyslog logging sshd. Strange behaviour: duplicated
 sshd	log.
In-Reply-To: <20110708204717.GH13176@auron.ufabc.int.br>
References: <20110708204717.GH13176@auron.ufabc.int.br>
Message-ID: <alpine.DEB.2.02.1107081441430.19490@asgard.lang.hm>

is there a reason not to just say

*.*  @@rsyslog-server

and forward all of your logs there?

I think the issue is that auth and security map to the same value (looking 
at the rsyslog documentation page, it identifies that auth and security do 
the same thing)

David Lang

On Fri, 8 Jul 2011, Silas Silva wrote:

> Date: Fri, 8 Jul 2011 17:47:17 -0300
> From: Silas Silva <silas.silva at ufabc.edu.br>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Rsyslog logging sshd. Strange behaviour: duplicated sshd
>     log.
> 
> Hello all!
>
> I'm using rsyslog in a cluster system to make a central log.  In a
> machine, I have the following rsyslog.conf snippet:
>
>    auth.*                          @@rsyslog-server
>    authpriv.*                      @@rsyslog-server
>    daemon.*                        @@rsyslog-server
>    kern.*                          @@rsyslog-server
>    security.*                      @@rsyslog-server
>    user.*                          @@rsyslog-server
>
> The problem is that, if I have security.* uncommented, sshd output
> (connect, disconnect, etc.) is duplicated.  If I just let it commmented,
> it logs just one line.  I'm pretty sure my /etc/ssh/sshd_config has
> SyslogFacility AUTH and LogLevel INFO.
>
> rsyslog-server also has the same configuration snippet, so if I let
> security.* uncommented, it duplicates sshd messages, 2 (because they are
> already duplicated) turns 4.
>
> Any help on this?
>
> Thank you very much.
>
>

From tbergfeld at hq.adiscon.com  Mon Jul 11 10:46:04 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Mon, 11 Jul 2011 10:46:04 +0200
Subject: [rsyslog] rsyslog 4.7.4 (v4-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FBE@GRFEXC.intern.adiscon.com>

With this release, v4-devel is switch to beta state. It primarily contains
bug fixes and forward compatibility patches.

ChangeLog:

http://www.rsyslog.com/changelog-for-4-7-4-v4-beta/

Download:

http://www.rsyslog.com/rsyslog-4-7-4-v4-beta/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From tbergfeld at hq.adiscon.com  Mon Jul 11 11:02:38 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Mon, 11 Jul 2011 11:02:38 +0200
Subject: [rsyslog] rsyslog 5.8.3 (v5-stable) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FBF@GRFEXC.intern.adiscon.com>

This release adds support for new systemd features as well as some
forward-compatibility patches. Note that the syntax of the outchannel action
has slightly changed. The previous syntax is still accepted, but a warning
message describing the new syntax is issued.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-8-3-v5-stable/

Download:

http://www.rsyslog.com/rsyslog-5-8-3-v5-stable/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld
--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From tbergfeld at hq.adiscon.com  Mon Jul 11 11:26:03 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Mon, 11 Jul 2011 11:26:03 +0200
Subject: [rsyslog] rsyslog 4.6.7 (v4-stable) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FC0@GRFEXC.intern.adiscon.com>

This update introduces some forward-compatibility constructs.

ChangeLog:

http://www.rsyslog.com/changelog-for-4-6-7-v4-stable/

Download:

http://www.rsyslog.com/rsyslog-4-6-7-v4-stable/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From tbergfeld at hq.adiscon.com  Mon Jul 11 11:36:27 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Mon, 11 Jul 2011 11:36:27 +0200
Subject: [rsyslog] rsyslog 5.9.2 (devel) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FC3@GRFEXC.intern.adiscon.com>

This release introduces support for enhanced systemd features as well as some
forward-compatiblity patches.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-9-2-v5-devel/

Download:

http://www.rsyslog.com/rsyslog-5-9-2-devel/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From tbergfeld at hq.adiscon.com  Mon Jul 11 11:48:17 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Mon, 11 Jul 2011 11:48:17 +0200
Subject: [rsyslog] rsyslog 6.1.11 (v6-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FC5@GRFEXC.intern.adiscon.com>

This release supports new enhancements in systemd and provides
forward-compatibility patches.

ChangeLog:

http://www.rsyslog.com/changelog-for-6-1-11-v6-beta/

Download:

http://www.rsyslog.com/rsyslog-6-1-11-beta/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From forums at clustermagnet.com  Mon Jul 11 17:11:56 2011
From: forums at clustermagnet.com (V B)
Date: Mon, 11 Jul 2011 11:11:56 -0400
Subject: [rsyslog] multithreading question
Message-ID: <CAAB=VsmOkOGKjShRe_eLcDJ546KCRnoOkx9JRcaqM5A7VpthNw@mail.gmail.com>

Gents,
  I have compiled rsyslog with --enable-pthreads, and can see the threads.

  20087 root      15   0  100m 4060 1704 R 37.0  0.1   0:52.24 rs:main Q:Reg

  20088 root      16   0  100m 4060 1704 R  4.0  0.1   0:07.15 rs:action 2
que
  20084 root      16   0  100m 4060 1704 S  1.0  0.1   0:03.75 rsyslogd


  Also, what this tells me (from this graph
http://www.rsyslog.com/doc/queues_analogy.html) is that most processing
power is in the input, pre-processor, and the filter engines.

  Any way we can speed this up?

  Also, it seems that throwing more cores at rsyslog would not help...  a 4
core machine should do just fine.  4GB ram sufficient?

Thanks!

From rgerhards at hq.adiscon.com  Mon Jul 11 17:39:03 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jul 2011 17:39:03 +0200
Subject: [rsyslog] multithreading question
In-Reply-To: <CAAB=VsmOkOGKjShRe_eLcDJ546KCRnoOkx9JRcaqM5A7VpthNw@mail.gmail.com>
References: <CAAB=VsmOkOGKjShRe_eLcDJ546KCRnoOkx9JRcaqM5A7VpthNw@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FCD@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of V B
> Sent: Monday, July 11, 2011 5:12 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] multithreading question
> 
> Gents,
>   I have compiled rsyslog with --enable-pthreads, and can see the
> threads.

Which version?

> 
>   20087 root      15   0  100m 4060 1704 R 37.0  0.1   0:52.24 rs:main
> Q:Reg
> 
>   20088 root      16   0  100m 4060 1704 R  4.0  0.1   0:07.15
> rs:action 2
> que
>   20084 root      16   0  100m 4060 1704 S  1.0  0.1   0:03.75 rsyslogd
> 
> 
>   Also, what this tells me (from this graph
> http://www.rsyslog.com/doc/queues_analogy.html) is that most processing
> power is in the input, pre-processor, and the filter engines.

Well, kind of. That's a pretty user-level paper leaving out many subtle
issues. You should at least also read the queue doc paper.

> 
>   Any way we can speed this up?

What's the problem?

> 
>   Also, it seems that throwing more cores at rsyslog would not help...
> a 4
> core machine should do just fine.

In most cases yes, but depends on the config.

>  4GB ram sufficient?
>
Depends on the config, usually far sufficient.

Rainer
 
> Thanks!
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Mon Jul 11 17:45:58 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jul 2011 17:45:58 +0200
Subject: [rsyslog] RFC: Dropping Emergency Config System
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FCE@GRFEXC.intern.adiscon.com>

Hi all,

since long, rsyslog has a so-called "emergency config system" which  provides
a very minimal config in case rsyslog can not load the real config. I am
working on that system, which creates some complexity inside the code. Most
importantly, I noticed that somewhere along development, that system notably
degraded, obviously without anybody noticing. All it currently does is spit
out startup error messages to some well known destinations (like the system
console). It does NOT process the kernel log or the regular log socket.

As nobody reported any problems with the system, I guess nobody really used
it. In order to streamline the code, I am about to drop it from v6 (even more
so because systemd handles many of the situations this system originally was
thought for [1]). Removing helps getting cleaner, less complex and faster to
work on code.

Any objections against dropping the emergency config system? If so, please
let know the exact reason because I need to remodel the system in any case
and this feedback would be very useful (plus prove the point that there is
real need for this system ;)).

Thanks,
Rainer

[1] http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html

From brian at interlinx.bc.ca  Mon Jul 11 17:19:21 2011
From: brian at interlinx.bc.ca (Brian J. Murrell)
Date: Mon, 11 Jul 2011 11:19:21 -0400
Subject: [rsyslog] combining multiline kernel messages into a single message
Message-ID: <ivf49q$98i$1@dough.gmane.org>

I am just starting down this road, so please forgive my ignorance and
any ill-conceived assumptions.

I want to centralize logging of multiple hosts to a single host.  There
is one artifact of doing so (and in fact it's not even particular to
forwarding -- it seems to happen on a single node) that I want to
resolve and that's the intermingling of log messages in the middle of
what should be multiline kernel messages.  Think stack traces.

The kernel dumps a few dozen lines onto /dev/kmsg which in reality
represent a single messages.  It happens with the OOM killer but
probably, the most common case is stack traces, which each line in the
trace is logged as a separate syslog line with a date and time and host
stamp, etc.

The problem is that it's possible for other messages to be printed in
between these multiple lines/messages from the kernel.  I'd like to try
to atomize these kernel messages so that they are guaranteed to be
together in the log without other messages interspersed in them.

Is this something that's realistic to achieve or is it fraught with to
many problems to be able to do reliably?

Thanx,
b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110711/7c97d4c7/attachment.pgp>

From david at lang.hm  Mon Jul 11 21:16:19 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 12:16:19 -0700 (PDT)
Subject: [rsyslog] multithreading question
In-Reply-To: <CAAB=VsmOkOGKjShRe_eLcDJ546KCRnoOkx9JRcaqM5A7VpthNw@mail.gmail.com>
References: <CAAB=VsmOkOGKjShRe_eLcDJ546KCRnoOkx9JRcaqM5A7VpthNw@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107111209010.24119@asgard.lang.hm>

On Mon, 11 Jul 2011, V B wrote:

> Gents,
>  I have compiled rsyslog with --enable-pthreads, and can see the threads.
>
>  20087 root      15   0  100m 4060 1704 R 37.0  0.1   0:52.24 rs:main Q:Reg
>
>  20088 root      16   0  100m 4060 1704 R  4.0  0.1   0:07.15 rs:action 2
> que
>  20084 root      16   0  100m 4060 1704 S  1.0  0.1   0:03.75 rsyslogd
>
>
>  Also, what this tells me (from this graph
> http://www.rsyslog.com/doc/queues_analogy.html) is that most processing
> power is in the input, pre-processor, and the filter engines.
>
>  Any way we can speed this up?
>
>  Also, it seems that throwing more cores at rsyslog would not help...  a 4
> core machine should do just fine.  4GB ram sufficient?

the number of threads rsyslog uses depends on the configuration (more 
inputs mean more threads for example), but with your configuration it 
looks like more cores would not make any difference.

ram needs to be large enough to handle your queues, depending on how large 
you make the queues and how you set max message size, and how much the 
queues get backed up it can eat more memory.

as for asking to speed it up, if what you snipped is the output from top, 
it seems to be showing that no thread in rsyslog is using more than 37% 
cpu, so you are not being limited by cpu speed.

there are a number of things that can be done in the configuration to make 
it take less cpu to process the log messages, but the first question is 
what are you doing that is noticing a problem (also, what version are you 
running, new versions are FAR faster than old versions)


David Lang

From david at lang.hm  Mon Jul 11 21:19:08 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 12:19:08 -0700 (PDT)
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FCE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FCE@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107111217011.24119@asgard.lang.hm>

systemd is not a valid reason for removing it (systemd is linux-only and 
idn't even on all linux systems)

that being said, as long as rsyslog can spit messages out to stderr to let 
someone know when there are problems starting up, I would not expect it to 
do anything more, and would probably be surprised (in a nasty way) if 
rsyslog processed logs and sent them somewhere I didn't specify.

David Lang

  On Mon, 
11 Jul 2011, Rainer Gerhards wrote:

> Date: Mon, 11 Jul 2011 17:45:58 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: [rsyslog] RFC: Dropping Emergency Config System
> 
> Hi all,
>
> since long, rsyslog has a so-called "emergency config system" which  provides
> a very minimal config in case rsyslog can not load the real config. I am
> working on that system, which creates some complexity inside the code. Most
> importantly, I noticed that somewhere along development, that system notably
> degraded, obviously without anybody noticing. All it currently does is spit
> out startup error messages to some well known destinations (like the system
> console). It does NOT process the kernel log or the regular log socket.
>
> As nobody reported any problems with the system, I guess nobody really used
> it. In order to streamline the code, I am about to drop it from v6 (even more
> so because systemd handles many of the situations this system originally was
> thought for [1]). Removing helps getting cleaner, less complex and faster to
> work on code.
>
> Any objections against dropping the emergency config system? If so, please
> let know the exact reason because I need to remodel the system in any case
> and this feedback would be very useful (plus prove the point that there is
> real need for this system ;)).
>
> Thanks,
> Rainer
>
> [1] http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From david at lang.hm  Mon Jul 11 21:29:48 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 12:29:48 -0700 (PDT)
Subject: [rsyslog] combining multiline kernel messages into a single
 message
In-Reply-To: <ivf49q$98i$1@dough.gmane.org>
References: <ivf49q$98i$1@dough.gmane.org>
Message-ID: <alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>

>I am just starting down this road, so please forgive my ignorance and
>any ill-conceived assumptions.
>
>I want to centralize logging of multiple hosts to a single host.  There
>is one artifact of doing so (and in fact it's not even particular to
>forwarding -- it seems to happen on a single node) that I want to
>resolve and that's the intermingling of log messages in the middle of
>what should be multiline kernel messages.  Think stack traces.
>
>The kernel dumps a few dozen lines onto /dev/kmsg which in reality
>represent a single messages.  It happens with the OOM killer but
>probably, the most common case is stack traces, which each line in the
>trace is logged as a separate syslog line with a date and time and host
>stamp, etc.
>
>The problem is that it's possible for other messages to be printed in
>between these multiple lines/messages from the kernel.  I'd like to try
>to atomize these kernel messages so that they are guaranteed to be
>together in the log without other messages interspersed in them.
>
>Is this something that's realistic to achieve or is it fraught with to
>many problems to be able to do reliably?

we do have some logic in the imfile to allow it to combine multiple line 
logs into one log entry, it has a problem right now that it doesn't 
sanitize the newlines so when you forward it things will probably get 
broken up again.

similar logic could probably be added to the kmesg input, but it is going 
to be tricky.

the problem that you have is how to clearly identify the beginning of a 
new message. in imfile it can do three things

1. each line is a separate log

2. if a line starts with whitespace, it's part of a earlier log entry

3. log entries are separated by a blank line

now, #2 and #3 both mean that the log entry cannot be sent until the next 
log entry arrives (otherwise you don't know if the next data to appear is 
part of the log entry you are working on or not)

with messages from the kernel, if they have timestamps on the front of 
them added by the kernel, it is going to be hard to tell if it is a 
continuation or not, but without those timestampes (or if you teach kmesg 
input to skip them), I think that approach #2 will work.

I would suggest dumping dmesg output from a few systems and try running 
them through the imfile mode #2 and see how the results look.

I think that the kernel is pretty consistant about having every new 
message start at the beginning of a line, but I'm not sure that it indents 
everything (especially oops and stack trace messages)

even if it doesn't indent them, you could add logic to the kmesg input 
module to watch for, and combine the multi-line log messages, but that 
could become very messy (and change from kernel version to kernel 
version), if the simple indentation approach does not work, I would 
suggest asking on the kernel mailing list before starting any other work.

David Lang

From epiphani at gmail.com  Mon Jul 11 21:30:25 2011
From: epiphani at gmail.com (Aaron Wiebe)
Date: Mon, 11 Jul 2011 15:30:25 -0400
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <alpine.DEB.2.02.1107111217011.24119@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FCE@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1107111217011.24119@asgard.lang.hm>
Message-ID: <CAPjSzr8OP_NK0chshXrhqMUDd0m+2yASn5G9pzJEep_=V+omcQ@mail.gmail.com>

There are also pretty valid reasons for having the ability to turn it
off.  If it's not a compile-time flag today, it should probably be
made one.  If there are errors, I'd like it to fail out rather than
start up anyway in a lot of cases.

On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
> systemd is not a valid reason for removing it (systemd is linux-only and
> idn't even on all linux systems)
>
> that being said, as long as rsyslog can spit messages out to stderr to let
> someone know when there are problems starting up, I would not expect it to
> do anything more, and would probably be surprised (in a nasty way) if
> rsyslog processed logs and sent them somewhere I didn't specify.
>
> David Lang
>
> ?On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>
>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>
>> Hi all,
>>
>> since long, rsyslog has a so-called "emergency config system" which
>> ?provides
>> a very minimal config in case rsyslog can not load the real config. I am
>> working on that system, which creates some complexity inside the code.
>> Most
>> importantly, I noticed that somewhere along development, that system
>> notably
>> degraded, obviously without anybody noticing. All it currently does is
>> spit
>> out startup error messages to some well known destinations (like the
>> system
>> console). It does NOT process the kernel log or the regular log socket.
>>
>> As nobody reported any problems with the system, I guess nobody really
>> used
>> it. In order to streamline the code, I am about to drop it from v6 (even
>> more
>> so because systemd handles many of the situations this system originally
>> was
>> thought for [1]). Removing helps getting cleaner, less complex and faster
>> to
>> work on code.
>>
>> Any objections against dropping the emergency config system? If so, please
>> let know the exact reason because I need to remodel the system in any case
>> and this feedback would be very useful (plus prove the point that there is
>> real need for this system ;)).
>>
>> Thanks,
>> Rainer
>>
>> [1]
>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From corey at sequestered.net  Mon Jul 11 21:48:47 2011
From: corey at sequestered.net (Corey Quinn)
Date: Mon, 11 Jul 2011 12:48:47 -0700
Subject: [rsyslog] Changelogs
Message-ID: <9ABA1130-F7B5-428F-9F11-3B09AF51A8D9@sequestered.net>

Is there a unified changelog somewhere I can peruse?  I'm trying to figure out at what point RELP began supporting fromhost selectors, but having a bit of a trouble narrowing it down.

-- Corey / KB1JWQ

From david at lang.hm  Mon Jul 11 22:11:59 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 13:11:59 -0700 (PDT)
Subject: [rsyslog] Changelogs
In-Reply-To: <9ABA1130-F7B5-428F-9F11-3B09AF51A8D9@sequestered.net>
References: <9ABA1130-F7B5-428F-9F11-3B09AF51A8D9@sequestered.net>
Message-ID: <alpine.DEB.2.02.1107111310430.24119@asgard.lang.hm>

the best way to search for something like this is to use git.

clone the tree and then you can either examine the logs, or setup a test 
and use git bisect to find exactly when this capibility was added.

David Lang


On Mon, 11 Jul 2011, Corey Quinn wrote:

> Is there a unified changelog somewhere I can peruse?  I'm trying to 
> figure out at what point RELP began supporting fromhost selectors, but 
> having a bit of a trouble narrowing it down.
>
> -- Corey / KB1JWQ
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Mon Jul 11 22:43:09 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jul 2011 22:43:09 +0200
Subject: [rsyslog] Changelogs
Message-ID: <001d01cc400b$22e5a66d$100013ac@intern.adiscon.com>

For a quick look, i usually find the file ChangeLog right in the root of the project sufficient.

Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:the best way to search for something like this is to use git.

clone the tree and then you can either examine the logs, or setup a test 
and use git bisect to find exactly when this capibility was added.

David Lang


On Mon, 11 Jul 2011, Corey Quinn wrote:

> Is there a unified changelog somewhere I can peruse?  I'm trying to 
> figure out at what point RELP began supporting fromhost selectors, but 
> having a bit of a trouble narrowing it down.
>
> -- Corey / KB1JWQ
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Mon Jul 11 22:50:18 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jul 2011 22:50:18 +0200
Subject: [rsyslog] RFC: Dropping Emergency Config System
Message-ID: <001f01cc400c$236ecc9f$100013ac@intern.adiscon.com>

The question is if we need more than stderr. It is surprisingly complicated to do this in a clean way, as the necessary plumbing is not present.

RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are also pretty valid reasons for having the ability to turn it
off.  If it's not a compile-time flag today, it should probably be
made one.  If there are errors, I'd like it to fail out rather than
start up anyway in a lot of cases.

On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
> systemd is not a valid reason for removing it (systemd is linux-only and
> idn't even on all linux systems)
>
> that being said, as long as rsyslog can spit messages out to stderr to let
> someone know when there are problems starting up, I would not expect it to
> do anything more, and would probably be surprised (in a nasty way) if
> rsyslog processed logs and sent them somewhere I didn't specify.
>
> David Lang
>
>  On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>
>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>
>> Hi all,
>>
>> since long, rsyslog has a so-called "emergency config system" which
>>  provides
>> a very minimal config in case rsyslog can not load the real config. I am
>> working on that system, which creates some complexity inside the code.
>> Most
>> importantly, I noticed that somewhere along development, that system
>> notably
>> degraded, obviously without anybody noticing. All it currently does is
>> spit
>> out startup error messages to some well known destinations (like the
>> system
>> console). It does NOT process the kernel log or the regular log socket.
>>
>> As nobody reported any problems with the system, I guess nobody really
>> used
>> it. In order to streamline the code, I am about to drop it from v6 (even
>> more
>> so because systemd handles many of the situations this system originally
>> was
>> thought for [1]). Removing helps getting cleaner, less complex and faster
>> to
>> work on code.
>>
>> Any objections against dropping the emergency config system? If so, please
>> let know the exact reason because I need to remodel the system in any case
>> and this feedback would be very useful (plus prove the point that there is
>> real need for this system ;)).
>>
>> Thanks,
>> Rainer
>>
>> [1]
>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From david at lang.hm  Mon Jul 11 22:56:47 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 13:56:47 -0700 (PDT)
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <001f01cc400c$236ecc9f$100013ac@intern.adiscon.com>
References: <001f01cc400c$236ecc9f$100013ac@intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107111356300.24119@asgard.lang.hm>

other than stderr, what does the current system try to do?

David Lang


On Mon, 11 Jul 2011, Rainer Gerhards wrote:

> Date: Mon, 11 Jul 2011 22:50:18 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> 
> The question is if we need more than stderr. It is surprisingly complicated to do this in a clean way, as the necessary plumbing is not present.
>
> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are also pretty valid reasons for having the ability to turn it
> off.  If it's not a compile-time flag today, it should probably be
> made one.  If there are errors, I'd like it to fail out rather than
> start up anyway in a lot of cases.
>
> On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
>> systemd is not a valid reason for removing it (systemd is linux-only and
>> idn't even on all linux systems)
>>
>> that being said, as long as rsyslog can spit messages out to stderr to let
>> someone know when there are problems starting up, I would not expect it to
>> do anything more, and would probably be surprised (in a nasty way) if
>> rsyslog processed logs and sent them somewhere I didn't specify.
>>
>> David Lang
>>
>>  On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>>
>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>>
>>> Hi all,
>>>
>>> since long, rsyslog has a so-called "emergency config system" which
>>>  provides
>>> a very minimal config in case rsyslog can not load the real config. I am
>>> working on that system, which creates some complexity inside the code.
>>> Most
>>> importantly, I noticed that somewhere along development, that system
>>> notably
>>> degraded, obviously without anybody noticing. All it currently does is
>>> spit
>>> out startup error messages to some well known destinations (like the
>>> system
>>> console). It does NOT process the kernel log or the regular log socket.
>>>
>>> As nobody reported any problems with the system, I guess nobody really
>>> used
>>> it. In order to streamline the code, I am about to drop it from v6 (even
>>> more
>>> so because systemd handles many of the situations this system originally
>>> was
>>> thought for [1]). Removing helps getting cleaner, less complex and faster
>>> to
>>> work on code.
>>>
>>> Any objections against dropping the emergency config system? If so, please
>>> let know the exact reason because I need to remodel the system in any case
>>> and this feedback would be very useful (plus prove the point that there is
>>> real need for this system ;)).
>>>
>>> Thanks,
>>> Rainer
>>>
>>> [1]
>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Mon Jul 11 23:03:12 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jul 2011 23:03:12 +0200
Subject: [rsyslog] RFC: Dropping Emergency Config System
Message-ID: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>

Out of my head. It is sysklogd legacy. Four rules, among them

*.err /dev/console
Panic.* *

Two more. Originally, it also read the system socket, which was lost some way around the road. I think it doesnt work for a couple of years now and nobody ever noticed. I just came across it due to new config. . .
Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:other than stderr, what does the current system try to do?

David Lang


On Mon, 11 Jul 2011, Rainer Gerhards wrote:

> Date: Mon, 11 Jul 2011 22:50:18 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> 
> The question is if we need more than stderr. It is surprisingly complicated to do this in a clean way, as the necessary plumbing is not present.
>
> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are also pretty valid reasons for having the ability to turn it
> off.  If it's not a compile-time flag today, it should probably be
> made one.  If there are errors, I'd like it to fail out rather than
> start up anyway in a lot of cases.
>
> On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
>> systemd is not a valid reason for removing it (systemd is linux-only and
>> idn't even on all linux systems)
>>
>> that being said, as long as rsyslog can spit messages out to stderr to let
>> someone know when there are problems starting up, I would not expect it to
>> do anything more, and would probably be surprised (in a nasty way) if
>> rsyslog processed logs and sent them somewhere I didn't specify.
>>
>> David Lang
>>
>>  On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>>
>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>>
>>> Hi all,
>>>
>>> since long, rsyslog has a so-called "emergency config system" which
>>>  provides
>>> a very minimal config in case rsyslog can not load the real config. I am
>>> working on that system, which creates some complexity inside the code.
>>> Most
>>> importantly, I noticed that somewhere along development, that system
>>> notably
>>> degraded, obviously without anybody noticing. All it currently does is
>>> spit
>>> out startup error messages to some well known destinations (like the
>>> system
>>> console). It does NOT process the kernel log or the regular log socket.
>>>
>>> As nobody reported any problems with the system, I guess nobody really
>>> used
>>> it. In order to streamline the code, I am about to drop it from v6 (even
>>> more
>>> so because systemd handles many of the situations this system originally
>>> was
>>> thought for [1]). Removing helps getting cleaner, less complex and faster
>>> to
>>> work on code.
>>>
>>> Any objections against dropping the emergency config system? If so, please
>>> let know the exact reason because I need to remodel the system in any case
>>> and this feedback would be very useful (plus prove the point that there is
>>> real need for this system ;)).
>>>
>>> Thanks,
>>> Rainer
>>>
>>> [1]
>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Mon Jul 11 23:08:56 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jul 2011 23:08:56 +0200
Subject: [rsyslog] RFC: Dropping Emergency Config System
Message-ID: <002101cc400e$ba64c620$100013ac@intern.adiscon.com>

Oh, and the current system does *not* use stderr.
Rainer

Rainer Gerhards <rgerhards at hq.adiscon.com> hat geschrieben:Out of my head. It is sysklogd legacy. Four rules, among them

*.err /dev/console
Panic.* *

Two more. Originally, it also read the system socket, which was lost some way around the road. I think it doesnt work for a couple of years now and nobody ever noticed. I just came across it due to new config. . .
Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:other than stderr, what does the current system try to do?

David Lang


On Mon, 11 Jul 2011, Rainer Gerhards wrote:

> Date: Mon, 11 Jul 2011 22:50:18 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> 
> The question is if we need more than stderr. It is surprisingly complicated to do this in a clean way, as the necessary plumbing is not present.
>
> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are also pretty valid reasons for having the ability to turn it
> off.  If it's not a compile-time flag today, it should probably be
> made one.  If there are errors, I'd like it to fail out rather than
> start up anyway in a lot of cases.
>
> On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
>> systemd is not a valid reason for removing it (systemd is linux-only and
>> idn't even on all linux systems)
>>
>> that being said, as long as rsyslog can spit messages out to stderr to let
>> someone know when there are problems starting up, I would not expect it to
>> do anything more, and would probably be surprised (in a nasty way) if
>> rsyslog processed logs and sent them somewhere I didn't specify.
>>
>> David Lang
>>
>>  On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>>
>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>>
>>> Hi all,
>>>
>>> since long, rsyslog has a so-called "emergency config system" which
>>>  provides
>>> a very minimal config in case rsyslog can not load the real config. I am
>>> working on that system, which creates some complexity inside the code.
>>> Most
>>> importantly, I noticed that somewhere along development, that system
>>> notably
>>> degraded, obviously without anybody noticing. All it currently does is
>>> spit
>>> out startup error messages to some well known destinations (like the
>>> system
>>> console). It does NOT process the kernel log or the regular log socket.
>>>
>>> As nobody reported any problems with the system, I guess nobody really
>>> used
>>> it. In order to streamline the code, I am about to drop it from v6 (even
>>> more
>>> so because systemd handles many of the situations this system originally
>>> was
>>> thought for [1]). Removing helps getting cleaner, less complex and faster
>>> to
>>> work on code.
>>>
>>> Any objections against dropping the emergency config system? If so, please
>>> let know the exact reason because I need to remodel the system in any case
>>> and this feedback would be very useful (plus prove the point that there is
>>> real need for this system ;)).
>>>
>>> Thanks,
>>> Rainer
>>>
>>> [1]
>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From david at lang.hm  Mon Jul 11 23:26:48 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 14:26:48 -0700 (PDT)
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>

I think it is probably better to fail noisily

thinking out loud here

it must at least fail with errors to stderr so that someone starting it 
manually can see that it can't read the config file.

this should be for any config failure (i.e. one line it doesn't 
understand), not just complete failure

if it is able to understand the config file enough to get destinations, it 
would probably be a good idea to spit logs to those destinations reporting 
the failure. This is more shaky, but I think it's probably a good idea.

David Lang

  On Mon, 11 Jul 2011, Rainer Gerhards wrote:

> Date: Mon, 11 Jul 2011 23:03:12 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> 
> Out of my head. It is sysklogd legacy. Four rules, among them
>
> *.err /dev/console
> Panic.* *
>
> Two more. Originally, it also read the system socket, which was lost some way around the road. I think it doesnt work for a couple of years now and nobody ever noticed. I just came across it due to new config. . .
> Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:other than stderr, what does the current system try to do?
>
> David Lang
>
>
> On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>
>> Date: Mon, 11 Jul 2011 22:50:18 +0200
>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>> To: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
>>
>> The question is if we need more than stderr. It is surprisingly complicated to do this in a clean way, as the necessary plumbing is not present.
>>
>> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are also pretty valid reasons for having the ability to turn it
>> off.  If it's not a compile-time flag today, it should probably be
>> made one.  If there are errors, I'd like it to fail out rather than
>> start up anyway in a lot of cases.
>>
>> On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
>>> systemd is not a valid reason for removing it (systemd is linux-only and
>>> idn't even on all linux systems)
>>>
>>> that being said, as long as rsyslog can spit messages out to stderr to let
>>> someone know when there are problems starting up, I would not expect it to
>>> do anything more, and would probably be surprised (in a nasty way) if
>>> rsyslog processed logs and sent them somewhere I didn't specify.
>>>
>>> David Lang
>>>
>>>  On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>>>
>>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>>>
>>>> Hi all,
>>>>
>>>> since long, rsyslog has a so-called "emergency config system" which
>>>>  provides
>>>> a very minimal config in case rsyslog can not load the real config. I am
>>>> working on that system, which creates some complexity inside the code.
>>>> Most
>>>> importantly, I noticed that somewhere along development, that system
>>>> notably
>>>> degraded, obviously without anybody noticing. All it currently does is
>>>> spit
>>>> out startup error messages to some well known destinations (like the
>>>> system
>>>> console). It does NOT process the kernel log or the regular log socket.
>>>>
>>>> As nobody reported any problems with the system, I guess nobody really
>>>> used
>>>> it. In order to streamline the code, I am about to drop it from v6 (even
>>>> more
>>>> so because systemd handles many of the situations this system originally
>>>> was
>>>> thought for [1]). Removing helps getting cleaner, less complex and faster
>>>> to
>>>> work on code.
>>>>
>>>> Any objections against dropping the emergency config system? If so, please
>>>> let know the exact reason because I need to remodel the system in any case
>>>> and this feedback would be very useful (plus prove the point that there is
>>>> real need for this system ;)).
>>>>
>>>> Thanks,
>>>> Rainer
>>>>
>>>> [1]
>>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From epiphani at gmail.com  Mon Jul 11 23:36:32 2011
From: epiphani at gmail.com (Aaron Wiebe)
Date: Mon, 11 Jul 2011 17:36:32 -0400
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
Message-ID: <CAPjSzr91G-hgR16sMDV+nMv3dKNDnnFUdcP79_G3jLq=6zXwCQ@mail.gmail.com>

> I think it is probably better to fail noisily

Precisely - make it so the startup fails, and exits nonzero so RC
scripts know it failed.

Still, having the option to go back in the case a failure shouldn't
result in a non-start is a good idea.  The distros will probably want
that option, since it makes sure that the service comes up in the case
something else is busted.

-Aaron

On Mon, Jul 11, 2011 at 5:26 PM,  <david at lang.hm> wrote:
> I think it is probably better to fail noisily
>
> thinking out loud here
>
> it must at least fail with errors to stderr so that someone starting it
> manually can see that it can't read the config file.
>
> this should be for any config failure (i.e. one line it doesn't understand),
> not just complete failure
>
> if it is able to understand the config file enough to get destinations, it
> would probably be a good idea to spit logs to those destinations reporting
> the failure. This is more shaky, but I think it's probably a good idea.
>
> David Lang
>
> ?On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>
>> Date: Mon, 11 Jul 2011 23:03:12 +0200
>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>> To: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
>>
>> Out of my head. It is sysklogd legacy. Four rules, among them
>>
>> *.err /dev/console
>> Panic.* *
>>
>> Two more. Originally, it also read the system socket, which was lost some
>> way around the road. I think it doesnt work for a couple of years now and
>> nobody ever noticed. I just came across it due to new config. . .
>> Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:other than stderr,
>> what does the current system try to do?
>>
>> David Lang
>>
>>
>> On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>>
>>> Date: Mon, 11 Jul 2011 22:50:18 +0200
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog at lists.adiscon.com
>>> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
>>>
>>> The question is if we need more than stderr. It is surprisingly
>>> complicated to do this in a clean way, as the necessary plumbing is not
>>> present.
>>>
>>> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are also
>>> pretty valid reasons for having the ability to turn it
>>> off. ?If it's not a compile-time flag today, it should probably be
>>> made one. ?If there are errors, I'd like it to fail out rather than
>>> start up anyway in a lot of cases.
>>>
>>> On Mon, Jul 11, 2011 at 3:19 PM, ?<david at lang.hm> wrote:
>>>>
>>>> systemd is not a valid reason for removing it (systemd is linux-only and
>>>> idn't even on all linux systems)
>>>>
>>>> that being said, as long as rsyslog can spit messages out to stderr to
>>>> let
>>>> someone know when there are problems starting up, I would not expect it
>>>> to
>>>> do anything more, and would probably be surprised (in a nasty way) if
>>>> rsyslog processed logs and sent them somewhere I didn't specify.
>>>>
>>>> David Lang
>>>>
>>>> ?On Mon, 11 Jul 2011, Rainer Gerhards wrote:
>>>>
>>>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
>>>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
>>>>>
>>>>> Hi all,
>>>>>
>>>>> since long, rsyslog has a so-called "emergency config system" which
>>>>> ?provides
>>>>> a very minimal config in case rsyslog can not load the real config. I
>>>>> am
>>>>> working on that system, which creates some complexity inside the code.
>>>>> Most
>>>>> importantly, I noticed that somewhere along development, that system
>>>>> notably
>>>>> degraded, obviously without anybody noticing. All it currently does is
>>>>> spit
>>>>> out startup error messages to some well known destinations (like the
>>>>> system
>>>>> console). It does NOT process the kernel log or the regular log socket.
>>>>>
>>>>> As nobody reported any problems with the system, I guess nobody really
>>>>> used
>>>>> it. In order to streamline the code, I am about to drop it from v6
>>>>> (even
>>>>> more
>>>>> so because systemd handles many of the situations this system
>>>>> originally
>>>>> was
>>>>> thought for [1]). Removing helps getting cleaner, less complex and
>>>>> faster
>>>>> to
>>>>> work on code.
>>>>>
>>>>> Any objections against dropping the emergency config system? If so,
>>>>> please
>>>>> let know the exact reason because I need to remodel the system in any
>>>>> case
>>>>> and this feedback would be very useful (plus prove the point that there
>>>>> is
>>>>> real need for this system ;)).
>>>>>
>>>>> Thanks,
>>>>> Rainer
>>>>>
>>>>> [1]
>>>>>
>>>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/002862.html
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From mike.forbes at koordinates.com  Tue Jul 12 00:07:06 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Tue, 12 Jul 2011 10:07:06 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
Message-ID: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>

Rsyslogd is using 1.4G of resident memory.

Here's my config snippets: http://paste.nothing.net.nz/806cba

There's 14 hosts sending data to this rsyslog server.

I've tried setting a ulimit
(syslog          soft     stack           1024)

 via pam & /etc/security/limits.conf

and ulimit -a as the syslog user shows

stack size              (kbytes, -s) 1024


Yet top clearly shows using 1.4G of resident memory:

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
16414 syslog    20   0 2006m 1.4g 1252 S    0 72.1   1:28.83 rsyslogd -c4

my rsyslogd version is rsyslogd 5.6.3

Any advice is welcome.

-- 
// Mike Forbes
GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844

From david at lang.hm  Tue Jul 12 00:28:28 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 15:28:28 -0700 (PDT)
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>

this probably means that rsyslog is not keeping up with the volume of logs 
being sent to it (causing them to queue and eat memory)

I'm not seeing anything really unusual in the configuration. you do have 
one dynafile template, I'm not really familiar with how much overhead that 
causes (I have too many servers to put each in it's own directory, and I 
roll the logs rather than writing them to a dynamic location)

it would be interesting to learn what your log rate per second is.

also, if you know when you are hitting peack volume of logs, it would be 
interesting to see what the cpu utilization (of each thread) looks like 
(using top, turn on the per-thread with 'h' and then see what shows up)

the ulimit you are setting is for the stack, not for the total memory, and 
unless you want rsyslog to be killed when it hits the limit, you don't 
want to control this via ulimits anyway.

at this point 5.6.3 is a bit old, newer versions are probably faster and 
may solve your problem just from being enough faster to keep up with the 
incoming logs.

you may also want to run iostat -x and see if you are maxing out your disk 
I/O during peak times.

David Lang


On Tue, 12 Jul 2011, Mike Forbes wrote:

> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
> 
> Rsyslogd is using 1.4G of resident memory.
>
> Here's my config snippets: http://paste.nothing.net.nz/806cba
>
> There's 14 hosts sending data to this rsyslog server.
>
> I've tried setting a ulimit
> (syslog          soft     stack           1024)
>
> via pam & /etc/security/limits.conf
>
> and ulimit -a as the syslog user shows
>
> stack size              (kbytes, -s) 1024
>
>
> Yet top clearly shows using 1.4G of resident memory:
>
> PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 16414 syslog    20   0 2006m 1.4g 1252 S    0 72.1   1:28.83 rsyslogd -c4
>
> my rsyslogd version is rsyslogd 5.6.3
>
> Any advice is welcome.
>
> -- 
> // Mike Forbes
> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mike.forbes at koordinates.com  Tue Jul 12 05:04:44 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Tue, 12 Jul 2011 15:04:44 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
Message-ID: <CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>

On 12 July 2011 10:28,  <david at lang.hm> wrote:
> this probably means that rsyslog is not keeping up with the volume of logs
> being sent to it (causing them to queue and eat memory)
> at this point 5.6.3 is a bit old, newer versions are probably faster and may
> solve your problem just from being enough faster to keep up with the
> incoming logs.

Pushed up to 5.8.1 and I still get the same amount of memory use.

iostat and cpu use seem reasonably sane, though at peak times most of
the threads peak to 99% CPU use.



> On Tue, 12 Jul 2011, Mike Forbes wrote:
>
>> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>
>> Rsyslogd is using 1.4G of resident memory.
>>
>> Here's my config snippets: http://paste.nothing.net.nz/806cba
>>
>> There's 14 hosts sending data to this rsyslog server.
>>
>> I've tried setting a ulimit
>> (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>
>> via pam & /etc/security/limits.conf
>>
>> and ulimit -a as the syslog user shows
>>
>> stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>
>>
>> Yet top clearly shows using 1.4G of resident memory:
>>
>> PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>> 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd -c4
>>
>> my rsyslogd version is rsyslogd 5.6.3
>>
>> Any advice is welcome.
>>
>> --
>> // Mike Forbes
>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
>



-- 
// Mike Forbes

Operations Engineer
Koordinates Ltd.

PO Box 1604, Shortland St, Auckland 1140, New Zealand

Cell +64-21-999416
Phone +64-9-966 0433
Fax +64-9-969 0045

Web http://koordinates.com/

GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844

From david at lang.hm  Tue Jul 12 05:13:42 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 20:13:42 -0700 (PDT)
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
	<CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107112012440.11437@asgard.lang.hm>

On Tue, 12 Jul 2011, Mike Forbes wrote:

> On 12 July 2011 10:28,  <david at lang.hm> wrote:
>> this probably means that rsyslog is not keeping up with the volume of logs
>> being sent to it (causing them to queue and eat memory)
>> at this point 5.6.3 is a bit old, newer versions are probably faster and may
>> solve your problem just from being enough faster to keep up with the
>> incoming logs.
>
> Pushed up to 5.8.1 and I still get the same amount of memory use.
>
> iostat and cpu use seem reasonably sane, though at peak times most of
> the threads peak to 99% CPU use.

Ok, this does sound like you aren't able to keep up with the incoming log 
volume, what sort of volume do you get?

David Lang

>
>
>> On Tue, 12 Jul 2011, Mike Forbes wrote:
>>
>>> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>>
>>> Rsyslogd is using 1.4G of resident memory.
>>>
>>> Here's my config snippets: http://paste.nothing.net.nz/806cba
>>>
>>> There's 14 hosts sending data to this rsyslog server.
>>>
>>> I've tried setting a ulimit
>>> (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>>
>>> via pam & /etc/security/limits.conf
>>>
>>> and ulimit -a as the syslog user shows
>>>
>>> stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>>
>>>
>>> Yet top clearly shows using 1.4G of resident memory:
>>>
>>> PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>>> 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd -c4
>>>
>>> my rsyslogd version is rsyslogd 5.6.3
>>>
>>> Any advice is welcome.
>>>
>>> --
>>> // Mike Forbes
>>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>
>
>
>
> -- 
> // Mike Forbes
>
> Operations Engineer
> Koordinates Ltd.
>
> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>
> Cell +64-21-999416
> Phone +64-9-966 0433
> Fax +64-9-969 0045
>
> Web http://koordinates.com/
>
> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mike.forbes at koordinates.com  Tue Jul 12 05:18:27 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Tue, 12 Jul 2011 15:18:27 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <alpine.DEB.2.02.1107112012440.11437@asgard.lang.hm>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
	<CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
	<alpine.DEB.2.02.1107112012440.11437@asgard.lang.hm>
Message-ID: <CAPCA4JpFUO-tnsqkGLTDPtY-bDrw0v2DxrVn1SA5Ng6HR8JFNg@mail.gmail.com>

On 12 July 2011 15:13,  <david at lang.hm> wrote:
> Ok, this does sound like you aren't able to keep up with the incoming log
> volume, what sort of volume do you get?

14 hosts, a total of about 70Mb of logs for today, logfile sizes are
between 1-7Mb


>>
>>
>>> On Tue, 12 Jul 2011, Mike Forbes wrote:
>>>
>>>> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>>>
>>>> Rsyslogd is using 1.4G of resident memory.
>>>>
>>>> Here's my config snippets: http://paste.nothing.net.nz/806cba
>>>>
>>>> There's 14 hosts sending data to this rsyslog server.
>>>>
>>>> I've tried setting a ulimit
>>>> (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>>>
>>>> via pam & /etc/security/limits.conf
>>>>
>>>> and ulimit -a as the syslog user shows
>>>>
>>>> stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>>>
>>>>
>>>> Yet top clearly shows using 1.4G of resident memory:
>>>>
>>>> PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>>>> 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd
>>>> -c4
>>>>
>>>> my rsyslogd version is rsyslogd 5.6.3
>>>>
>>>> Any advice is welcome.
>>>>
>>>> --
>>>> // Mike Forbes
>>>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>> --
>> // Mike Forbes
>>
>> Operations Engineer
>> Koordinates Ltd.
>>
>> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>>
>> Cell +64-21-999416
>> Phone +64-9-966 0433
>> Fax +64-9-969 0045
>>
>> Web http://koordinates.com/
>>
>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
>



-- 
// Mike Forbes

Operations Engineer
Koordinates Ltd.

PO Box 1604, Shortland St, Auckland 1140, New Zealand

Cell +64-21-999416
Phone +64-9-966 0433
Fax +64-9-969 0045

Web http://koordinates.com/

GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844

From david at lang.hm  Tue Jul 12 05:38:29 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 20:38:29 -0700 (PDT)
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <CAPCA4JpFUO-tnsqkGLTDPtY-bDrw0v2DxrVn1SA5Ng6HR8JFNg@mail.gmail.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
	<CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
	<alpine.DEB.2.02.1107112012440.11437@asgard.lang.hm>
	<CAPCA4JpFUO-tnsqkGLTDPtY-bDrw0v2DxrVn1SA5Ng6HR8JFNg@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107112030030.11437@asgard.lang.hm>

On Tue, 12 Jul 2011, Mike Forbes wrote:

> On 12 July 2011 15:13,  <david at lang.hm> wrote:
>> Ok, this does sound like you aren't able to keep up with the incoming log
>> volume, what sort of volume do you get?
>
> 14 hosts, a total of about 70Mb of logs for today, logfile sizes are
> between 1-7Mb

I was thinking in terms of logs per min or logs per second

70MB of logs for a day is a fair bit (and depending on how bunched up they 
are when they arrive, I could see the data rate being pretty high)

as an experiment, can you try disabeling the per-hosts dynafile and 
replace it with something just writing all the logs into one file? I am 
wondering how much that ends up costing in CPU time.

also, unless you have a raid card with a good amount of cach on it, you 
could be having problems with the disk keeping up as you seek back and 
forth to the different files.

so I would suggest doing the following tests

remove the dynafile option and just have everything in one file

change the dynafile option to remove the hostname but keep the date/time 
portions.

I think you will see the peak load go up as you enable dynafiles at all, 
but UI suspect that you will have significantly lower peaks with your I/O 
useage, and that should keep the memory consumption down (as rsyslog never 
backs up waiting for the disk)


In your place, I would be rotating the files periodically (with a script 
run out of cron), and having the log rotation script break them down to 
the different servers if you need that.

David Lang

>
>>>
>>>
>>>> On Tue, 12 Jul 2011, Mike Forbes wrote:
>>>>
>>>>> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>>>>
>>>>> Rsyslogd is using 1.4G of resident memory.
>>>>>
>>>>> Here's my config snippets: http://paste.nothing.net.nz/806cba
>>>>>
>>>>> There's 14 hosts sending data to this rsyslog server.
>>>>>
>>>>> I've tried setting a ulimit
>>>>> (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>>>>
>>>>> via pam & /etc/security/limits.conf
>>>>>
>>>>> and ulimit -a as the syslog user shows
>>>>>
>>>>> stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>>>>
>>>>>
>>>>> Yet top clearly shows using 1.4G of resident memory:
>>>>>
>>>>> PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>>>>> 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd
>>>>> -c4
>>>>>
>>>>> my rsyslogd version is rsyslogd 5.6.3
>>>>>
>>>>> Any advice is welcome.
>>>>>
>>>>> --
>>>>> // Mike Forbes
>>>>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> // Mike Forbes
>>>
>>> Operations Engineer
>>> Koordinates Ltd.
>>>
>>> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>>>
>>> Cell +64-21-999416
>>> Phone +64-9-966 0433
>>> Fax +64-9-969 0045
>>>
>>> Web http://koordinates.com/
>>>
>>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>
>
>
>
> -- 
> // Mike Forbes
>
> Operations Engineer
> Koordinates Ltd.
>
> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>
> Cell +64-21-999416
> Phone +64-9-966 0433
> Fax +64-9-969 0045
>
> Web http://koordinates.com/
>
> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Tue Jul 12 05:39:31 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jul 2011 20:39:31 -0700 (PDT)
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
	<CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107112038550.11437@asgard.lang.hm>

On Tue, 12 Jul 2011, Mike Forbes wrote:

> On 12 July 2011 10:28,  <david at lang.hm> wrote:
>> this probably means that rsyslog is not keeping up with the volume of logs
>> being sent to it (causing them to queue and eat memory)
>> at this point 5.6.3 is a bit old, newer versions are probably faster and may
>> solve your problem just from being enough faster to keep up with the
>> incoming logs.
>
> Pushed up to 5.8.1 and I still get the same amount of memory use.
>
> iostat and cpu use seem reasonably sane, though at peak times most of
> the threads peak to 99% CPU use.

sorry, I forgot to ask, does the iostat -x option show any of the drives 
peaking at 100%?

David Lang

>
>
>> On Tue, 12 Jul 2011, Mike Forbes wrote:
>>
>>> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>>
>>> Rsyslogd is using 1.4G of resident memory.
>>>
>>> Here's my config snippets: http://paste.nothing.net.nz/806cba
>>>
>>> There's 14 hosts sending data to this rsyslog server.
>>>
>>> I've tried setting a ulimit
>>> (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>>
>>> via pam & /etc/security/limits.conf
>>>
>>> and ulimit -a as the syslog user shows
>>>
>>> stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>>
>>>
>>> Yet top clearly shows using 1.4G of resident memory:
>>>
>>> PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>>> 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd -c4
>>>
>>> my rsyslogd version is rsyslogd 5.6.3
>>>
>>> Any advice is welcome.
>>>
>>> --
>>> // Mike Forbes
>>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>
>
>
>
> -- 
> // Mike Forbes
>
> Operations Engineer
> Koordinates Ltd.
>
> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>
> Cell +64-21-999416
> Phone +64-9-966 0433
> Fax +64-9-969 0045
>
> Web http://koordinates.com/
>
> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mike.forbes at koordinates.com  Tue Jul 12 05:40:22 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Tue, 12 Jul 2011 15:40:22 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <alpine.DEB.2.02.1107112038550.11437@asgard.lang.hm>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
	<CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
	<alpine.DEB.2.02.1107112038550.11437@asgard.lang.hm>
Message-ID: <CAPCA4JpL=WD+yon9RCumuXb6nkfTgT3mbrgoOVzsDkUWTBepCA@mail.gmail.com>

On 12 July 2011 15:39,  <david at lang.hm> wrote:
> sorry, I forgot to ask, does the iostat -x option show any of the drives
> peaking at 100%?

Not that i've seen so far, no.

I'll give your other suggestions a try

Cheers,
>>
>>
>>> On Tue, 12 Jul 2011, Mike Forbes wrote:
>>>
>>>> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>>>
>>>> Rsyslogd is using 1.4G of resident memory.
>>>>
>>>> Here's my config snippets: http://paste.nothing.net.nz/806cba
>>>>
>>>> There's 14 hosts sending data to this rsyslog server.
>>>>
>>>> I've tried setting a ulimit
>>>> (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>>>
>>>> via pam & /etc/security/limits.conf
>>>>
>>>> and ulimit -a as the syslog user shows
>>>>
>>>> stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>>>
>>>>
>>>> Yet top clearly shows using 1.4G of resident memory:
>>>>
>>>> PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>>>> 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd
>>>> -c4
>>>>
>>>> my rsyslogd version is rsyslogd 5.6.3
>>>>
>>>> Any advice is welcome.
>>>>
>>>> --
>>>> // Mike Forbes
>>>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>> --
>> // Mike Forbes
>>
>> Operations Engineer
>> Koordinates Ltd.
>>
>> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>>
>> Cell +64-21-999416
>> Phone +64-9-966 0433
>> Fax +64-9-969 0045
>>
>> Web http://koordinates.com/
>>
>> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
>



-- 
// Mike Forbes

Operations Engineer
Koordinates Ltd.

PO Box 1604, Shortland St, Auckland 1140, New Zealand

Cell +64-21-999416
Phone +64-9-966 0433
Fax +64-9-969 0045

Web http://koordinates.com/

GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844

From mike.forbes at koordinates.com  Tue Jul 12 06:15:18 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Tue, 12 Jul 2011 16:15:18 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <alpine.DEB.2.02.1107112030030.11437@asgard.lang.hm>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<alpine.DEB.2.02.1107111519330.11437@asgard.lang.hm>
	<CAPCA4JoiQL8Bob+hPecAWBcmTQqZJ39E80ir_zMi+jMYssiwUA@mail.gmail.com>
	<alpine.DEB.2.02.1107112012440.11437@asgard.lang.hm>
	<CAPCA4JpFUO-tnsqkGLTDPtY-bDrw0v2DxrVn1SA5Ng6HR8JFNg@mail.gmail.com>
	<alpine.DEB.2.02.1107112030030.11437@asgard.lang.hm>
Message-ID: <CAPCA4JpZCLthmxowgWRJynu2VZHUwsVYsFOp=MY_oOZ9Xau07w@mail.gmail.com>

On 12 July 2011 15:38,  <david at lang.hm> wrote:
> remove the dynafile option and just have everything in one file

Memory use back up to ~1.2Gb with all the logs dropping into one file.

:(

From rgerhards at hq.adiscon.com  Tue Jul 12 09:14:44 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jul 2011 09:14:44 +0200
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FD1@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, July 11, 2011 11:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> 
> I think it is probably better to fail noisily
> 
> thinking out loud here
> 
> it must at least fail with errors to stderr so that someone starting it
manually
> can see that it can't read the config file.
> 
> this should be for any config failure (i.e. one line it doesn't
understand), not
> just complete failure

Thats already done since ... well 2 years? One? Along those lines...

> 
> if it is able to understand the config file enough to get destinations, it
would
> probably be a good idea to spit logs to those destinations reporting the
> failure. This is more shaky, but I think it's probably a good idea.

We currently take the "use a partial config approach". The emergency config
primarily kicked in when no actions at all were defined. For 6.3.3, I'll
probably start with stderr only -- I want to get this release out of the
door. We can improve the system further on. I wonder if a hardcoded
destination would make sense (like /var/log/emergency.log). But this may be
unsuitable for some cases. OTOH it is cleaner to terminate the run -- but
that leaves the system without a logger if not handled correctly.

Rainer

> 
> David Lang
> 
>   On Mon, 11 Jul 2011, Rainer Gerhards wrote:
> 
> > Date: Mon, 11 Jul 2011 23:03:12 +0200
> > From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog at lists.adiscon.com
> > Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> >
> > Out of my head. It is sysklogd legacy. Four rules, among them
> >
> > *.err /dev/console
> > Panic.* *
> >
> > Two more. Originally, it also read the system socket, which was lost some
> way around the road. I think it doesnt work for a couple of years now and
> nobody ever noticed. I just came across it due to new config. . .
> > Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:other than
> stderr, what does the current system try to do?
> >
> > David Lang
> >
> >
> > On Mon, 11 Jul 2011, Rainer Gerhards wrote:
> >
> >> Date: Mon, 11 Jul 2011 22:50:18 +0200
> >> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> >> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >> To: rsyslog at lists.adiscon.com
> >> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> >>
> >> The question is if we need more than stderr. It is surprisingly
complicated
> to do this in a clean way, as the necessary plumbing is not present.
> >>
> >> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are
> also
> >> pretty valid reasons for having the ability to turn it off.  If it's
> >> not a compile-time flag today, it should probably be made one.  If
> >> there are errors, I'd like it to fail out rather than start up anyway in
a lot of
> cases.
> >>
> >> On Mon, Jul 11, 2011 at 3:19 PM,  <david at lang.hm> wrote:
> >>> systemd is not a valid reason for removing it (systemd is linux-only
> >>> and idn't even on all linux systems)
> >>>
> >>> that being said, as long as rsyslog can spit messages out to stderr
> >>> to let someone know when there are problems starting up, I would not
> >>> expect it to do anything more, and would probably be surprised (in a
> >>> nasty way) if rsyslog processed logs and sent them somewhere I didn't
> specify.
> >>>
> >>> David Lang
> >>>
> >>>  On Mon, 11 Jul 2011, Rainer Gerhards wrote:
> >>>
> >>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
> >>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> >>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
> >>>>
> >>>> Hi all,
> >>>>
> >>>> since long, rsyslog has a so-called "emergency config system" which
> >>>> provides a very minimal config in case rsyslog can not load the
> >>>> real config. I am working on that system, which creates some
> >>>> complexity inside the code.
> >>>> Most
> >>>> importantly, I noticed that somewhere along development, that
> >>>> system notably degraded, obviously without anybody noticing. All it
> >>>> currently does is spit out startup error messages to some well
> >>>> known destinations (like the system console). It does NOT process
> >>>> the kernel log or the regular log socket.
> >>>>
> >>>> As nobody reported any problems with the system, I guess nobody
> >>>> really used it. In order to streamline the code, I am about to drop
> >>>> it from v6 (even more so because systemd handles many of the
> >>>> situations this system originally was thought for [1]). Removing
> >>>> helps getting cleaner, less complex and faster to work on code.
> >>>>
> >>>> Any objections against dropping the emergency config system? If so,
> >>>> please let know the exact reason because I need to remodel the
> >>>> system in any case and this feedback would be very useful (plus
> >>>> prove the point that there is real need for this system ;)).
> >>>>
> >>>> Thanks,
> >>>> Rainer
> >>>>
> >>>> [1]
> >>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/00286
> >>>> 2.html _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Tue Jul 12 09:16:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jul 2011 09:16:24 +0200
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <CAPjSzr91G-hgR16sMDV+nMv3dKNDnnFUdcP79_G3jLq=6zXwCQ@mail.gmail.com>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com><alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
	<CAPjSzr91G-hgR16sMDV+nMv3dKNDnnFUdcP79_G3jLq=6zXwCQ@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FD2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Aaron Wiebe
> Sent: Monday, July 11, 2011 11:37 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> 
> > I think it is probably better to fail noisily
> 
> Precisely - make it so the startup fails, and exits nonzero so RC scripts
know it
> failed.
> 
> Still, having the option to go back in the case a failure shouldn't result
in a
> non-start is a good idea.  The distros will probably want that option,
since it
> makes sure that the service comes up in the case something else is busted.

Aaron, sorry, definitely my fault, but I don't get you here. Are you saying a
fallback is good/required -- or not? As I just wrote in reply to David, I'll
probably make it stderr-and-exit today, but am very open to other
suggestions.

Thanks,
Rainer
> 
> -Aaron
> 
> On Mon, Jul 11, 2011 at 5:26 PM,  <david at lang.hm> wrote:
> > I think it is probably better to fail noisily
> >
> > thinking out loud here
> >
> > it must at least fail with errors to stderr so that someone starting
> > it manually can see that it can't read the config file.
> >
> > this should be for any config failure (i.e. one line it doesn't
> > understand), not just complete failure
> >
> > if it is able to understand the config file enough to get
> > destinations, it would probably be a good idea to spit logs to those
> > destinations reporting the failure. This is more shaky, but I think it's
> probably a good idea.
> >
> > David Lang
> >
> > ?On Mon, 11 Jul 2011, Rainer Gerhards wrote:
> >
> >> Date: Mon, 11 Jul 2011 23:03:12 +0200
> >> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> >> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >> To: rsyslog at lists.adiscon.com
> >> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> >>
> >> Out of my head. It is sysklogd legacy. Four rules, among them
> >>
> >> *.err /dev/console
> >> Panic.* *
> >>
> >> Two more. Originally, it also read the system socket, which was lost
> >> some way around the road. I think it doesnt work for a couple of
> >> years now and nobody ever noticed. I just came across it due to new
> config. . .
> >> Rainer"david at lang.hm" <david at lang.hm> hat geschrieben:other than
> >> stderr, what does the current system try to do?
> >>
> >> David Lang
> >>
> >>
> >> On Mon, 11 Jul 2011, Rainer Gerhards wrote:
> >>
> >>> Date: Mon, 11 Jul 2011 22:50:18 +0200
> >>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> >>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>> To: rsyslog at lists.adiscon.com
> >>> Subject: Re: [rsyslog] RFC: Dropping Emergency Config System
> >>>
> >>> The question is if we need more than stderr. It is surprisingly
> >>> complicated to do this in a clean way, as the necessary plumbing is
> >>> not present.
> >>>
> >>> RainerAaron Wiebe <epiphani at gmail.com> hat geschrieben:There are
> >>> also pretty valid reasons for having the ability to turn it off. ?If
> >>> it's not a compile-time flag today, it should probably be made one.
> >>> If there are errors, I'd like it to fail out rather than start up
> >>> anyway in a lot of cases.
> >>>
> >>> On Mon, Jul 11, 2011 at 3:19 PM, ?<david at lang.hm> wrote:
> >>>>
> >>>> systemd is not a valid reason for removing it (systemd is
> >>>> linux-only and idn't even on all linux systems)
> >>>>
> >>>> that being said, as long as rsyslog can spit messages out to stderr
> >>>> to let someone know when there are problems starting up, I would
> >>>> not expect it to do anything more, and would probably be surprised
> >>>> (in a nasty way) if rsyslog processed logs and sent them somewhere
> >>>> I didn't specify.
> >>>>
> >>>> David Lang
> >>>>
> >>>> ?On Mon, 11 Jul 2011, Rainer Gerhards wrote:
> >>>>
> >>>>> Date: Mon, 11 Jul 2011 17:45:58 +0200
> >>>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> >>>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>>>> Subject: [rsyslog] RFC: Dropping Emergency Config System
> >>>>>
> >>>>> Hi all,
> >>>>>
> >>>>> since long, rsyslog has a so-called "emergency config system"
> >>>>> which
> >>>>> ?provides
> >>>>> a very minimal config in case rsyslog can not load the real
> >>>>> config. I am working on that system, which creates some complexity
> >>>>> inside the code.
> >>>>> Most
> >>>>> importantly, I noticed that somewhere along development, that
> >>>>> system notably degraded, obviously without anybody noticing. All
> >>>>> it currently does is spit out startup error messages to some well
> >>>>> known destinations (like the system console). It does NOT process
> >>>>> the kernel log or the regular log socket.
> >>>>>
> >>>>> As nobody reported any problems with the system, I guess nobody
> >>>>> really used it. In order to streamline the code, I am about to
> >>>>> drop it from v6 (even more so because systemd handles many of the
> >>>>> situations this system originally was thought for [1]). Removing
> >>>>> helps getting cleaner, less complex and faster to work on code.
> >>>>>
> >>>>> Any objections against dropping the emergency config system? If
> >>>>> so, please let know the exact reason because I need to remodel the
> >>>>> system in any case and this feedback would be very useful (plus
> >>>>> prove the point that there is real need for this system ;)).
> >>>>>
> >>>>> Thanks,
> >>>>> Rainer
> >>>>>
> >>>>> [1]
> >>>>>
> >>>>> http://lists.freedesktop.org/archives/systemd-devel/2011-July/0028
> >>>>> 62.html _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com
> >>>>>
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>>
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Tue Jul 12 09:53:34 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jul 2011 09:53:34 +0200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>

The version is too old. TLS had a memory leak, which has been fixed in 5.6.4.
Note that 5.6.5 is the current stable, so using 5.6.4 calls for unneeded
problems, aka upgrade to the latest one ;)

RAiner

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mike Forbes
> Sent: Tuesday, July 12, 2011 12:07 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
> 
> Rsyslogd is using 1.4G of resident memory.
> 
> Here's my config snippets: http://paste.nothing.net.nz/806cba
> 
> There's 14 hosts sending data to this rsyslog server.
> 
> I've tried setting a ulimit
> (syslog          soft     stack           1024)
> 
>  via pam & /etc/security/limits.conf
> 
> and ulimit -a as the syslog user shows
> 
> stack size              (kbytes, -s) 1024
> 
> 
> Yet top clearly shows using 1.4G of resident memory:
> 
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 16414 syslog    20   0 2006m 1.4g 1252 S    0 72.1   1:28.83 rsyslogd -
> c4
> 
> my rsyslogd version is rsyslogd 5.6.3
> 
> Any advice is welcome.
> 
> --
> // Mike Forbes
> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mike.forbes at koordinates.com  Tue Jul 12 09:57:47 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Tue, 12 Jul 2011 19:57:47 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>
Message-ID: <CAPCA4Jq=DiSwdi8xNwM6qbmZSASdgD4jyCLq2z3eMhcj4PDqCw@mail.gmail.com>

On 12 July 2011 19:53, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
> The version is too old. TLS had a memory leak, which has been fixed in 5.6.4.
> Note that 5.6.5 is the current stable, so using 5.6.4 calls for unneeded
> problems, aka upgrade to the latest one ;)
>
> RAiner

Thanks Rainer,

You'll see in my further emails i'm getting the same results with 5.8.1

From rgerhards at hq.adiscon.com  Tue Jul 12 09:58:27 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jul 2011 09:58:27 +0200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FD5@GRFEXC.intern.adiscon.com>

Sorry, got lost in the releases myself (After the release marathon ;)). The
current version is 5.8.3. Looking at the ChangeLog, 5.8.2 addresses memory
leaks. It usually is of advantage to try the latest stable version if you
have problems of this kind.

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, July 12, 2011 9:54 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] High memory use by rsyslogd - 5.6.3
> 
> The version is too old. TLS had a memory leak, which has been fixed in
> 5.6.4.
> Note that 5.6.5 is the current stable, so using 5.6.4 calls for
> unneeded
> problems, aka upgrade to the latest one ;)
> 
> RAiner
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Mike Forbes
> > Sent: Tuesday, July 12, 2011 12:07 AM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
> >
> > Rsyslogd is using 1.4G of resident memory.
> >
> > Here's my config snippets: http://paste.nothing.net.nz/806cba
> >
> > There's 14 hosts sending data to this rsyslog server.
> >
> > I've tried setting a ulimit
> > (syslog          soft     stack           1024)
> >
> >  via pam & /etc/security/limits.conf
> >
> > and ulimit -a as the syslog user shows
> >
> > stack size              (kbytes, -s) 1024
> >
> >
> > Yet top clearly shows using 1.4G of resident memory:
> >
> >  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> > 16414 syslog    20   0 2006m 1.4g 1252 S    0 72.1   1:28.83 rsyslogd
> -
> > c4
> >
> > my rsyslogd version is rsyslogd 5.6.3
> >
> > Any advice is welcome.
> >
> > --
> > // Mike Forbes
> > GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Tue Jul 12 10:04:54 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jul 2011 10:04:54 +0200
Subject: [rsyslog] combining multiline kernel messages into a single
	message
In-Reply-To: <alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>
References: <ivf49q$98i$1@dough.gmane.org>
	<alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FD7@GRFEXC.intern.adiscon.com>

I think this would better be addressed inside the kernel. They need to define
a precise format so that one knows how to match messages. IF that's not
present, I think the results can be very surprising.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, July 11, 2011 9:30 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] combining multiline kernel messages into a
> single message
> 
> >I am just starting down this road, so please forgive my ignorance and
> >any ill-conceived assumptions.
> >
> >I want to centralize logging of multiple hosts to a single host.
> There
> >is one artifact of doing so (and in fact it's not even particular to
> >forwarding -- it seems to happen on a single node) that I want to
> >resolve and that's the intermingling of log messages in the middle of
> >what should be multiline kernel messages.  Think stack traces.
> >
> >The kernel dumps a few dozen lines onto /dev/kmsg which in reality
> >represent a single messages.  It happens with the OOM killer but
> >probably, the most common case is stack traces, which each line in the
> >trace is logged as a separate syslog line with a date and time and
> host
> >stamp, etc.
> >
> >The problem is that it's possible for other messages to be printed in
> >between these multiple lines/messages from the kernel.  I'd like to
> try
> >to atomize these kernel messages so that they are guaranteed to be
> >together in the log without other messages interspersed in them.
> >
> >Is this something that's realistic to achieve or is it fraught with to
> >many problems to be able to do reliably?
> 
> we do have some logic in the imfile to allow it to combine multiple
> line
> logs into one log entry, it has a problem right now that it doesn't
> sanitize the newlines so when you forward it things will probably get
> broken up again.
> 
> similar logic could probably be added to the kmesg input, but it is
> going
> to be tricky.
> 
> the problem that you have is how to clearly identify the beginning of a
> new message. in imfile it can do three things
> 
> 1. each line is a separate log
> 
> 2. if a line starts with whitespace, it's part of a earlier log entry
> 
> 3. log entries are separated by a blank line
> 
> now, #2 and #3 both mean that the log entry cannot be sent until the
> next
> log entry arrives (otherwise you don't know if the next data to appear
> is
> part of the log entry you are working on or not)
> 
> with messages from the kernel, if they have timestamps on the front of
> them added by the kernel, it is going to be hard to tell if it is a
> continuation or not, but without those timestampes (or if you teach
> kmesg
> input to skip them), I think that approach #2 will work.
> 
> I would suggest dumping dmesg output from a few systems and try running
> them through the imfile mode #2 and see how the results look.
> 
> I think that the kernel is pretty consistant about having every new
> message start at the beginning of a line, but I'm not sure that it
> indents
> everything (especially oops and stack trace messages)
> 
> even if it doesn't indent them, you could add logic to the kmesg input
> module to watch for, and combine the multi-line log messages, but that
> could become very messy (and change from kernel version to kernel
> version), if the simple indentation approach does not work, I would
> suggest asking on the kernel mailing list before starting any other
> work.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From sean at conman.org  Tue Jul 12 10:06:37 2011
From: sean at conman.org (Sean Conner)
Date: Tue, 12 Jul 2011 04:06:37 -0400
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FD1@GRFEXC.intern.adiscon.com>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD1@GRFEXC.intern.adiscon.com>
Message-ID: <20110712080636.GC14056@brevard.conman.org>

It was thus said that the Great Rainer Gerhards once stated:
> 
> We currently take the "use a partial config approach". The emergency config
> primarily kicked in when no actions at all were defined. For 6.3.3, I'll
> probably start with stderr only -- I want to get this release out of the
> door. We can improve the system further on. I wonder if a hardcoded
> destination would make sense (like /var/log/emergency.log). But this may be
> unsuitable for some cases. OTOH it is cleaner to terminate the run -- but
> that leaves the system without a logger if not handled correctly.

  Many years ago, back when I was in college, I was writing a program for a
friend and wanted to log all errors (this being an MS-DOS system, there was
no syslogd).  I had my program log the errors to a file, but then I thought:
what happens when the disk fills up?  

  My answer:  write to the printer [1].  That I could do easily.  Then I
thought:  what happens when the paper runs out?

  My answer:  I didn't have one.  I couldn't sprew to the "console" because
as soon as my program exited, the console would be cleared [2].  I was
stuck.

  But being in college, I thought to ask one of my instructors what to do. 
The one I asked had worked at IBM for over twenty years and what he told me
at first shocked me, but later I came to the realization that he was pretty
much correct:  "If you don't know how to handle an error, then don't check
for it."

  Now, for the issue at hand.  A minimum viable configuration (I think)
would be to open "/dev/log" (or whatever the native logging socket it for a
particular system) and to log *everything* to an equivalent of
"/dev/console".  End of story.  No worries about stderr and upon boot-up or
rsyslogd starting up, it would be noisy enough that someone might see
something.  

  -spc (Trying to cover every conceivable problem can lead to madness)

[1]	My friend ran a BBS system, so the computer and printer were always
	on.

[2]	Well, technically, the BBS system would regain control and redraw
	its screen.

From tbergfeld at hq.adiscon.com  Tue Jul 12 10:37:22 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Tue, 12 Jul 2011 10:37:22 +0200
Subject: [rsyslog] libestr 0.1.1 released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FE2@GRFEXC.intern.adiscon.com>

This release offers API enhancements. Most importantly, they are required by
rsyslog v6.3.3.

Download:
http://libestr.adiscon.com/download/libestr-0-1-1/

Changelog:
Version 0.1.1 (rgerhards), 2011-07-12
- added new API functions:
   * es_newStrFromNumber(),
   * es_str2num()
   * es_strncmp()
   * es_strncasecmp()
   * es_strContains()
   * es_strCaseContains()
   * es_tolower()

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.

From epiphani at gmail.com  Tue Jul 12 14:45:13 2011
From: epiphani at gmail.com (Aaron Wiebe)
Date: Tue, 12 Jul 2011 08:45:13 -0400
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FD2@GRFEXC.intern.adiscon.com>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
	<CAPjSzr91G-hgR16sMDV+nMv3dKNDnnFUdcP79_G3jLq=6zXwCQ@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD2@GRFEXC.intern.adiscon.com>
Message-ID: <CAPjSzr_7j5PTAU4hrER11ZwMYFp8TLMRRcawKZ+EaPOqGDAEwA@mail.gmail.com>

Sorry, I should have been a bit more clear...

On Tue, Jul 12, 2011 at 3:16 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Aaron, sorry, definitely my fault, but I don't get you here. Are you saying a
> fallback is good/required -- or not? As I just wrote in reply to David, I'll
> probably make it stderr-and-exit today, but am very open to other
> suggestions.

I like stderr-and-exit - and prefer it for my use case.  That being
said, I think it should probably be a compile-time option.  There are
use cases, like in generic out of the box distros, where a badly
configured syslog should still come up - since its somewhat essential
to discovering other possible issues on the machine.

The problem is most of us here use syslog beyond the generic host
logging that a large majority of people use it for.  I think a
compile-time option to stderr and exit out is probably the best of
both worlds.

-Aaron

From brian at interlinx.bc.ca  Tue Jul 12 16:46:07 2011
From: brian at interlinx.bc.ca (Brian J. Murrell)
Date: Tue, 12 Jul 2011 10:46:07 -0400
Subject: [rsyslog] combining multiline kernel messages into a single
	message
In-Reply-To: <alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>
References: <ivf49q$98i$1@dough.gmane.org>
	<alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>
Message-ID: <ivhmng$shb$1@dough.gmane.org>

On 11-07-11 03:29 PM, david at lang.hm wrote:
> 
> we do have some logic in the imfile to allow it to combine multiple line
> logs into one log entry,

Interesting.

> it has a problem right now that it doesn't
> sanitize the newlines so when you forward it things will probably get
> broken up again.

OK.

One thing I did notice when I sent a multiline message to rsyslogd just
using logger is that it converted the newline to a literal "#012".  Is
that what rsyslogd does with a multiline message?  Is it configurable?
Do most people really just have long syslog lines with a bunch of
"#012"s in them?

> similar logic could probably be added to the kmesg input, but it is
> going to be tricky.

Yeah.  I don't think I imagined it would always be perfect.

> the problem that you have is how to clearly identify the beginning of a
> new message.

Indeed.  I was thinking of using time as a heuristic.  Perhaps since for
a given spate of messages, like an OOM or other stack trace, the
messages all come in so tightly packed (in terms of time) I wonder if
can be reasonable to allow an amount of time to expire to mark the end
of a multiline message.

> in imfile it can do three things
> 
> 1. each line is a separate log
> 
> 2. if a line starts with whitespace, it's part of a earlier log entry

An interesting heuristic.  I wonder if we can apply this to the kernel,
or have the kernel fixed to abide by some such indicator.

> 3. log entries are separated by a blank line

Another interesting mechanism.  Although that does mean banning empty
lines elsewhere, not that I know that there are any used anywhere else.

> now, #2 and #3 both mean that the log entry cannot be sent until the
> next log entry arrives

Indeed.  With time however you only have to wait long enough.

> with messages from the kernel, if they have timestamps on the front of
> them added by the kernel, it is going to be hard to tell if it is a
> continuation or not, but without those timestampes (or if you teach
> kmesg input to skip them), I think that approach #2 will work.

Yeah, I need to look more deeply into how the kernel is pushing messages
up to userspace and what they look like in that channel.

> I would suggest dumping dmesg output from a few systems and try running
> them through the imfile mode #2 and see how the results look.

Yeah, that's not going to work just given the first sample I looked at
(sorry for the wrapping -- stupid thunderbird):

[ 7680.300076] INFO: task dbus-daemon:3613 blocked for more than 120
seconds.
[ 7680.300082] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[ 7680.300085] dbus-daemon     D f48035b0     0  3613      1 0x00000000
[ 7680.300091]  eff19dc0 00000086 f4803584 f48035b0 d9ea3af8 000006cf
f081daec c183a8c0
[ 7680.300098]  d9e2bc15 000006cf f081dae8 c183a8c0 c183a8c0 f48068c0
f081d860 f0b80ca0
[ 7680.300104]  c105685c f8eaa4e0 eff19d80 c1509dad eff19dbc f8e99444
0000b4c0 00000028
[ 7680.300111] Call Trace:
[ 7680.300122]  [<c105685c>] ? irq_exit+0x3c/0x80
[ 7680.300139]  [<c1509dad>] ? _raw_spin_lock+0xd/0x10
[ 7680.300166]  [<f8e99444>] ? rpcauth_lookup_credcache+0x134/0x1f0 [sunrpc]
[ 7680.300190]  [<f8f2921d>] ? nfs_access_get_cached+0x5d/0x160 [nfs]
[ 7680.300195]  [<c1508dc6>] __mutex_lock_slowpath+0xd6/0x140
[ 7680.300207]  [<f8f29342>] ? nfs_do_access+0x22/0xc0 [nfs]
[ 7680.300211]  [<c1508935>] mutex_lock+0x25/0x40
[ 7680.300216]  [<c1131ee7>] do_lookup+0x157/0x260
[ 7680.300219]  [<c11327bb>] link_path_walk+0x13b/0xb10
[ 7680.300224]  [<c1249ec3>] ? apparmor_file_alloc_security+0x23/0x50
[ 7680.300227]  [<c1249ec3>] ? apparmor_file_alloc_security+0x23/0x50
[ 7680.300231]  [<c1133404>] do_path_lookup+0x44/0x120
[ 7680.300234]  [<c113414f>] do_filp_open+0x18f/0x6e0
[ 7680.300238]  [<c113384a>] ? user_path_at+0x4a/0x80
[ 7680.300242]  [<c127bc12>] ? copy_to_user+0x42/0x60
[ 7680.300246]  [<c1136d50>] ? filldir+0x0/0xe0
[ 7680.300250]  [<c113fee9>] ? vfsmount_lock_local_unlock+0x19/0x20
[ 7680.300255]  [<c1126736>] do_sys_open+0x56/0x120
[ 7680.300258]  [<c112853d>] ? fput+0x1d/0x30
[ 7680.300261]  [<c112682e>] sys_open+0x2e/0x40
[ 7680.300265]  [<c150a1d4>] syscall_call+0x7/0xb

> I think that the kernel is pretty consistant about having every new
> message start at the beginning of a line, but I'm not sure that it
> indents everything (especially oops and stack trace messages)

Right.  It seems to return to position "0" for the Call Trace: line
before the stack trace as well as other places in what I would consider
a single message.

> even if it doesn't indent them, you could add logic to the kmesg input
> module to watch for,

Indeed, I had wondered about teaching the kmesg input what various
multiline messages look like too, combined with time, it might be fairly
effective.

> but that
> could become very messy (and change from kernel version to kernel
> version),

Agreed.

> if the simple indentation approach does not work, I would
> suggest asking on the kernel mailing list before starting any other work.

Yes, good idea.  I'm not holding my breath, but worth a query.  :-)

Cheers,
b.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110712/32cf083e/attachment.pgp>

From david at lang.hm  Tue Jul 12 17:24:40 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 12 Jul 2011 08:24:40 -0700 (PDT)
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <20110712080636.GC14056@brevard.conman.org>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD1@GRFEXC.intern.adiscon.com>
	<20110712080636.GC14056@brevard.conman.org>
Message-ID: <alpine.DEB.2.02.1107120823150.21382@asgard.lang.hm>

On Tue, 12 Jul 2011, Sean Conner wrote:

>  Now, for the issue at hand.  A minimum viable configuration (I think)
> would be to open "/dev/log" (or whatever the native logging socket it for a
> particular system) and to log *everything* to an equivalent of
> "/dev/console".  End of story.  No worries about stderr and upon boot-up or
> rsyslogd starting up, it would be noisy enough that someone might see
> something.

the problem with this is that doing so can prevent an admin from being 
able to fix the machine.

it's _really_ hard to figure out what's wrong if you have the screen 
scrolling as fast as it can spewing log entries at you.

David Lang

From david at lang.hm  Tue Jul 12 17:31:45 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 12 Jul 2011 08:31:45 -0700 (PDT)
Subject: [rsyslog] combining multiline kernel messages into a single
 message
In-Reply-To: <ivhmng$shb$1@dough.gmane.org>
References: <ivf49q$98i$1@dough.gmane.org>
	<alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>
	<ivhmng$shb$1@dough.gmane.org>
Message-ID: <alpine.DEB.2.02.1107120826060.21382@asgard.lang.hm>

> On 11-07-11 03:29 PM, david at lang.hm wrote:
> > 
> > we do have some logic in the imfile to allow it to combine multiple line
> > logs into one log entry,
> 
> Interesting.
> 
> > it has a problem right now that it doesn't
> > sanitize the newlines so when you forward it things will probably get
> > broken up again.
> 
> OK.
> 
> One thing I did notice when I sent a multiline message to rsyslogd just
> using logger is that it converted the newline to a literal "#012".  Is
> that what rsyslogd does with a multiline message?  Is it configurable?
> Do most people really just have long syslog lines with a bunch of
> "#012"s in them?

most people don't put multi-line messages in syslog :-) technically the 
syslog spec says that a new line starts a new message.

rsyslog defaults to escaping all control characters, so in a case where 
you do manage to get newlines inside a message, they should be escaped to 
#012 so that everything you send the message to will not break up the 
message.

> > similar logic could probably be added to the kmesg input, but it is
> > going to be tricky.
> 
> Yeah.  I don't think I imagined it would always be perfect.
> 
> > the problem that you have is how to clearly identify the beginning of a
> > new message.
> 
> Indeed.  I was thinking of using time as a heuristic.  Perhaps since for
> a given spate of messages, like an OOM or other stack trace, the
> messages all come in so tightly packed (in terms of time) I wonder if
> can be reasonable to allow an amount of time to expire to mark the end
> of a multiline message.

the problem is that you can get _lots_ of messages in a short amount of 
time, and they may not all be related. also, it's actually pretty 
expensive to lookup the time when you are receiving a message.

> > in imfile it can do three things
> > 
> > 1. each line is a separate log
> > 
> > 2. if a line starts with whitespace, it's part of a earlier log entry
> 
> An interesting heuristic.  I wonder if we can apply this to the kernel,
> or have the kernel fixed to abide by some such indicator.
> 
> > 3. log entries are separated by a blank line
> 
> Another interesting mechanism.  Although that does mean banning empty
> lines elsewhere, not that I know that there are any used anywhere else.
> 
> > now, #2 and #3 both mean that the log entry cannot be sent until the
> > next log entry arrives
> 
> Indeed.  With time however you only have to wait long enough.
> 
> > with messages from the kernel, if they have timestamps on the front of
> > them added by the kernel, it is going to be hard to tell if it is a
> > continuation or not, but without those timestampes (or if you teach
> > kmesg input to skip them), I think that approach #2 will work.
> 
> Yeah, I need to look more deeply into how the kernel is pushing messages
> up to userspace and what they look like in that channel.
> 
> > I would suggest dumping dmesg output from a few systems and try running
> > them through the imfile mode #2 and see how the results look.
> 
> Yeah, that's not going to work just given the first sample I looked at
> (sorry for the wrapping -- stupid thunderbird):
> 
> [ 7680.300076] INFO: task dbus-daemon:3613 blocked for more than 120
> seconds.
> [ 7680.300082] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> [ 7680.300085] dbus-daemon     D f48035b0     0  3613      1 0x00000000
> [ 7680.300091]  eff19dc0 00000086 f4803584 f48035b0 d9ea3af8 000006cf
> f081daec c183a8c0
> [ 7680.300098]  d9e2bc15 000006cf f081dae8 c183a8c0 c183a8c0 f48068c0
> f081d860 f0b80ca0
> [ 7680.300104]  c105685c f8eaa4e0 eff19d80 c1509dad eff19dbc f8e99444
> 0000b4c0 00000028
> [ 7680.300111] Call Trace:
> [ 7680.300122]  [<c105685c>] ? irq_exit+0x3c/0x80
> [ 7680.300139]  [<c1509dad>] ? _raw_spin_lock+0xd/0x10
> [ 7680.300166]  [<f8e99444>] ? rpcauth_lookup_credcache+0x134/0x1f0 [sunrpc]
> [ 7680.300190]  [<f8f2921d>] ? nfs_access_get_cached+0x5d/0x160 [nfs]
> [ 7680.300195]  [<c1508dc6>] __mutex_lock_slowpath+0xd6/0x140
> [ 7680.300207]  [<f8f29342>] ? nfs_do_access+0x22/0xc0 [nfs]
> [ 7680.300211]  [<c1508935>] mutex_lock+0x25/0x40
> [ 7680.300216]  [<c1131ee7>] do_lookup+0x157/0x260
> [ 7680.300219]  [<c11327bb>] link_path_walk+0x13b/0xb10
> [ 7680.300224]  [<c1249ec3>] ? apparmor_file_alloc_security+0x23/0x50
> [ 7680.300227]  [<c1249ec3>] ? apparmor_file_alloc_security+0x23/0x50
> [ 7680.300231]  [<c1133404>] do_path_lookup+0x44/0x120
> [ 7680.300234]  [<c113414f>] do_filp_open+0x18f/0x6e0
> [ 7680.300238]  [<c113384a>] ? user_path_at+0x4a/0x80
> [ 7680.300242]  [<c127bc12>] ? copy_to_user+0x42/0x60
> [ 7680.300246]  [<c1136d50>] ? filldir+0x0/0xe0
> [ 7680.300250]  [<c113fee9>] ? vfsmount_lock_local_unlock+0x19/0x20
> [ 7680.300255]  [<c1126736>] do_sys_open+0x56/0x120
> [ 7680.300258]  [<c112853d>] ? fput+0x1d/0x30
> [ 7680.300261]  [<c112682e>] sys_open+0x2e/0x40
> [ 7680.300265]  [<c150a1d4>] syscall_call+0x7/0xb
> 
> > I think that the kernel is pretty consistant about having every new
> > message start at the beginning of a line, but I'm not sure that it
> > indents everything (especially oops and stack trace messages)
> 
> Right.  It seems to return to position "0" for the Call Trace: line
> before the stack trace as well as other places in what I would consider
> a single message.

even with this being 3-4 messages, it would still be far more coherant 
than all of the lines arriving seprately (especially since separate lines 
may get reordered in transit)

> 
> > even if it doesn't indent them, you could add logic to the kmesg input
> > module to watch for,
> 
> Indeed, I had wondered about teaching the kmesg input what various
> multiline messages look like too, combined with time, it might be fairly
> effective.
> 
> > but that
> > could become very messy (and change from kernel version to kernel
> > version),
> 
> Agreed.
> 
> > if the simple indentation approach does not work, I would
> > suggest asking on the kernel mailing list before starting any other work.
> 
> Yes, good idea.  I'm not holding my breath, but worth a query.  :-)

do you want to make this query, or do you want me to?

David Lang


From brian at interlinx.bc.ca  Tue Jul 12 18:05:14 2011
From: brian at interlinx.bc.ca (Brian J. Murrell)
Date: Tue, 12 Jul 2011 12:05:14 -0400
Subject: [rsyslog] combining multiline kernel messages into a single
	message
In-Reply-To: <alpine.DEB.2.02.1107120826060.21382@asgard.lang.hm>
References: <ivf49q$98i$1@dough.gmane.org>	<alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>	<ivhmng$shb$1@dough.gmane.org>
	<alpine.DEB.2.02.1107120826060.21382@asgard.lang.hm>
Message-ID: <ivhrbs$s6s$1@dough.gmane.org>

On 11-07-12 11:31 AM, david at lang.hm wrote:
> 
> most people don't put multi-line messages in syslog :-) technically the
> syslog spec says that a new line starts a new message.

Yeah.

> rsyslog defaults to escaping all control characters, so in a case where
> you do manage to get newlines inside a message, they should be escaped
> to #012 so that everything you send the message to will not break up the
> message.

OK.  Fair enough.  I just wanted to confirm that the presence of the
"#012" was normal and what most people dealing with multiline messages
are accepting as expected behavior.

> the problem is that you can get _lots_ of messages in a short amount of
> time, and they may not all be related.

That's possible in theory, yes I agree, but keep in mind we are
isolating this problem to the kmesg input for a single machine.

> also, it's actually pretty
> expensive to lookup the time when you are receiving a message.

Well, in the case of my installation, the kernel includes the timestamp
with the message.  Perhaps that would be a requirement of the kmesg
input processor trying to infer multiline messages.

> even with this being 3-4 messages, it would still be far more coherant
> than all of the lines arriving seprately (especially since separate
> lines may get reordered in transit)

Hrm.  Is that advocacy for my thought that those couple of dozen lines
being sent by the kernel as a single message rather than a few dozen?

> do you want to make this query, or do you want me to?

Well, you probably have some good will there that will yield you more
attention than I, so if you don't mind, please go ahead.  Can you let me
know when you have so I can follow?

Thanx,
b.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110712/6eabc685/attachment.pgp>

From david at lang.hm  Tue Jul 12 20:53:22 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 12 Jul 2011 11:53:22 -0700 (PDT)
Subject: [rsyslog] combining multiline kernel messages into a single
 message
In-Reply-To: <ivhrbs$s6s$1@dough.gmane.org>
References: <ivf49q$98i$1@dough.gmane.org>
	<alpine.DEB.2.02.1107111220330.24119@asgard.lang.hm>
	<ivhmng$shb$1@dough.gmane.org>
	<alpine.DEB.2.02.1107120826060.21382@asgard.lang.hm>
	<ivhrbs$s6s$1@dough.gmane.org>
Message-ID: <alpine.DEB.2.02.1107121148570.11437@asgard.lang.hm>

> On 11-07-12 11:31 AM, david at lang.hm wrote:
> > 
> > most people don't put multi-line messages in syslog :-) technically the
> > syslog spec says that a new line starts a new message.
> 
> Yeah.
> 
> > rsyslog defaults to escaping all control characters, so in a case where
> > you do manage to get newlines inside a message, they should be escaped
> > to #012 so that everything you send the message to will not break up the
> > message.
> 
> OK.  Fair enough.  I just wanted to confirm that the presence of the
> "#012" was normal and what most people dealing with multiline messages
> are accepting as expected behavior.

'most people' won't expect it, but it is the expected behavior of rsyslog.

I'd be interested in hearing a better way of handling the newlines

> > the problem is that you can get _lots_ of messages in a short amount of
> > time, and they may not all be related.
> 
> That's possible in theory, yes I agree, but keep in mind we are
> isolating this problem to the kmesg input for a single machine.

that doesn't really help much. I've seen the system generate a LOT of 
messages in a very short amount of time. think hardware failures, or 
iptables logs for examples

> > also, it's actually pretty
> > expensive to lookup the time when you are receiving a message.
> 
> Well, in the case of my installation, the kernel includes the timestamp
> with the message.  Perhaps that would be a requirement of the kmesg
> input processor trying to infer multiline messages.

that's a boot or compile time option, whatever is done needs to work with 
or without the timestamp.

> > even with this being 3-4 messages, it would still be far more coherant
> > than all of the lines arriving seprately (especially since separate
> > lines may get reordered in transit)
> 
> Hrm.  Is that advocacy for my thought that those couple of dozen lines
> being sent by the kernel as a single message rather than a few dozen?

I'm saying that in the example you gave (where you say all those lines 
should be one messaage), I'm saying that making it several messages (with 
indented lines being part of the prior message) is still a huge win 
compared to stock.

> > do you want to make this query, or do you want me to?
> 
> Well, you probably have some good will there that will yield you more
> attention than I, so if you don't mind, please go ahead.  Can you let me
> know when you have so I can follow?

I'll try to get this out today sometime.

David Lang

From sean at conman.org  Tue Jul 12 21:36:45 2011
From: sean at conman.org (Sean Conner)
Date: Tue, 12 Jul 2011 15:36:45 -0400
Subject: [rsyslog] Illegal PRI values---how to handle?
Message-ID: <20110712193645.GD14056@brevard.conman.org>


  While not technically related to rsyslogd, I figured this might be the
best (only?) place to mention this.  I just noticed the following message
being sent by Mac OS X 10.6.8 [1] (raw value):

<198>Jul 12 15:19:13 marvin launchproxy[1243]: /usr/libexec/sshd-keygen-wrapper: Connection from: 192.168.1.10 on port: 33247 

It's fine, except that the PRI value is an invalid value---it should be
between 0 and 191 inclusive.  Broken down, this is LOG_USER8(?)/LOG_INFO and
is otherwise a nicely formatted syslog message.

Since I don't use rsyslogd [2], my first question is:  how does rsyslogd
deal with such an entry, and my second question is:  what is Apple doing
here with a non-valid PRI value?

  -spc (More curious than anything ... )

[1]	A right pain to setup forwarding of syslog messages under Mac OS X.
	At least Apple appears to have stopped overwriting local changes on
	updates now.

[2]	I wrote my own syslog daemon and this entry revealed a bug---not a
	show stopping bug, but a bug nontheless.

From sean at conman.org  Tue Jul 12 21:42:39 2011
From: sean at conman.org (Sean Conner)
Date: Tue, 12 Jul 2011 15:42:39 -0400
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <alpine.DEB.2.02.1107120823150.21382@asgard.lang.hm>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD1@GRFEXC.intern.adiscon.com>
	<20110712080636.GC14056@brevard.conman.org>
	<alpine.DEB.2.02.1107120823150.21382@asgard.lang.hm>
Message-ID: <20110712194239.GE14056@brevard.conman.org>

It was thus said that the Great david at lang.hm once stated:
> On Tue, 12 Jul 2011, Sean Conner wrote:
> 
> > Now, for the issue at hand.  A minimum viable configuration (I think)
> >would be to open "/dev/log" (or whatever the native logging socket it for a
> >particular system) and to log *everything* to an equivalent of
> >"/dev/console".  End of story.  No worries about stderr and upon boot-up or
> >rsyslogd starting up, it would be noisy enough that someone might see
> >something.
> 
> the problem with this is that doing so can prevent an admin from being 
> able to fix the machine.
> 
> it's _really_ hard to figure out what's wrong if you have the screen 
> scrolling as fast as it can spewing log entries at you.

  Yes, and if started from the startup scripts, you lose stdout or stderr
which makes it harder to diagnose.  In this case, there *are* no good
solutions so the tradeoff is:  is silence better than the console being
spammed?

  -spc


From mike.forbes at koordinates.com  Tue Jul 12 22:11:21 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Wed, 13 Jul 2011 08:11:21 +1200
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FD5@GRFEXC.intern.adiscon.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD5@GRFEXC.intern.adiscon.com>
Message-ID: <CAPCA4JqwwyYYow=zOqku+pGLKH5hJ2Kd7y_62N0uOu+Dpz=0VA@mail.gmail.com>

On 12 July 2011 19:58, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
> Sorry, got lost in the releases myself (After the release marathon ;)). The
> current version is 5.8.3. Looking at the ChangeLog, 5.8.2 addresses memory
> leaks. It usually is of advantage to try the latest stable version if you
> have problems of this kind.

Ah ha.

I was using the latest ubuntu release, that I had backported to lucid (5.8.1)

Now using 5.8.3 and memory use looks a *lot* better. (Sitting around
240M resident)


Thanks!

I'll make a suggestion to the ubuntu maintainer to move up to 5.8.3 :)


>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Tuesday, July 12, 2011 9:54 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] High memory use by rsyslogd - 5.6.3
>>
>> The version is too old. TLS had a memory leak, which has been fixed in
>> 5.6.4.
>> Note that 5.6.5 is the current stable, so using 5.6.4 calls for
>> unneeded
>> problems, aka upgrade to the latest one ;)
>>
>> RAiner
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Mike Forbes
>> > Sent: Tuesday, July 12, 2011 12:07 AM
>> > To: rsyslog at lists.adiscon.com
>> > Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>> >
>> > Rsyslogd is using 1.4G of resident memory.
>> >
>> > Here's my config snippets: http://paste.nothing.net.nz/806cba
>> >
>> > There's 14 hosts sending data to this rsyslog server.
>> >
>> > I've tried setting a ulimit
>> > (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>> >
>> > ?via pam & /etc/security/limits.conf
>> >
>> > and ulimit -a as the syslog user shows
>> >
>> > stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>> >
>> >
>> > Yet top clearly shows using 1.4G of resident memory:
>> >
>> > ?PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>> > 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd
>> -
>> > c4
>> >
>> > my rsyslogd version is rsyslogd 5.6.3
>> >
>> > Any advice is welcome.
>> >
>> > --
>> > // Mike Forbes
>> > GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>



-- 
// Mike Forbes

Operations Engineer
Koordinates Ltd.

PO Box 1604, Shortland St, Auckland 1140, New Zealand

Cell +64-21-999416
Phone +64-9-966 0433
Fax +64-9-969 0045

Web http://koordinates.com/

GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844

From david at lang.hm  Tue Jul 12 22:35:58 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 12 Jul 2011 13:35:58 -0700 (PDT)
Subject: [rsyslog] RFC: Dropping Emergency Config System
In-Reply-To: <20110712194239.GE14056@brevard.conman.org>
References: <002001cc400d$ee35b684$100013ac@intern.adiscon.com>
	<alpine.DEB.2.02.1107111423000.24119@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD1@GRFEXC.intern.adiscon.com>
	<20110712080636.GC14056@brevard.conman.org>
	<alpine.DEB.2.02.1107120823150.21382@asgard.lang.hm>
	<20110712194239.GE14056@brevard.conman.org>
Message-ID: <alpine.DEB.2.02.1107121333360.11437@asgard.lang.hm>

On Tue, 12 Jul 2011, Sean Conner wrote:

> It was thus said that the Great david at lang.hm once stated:
>> On Tue, 12 Jul 2011, Sean Conner wrote:
>>
>>> Now, for the issue at hand.  A minimum viable configuration (I think)
>>> would be to open "/dev/log" (or whatever the native logging socket it for a
>>> particular system) and to log *everything* to an equivalent of
>>> "/dev/console".  End of story.  No worries about stderr and upon boot-up or
>>> rsyslogd starting up, it would be noisy enough that someone might see
>>> something.
>>
>> the problem with this is that doing so can prevent an admin from being
>> able to fix the machine.
>>
>> it's _really_ hard to figure out what's wrong if you have the screen
>> scrolling as fast as it can spewing log entries at you.
>
>  Yes, and if started from the startup scripts, you lose stdout or stderr
> which makes it harder to diagnose.  In this case, there *are* no good
> solutions so the tradeoff is:  is silence better than the console being
> spammed?

Since the console being spammed can make it impossible to fix, yes. 
detectable silence is better.

when dieing you also return a non-zero exit code, so the startup scripts 
should report a failure. you can then login and start it manually to see 
the full output.

also, startup scripts should not blackhole the error output of a startup 
script, doing so leads to significant problems.

David LAng

From david at lang.hm  Tue Jul 12 22:39:26 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 12 Jul 2011 13:39:26 -0700 (PDT)
Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
In-Reply-To: <CAPCA4JqwwyYYow=zOqku+pGLKH5hJ2Kd7y_62N0uOu+Dpz=0VA@mail.gmail.com>
References: <CAPCA4JrwPrb9+XRcJY+k5m6UdVEJtU+PEsCYQxVpDyZ4aQvB3w@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD4@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7280FD5@GRFEXC.intern.adiscon.com>
	<CAPCA4JqwwyYYow=zOqku+pGLKH5hJ2Kd7y_62N0uOu+Dpz=0VA@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107121336180.11437@asgard.lang.hm>

On Wed, 13 Jul 2011, Mike Forbes wrote:

> On 12 July 2011 19:58, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>> Sorry, got lost in the releases myself (After the release marathon ;)). The
>> current version is 5.8.3. Looking at the ChangeLog, 5.8.2 addresses memory
>> leaks. It usually is of advantage to try the latest stable version if you
>> have problems of this kind.
>
> Ah ha.
>
> I was using the latest ubuntu release, that I had backported to lucid (5.8.1)
>
> Now using 5.8.3 and memory use looks a *lot* better. (Sitting around
> 240M resident)
>
>
> Thanks!
>
> I'll make a suggestion to the ubuntu maintainer to move up to 5.8.3 :)

for good or bad, rsyslog is a program under very rapid development. this 
is good since lots of improvements are available, but it's bad when 
combined with distro packagers who don't keep up.

David Lang

>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>> Sent: Tuesday, July 12, 2011 9:54 AM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] High memory use by rsyslogd - 5.6.3
>>>
>>> The version is too old. TLS had a memory leak, which has been fixed in
>>> 5.6.4.
>>> Note that 5.6.5 is the current stable, so using 5.6.4 calls for
>>> unneeded
>>> problems, aka upgrade to the latest one ;)
>>>
>>> RAiner
>>>
>>> > -----Original Message-----
>>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> > bounces at lists.adiscon.com] On Behalf Of Mike Forbes
>>> > Sent: Tuesday, July 12, 2011 12:07 AM
>>> > To: rsyslog at lists.adiscon.com
>>> > Subject: [rsyslog] High memory use by rsyslogd - 5.6.3
>>> >
>>> > Rsyslogd is using 1.4G of resident memory.
>>> >
>>> > Here's my config snippets: http://paste.nothing.net.nz/806cba
>>> >
>>> > There's 14 hosts sending data to this rsyslog server.
>>> >
>>> > I've tried setting a ulimit
>>> > (syslog ? ? ? ? ?soft ? ? stack ? ? ? ? ? 1024)
>>> >
>>> > ?via pam & /etc/security/limits.conf
>>> >
>>> > and ulimit -a as the syslog user shows
>>> >
>>> > stack size ? ? ? ? ? ? ?(kbytes, -s) 1024
>>> >
>>> >
>>> > Yet top clearly shows using 1.4G of resident memory:
>>> >
>>> > ?PID USER ? ? ?PR ?NI ?VIRT ?RES ?SHR S %CPU %MEM ? ?TIME+ ?COMMAND
>>> > 16414 syslog ? ?20 ? 0 2006m 1.4g 1252 S ? ?0 72.1 ? 1:28.83 rsyslogd
>>> -
>>> > c4
>>> >
>>> > my rsyslogd version is rsyslogd 5.6.3
>>> >
>>> > Any advice is welcome.
>>> >
>>> > --
>>> > // Mike Forbes
>>> > GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
>>> > _______________________________________________
>>> > rsyslog mailing list
>>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> > http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>
> -- 
> // Mike Forbes
>
> Operations Engineer
> Koordinates Ltd.
>
> PO Box 1604, Shortland St, Auckland 1140, New Zealand
>
> Cell +64-21-999416
> Phone +64-9-966 0433
> Fax +64-9-969 0045
>
> Web http://koordinates.com/
>
> GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From andreas+rsyslog at majestyk.de  Wed Jul 13 10:19:17 2011
From: andreas+rsyslog at majestyk.de (Andreas Grosse)
Date: Wed, 13 Jul 2011 10:19:17 +0200
Subject: [rsyslog] rsyslog 5.8.1 imuxsock failing to detect input format
	correctly
Message-ID: <20110713081917.GG29559@majestyk.de>

Hi, 
on my machine I am using rsyslog 5.8.1 for remote syslog, and syslog-ng
for local log processing and filtering. The syslog-ng is set up to sent
it's data to the rsyslog daemon. When I have the syslog-ng provide it's 
data to the rsyslog using a tcp connection on localhost, the data I 
receive on the remote end is fine. If I use a datagram socket for the 
communication between syslog-ng and rsyslog (using the imuxsock input 
plugin) the data format is changed into the following:

"<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - - 
exim-out[27081]: 2011-07-12 17:12:00 Start queue run: pid=27081\n"
"<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - - 
exim-out[27081]: 2011-07-12 17:12:00 End queue run: pid=27081\n"

As you can see, the imuxsock plugin adds it's own timestamps, although
the documentation says that application-provided timestamps are ignored
by default. I tried setting the $InputUnixListenSocketIgnoreMsgTimestamp
configuration value explicitly, but to no avail.

I also tried to change the message format of the syslog-ng which is
providing the logs. Using the default syslog-ng settings, the logs that
arrive at the rsyslog daemon look like this:
<22>Jul 13 09:55:00 agrdevel2 exim-out[25592]: 2011-07-13 09:55:00 Start
queue run: pid=25592
<22>Jul 13 09:55:00 agrdevel2 exim-out[25592]: 2011-07-13 09:55:00 End
queue run: pid=25592

Using the flag 'syslog-protocol' in the syslog-ng configuration, which
is supposed to have the messages formatted according to the IETF syslog
protocol standard, the messages arriving at the rsyslog daemon look like
this: 
<22>1 2011-07-13T09:56:00+02:00 agrdevel2 exim-out 25651 - [meta
sequenceId="3"] 2011-07-13 09:56:00 Start queue run: pid=25651
<22>1 2011-07-13T09:56:00+02:00 agrdevel2 exim-out 25651 - [meta
sequenceId="4"] 2011-07-13 09:56:00 End queue run: pid=25651

Unfortunately, in both cases the result is the same. It looks to me like
the imuxsock plugin fails to correctly handle the incoming message
format; date stamps are duplicated, and the fields which are supposed to
contain application name and pid only contain dashes.

Is there anything I failed to configure correctly, or is this a bug in the 
imuxsock plugin? Is there a better way to hook up a local syslog-ng to a
local rsyslog?

Best regards,
 Andreas Grosse

From rgerhards at hq.adiscon.com  Wed Jul 13 11:26:23 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jul 2011 11:26:23 +0200
Subject: [rsyslog] rsyslog 5.8.1 imuxsock failing to detect input
	formatcorrectly
In-Reply-To: <20110713081917.GG29559@majestyk.de>
References: <20110713081917.GG29559@majestyk.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FF0@GRFEXC.intern.adiscon.com>

Imuxsock expects the format that the syslog() API provides. That is the plain
old legacy format. The multi-parser interface is not supported, because there
usually is no need AND it would severely complicate "normal" processing flow.

HTH
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andreas Grosse
> Sent: Wednesday, July 13, 2011 10:19 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] rsyslog 5.8.1 imuxsock failing to detect input
> formatcorrectly
> 
> Hi,
> on my machine I am using rsyslog 5.8.1 for remote syslog, and syslog-ng for
> local log processing and filtering. The syslog-ng is set up to sent it's
data to
> the rsyslog daemon. When I have the syslog-ng provide it's data to the
> rsyslog using a tcp connection on localhost, the data I receive on the
remote
> end is fine. If I use a datagram socket for the communication between
> syslog-ng and rsyslog (using the imuxsock input
> plugin) the data format is changed into the following:
> 
> "<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - -
> exim-out[27081]: 2011-07-12 17:12:00 Start queue run: pid=27081\n"
> "<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - -
> exim-out[27081]: 2011-07-12 17:12:00 End queue run: pid=27081\n"
> 
> As you can see, the imuxsock plugin adds it's own timestamps, although the
> documentation says that application-provided timestamps are ignored by
> default. I tried setting the $InputUnixListenSocketIgnoreMsgTimestamp
> configuration value explicitly, but to no avail.
> 
> I also tried to change the message format of the syslog-ng which is
providing
> the logs. Using the default syslog-ng settings, the logs that arrive at the
> rsyslog daemon look like this:
> <22>Jul 13 09:55:00 agrdevel2 exim-out[25592]: 2011-07-13 09:55:00 Start
> queue run: pid=25592 <22>Jul 13 09:55:00 agrdevel2 exim-out[25592]: 2011-
> 07-13 09:55:00 End queue run: pid=25592
> 
> Using the flag 'syslog-protocol' in the syslog-ng configuration, which is
> supposed to have the messages formatted according to the IETF syslog
> protocol standard, the messages arriving at the rsyslog daemon look like
> this:
> <22>1 2011-07-13T09:56:00+02:00 agrdevel2 exim-out 25651 - [meta
> sequenceId="3"] 2011-07-13 09:56:00 Start queue run: pid=25651
> <22>1 2011-07-13T09:56:00+02:00 agrdevel2 exim-out 25651 - [meta
> sequenceId="4"] 2011-07-13 09:56:00 End queue run: pid=25651
> 
> Unfortunately, in both cases the result is the same. It looks to me like
the
> imuxsock plugin fails to correctly handle the incoming message format; date
> stamps are duplicated, and the fields which are supposed to contain
> application name and pid only contain dashes.
> 
> Is there anything I failed to configure correctly, or is this a bug in the
> imuxsock plugin? Is there a better way to hook up a local syslog-ng to a
local
> rsyslog?
> 
> Best regards,
>  Andreas Grosse
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From tbergfeld at hq.adiscon.com  Wed Jul 13 15:22:01 2011
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Wed, 13 Jul 2011 15:22:01 +0200
Subject: [rsyslog] rsyslog 6.3.3 (devel) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FF7@GRFEXC.intern.adiscon.com>

This is a very important milestone release. It features the new config parser
and thus provides the basis for a more intuitive config format. With 6.3.3
there are already some enhancements to the format. However, more changes will
come up with the next minor releases. For details, please check this link:

http://www.rsyslog.com/rsyslog-6-3-3-config-format-improvements/

It is worth noting that the performance of script-based filters ("if ...
then") has notable been improved. Preliminary benchmarks show an improvement
of at least a factor of three (more detailed benchmarks will be done after
the new scoped object statements have been introduced).

We would appreciate early adoption of this release. One goal in releasing it
is to see if the new parser actually is able to handle all legacy
configurations found in practice (note that the parser was written from
scratch).

ChangeLog:

http://www.rsyslog.com/changelog-for-6-3-3-v6-devel/

Download:

http://www.rsyslog.com/rsyslog-6-3-3-devel/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

--

Support
=======
Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html.


From rgerhards at hq.adiscon.com  Wed Jul 13 15:50:00 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jul 2011 15:50:00 +0200
Subject: [rsyslog] new config parser out...
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com>

Hi all,

I wanted to thank all those users who have commented during the past three
(!) years on config format questions. I have consolidated all input and hope
I have come up with a decent solution. Obviously, not everyone will like
everything, but I hope I could find a good compromise.

So far, my blog (and the rsyslog site) has the best glimpse at the new
format:

http://blog.gerhards.net/2011/07/rsyslog-633-config-format-improvements.html

It is compatible with the old legacy format, supports simple control-flow
structures (no loops, by intension) and builds heavily on name value pairs
for things like actions, inputs, global settings, ...

A real-world sample I used for parser development can be found at
http://git.adiscon.com/?p=rsyslog.git;a=blob;f=grammar/debian.new;h=4dbb5907b
f4d319799c3af392d86c4714dd2fd48;hb=HEAD

You'll also find the grammar files inside that source directory in the git
tree. It may be interesting for those used to flex/bison. My next step is to
develop the necessary code for the name/value pair objects. That requires
some more plumbing inside the core and changes to *all* loadable modules.
Sounds like a lot of work, but I still hope to get this done before the
summer break.

I have also started thinking about v7. It will contain a tree-based execution
engine, which potentially offers even higher speed and far more options for
configuration. This change is too big to make it into v6. Note that v6 will
support "if .. then" and probably "if .. then .. else" but not nesting of
these statements -- because the rule engine does not support that. The main
goal for v7 is to support nesting, including the (considerable) relevant
engine changes.

I hope the new format is useful and does not upset too many. Sorry for the
silence on the final selection. Past experience told me that there were too
many totally conflicting views of what  the format should look like. I was
deeply concerned that a broader detail discussion would have derailed this
effort again. So I used known arguments and my best judgment to create the
final format. Please all be assured that your arguments were deeply
considered and extremely useful in getting this done.

For example, a recent mailing list discussion brought up very good argument
why we actually needed to support old- and new-style config for include
files. It turned out that actually the best way to solve that problem was to
actually extend legacy format rather than completely replace it. This has the
added advantage that textbooks, courses and a myriad of Internet-HowTos do
not need to be rewritten. Besides that, I think that constructs like

mail.info /var/log/maillog

are really hard to beat in simplicity and clearness, so I think it is
valuable to have them as part of the config language.

Thanks again to everyone for helping make this happen.

Rainer

From aoz.syn at gmail.com  Wed Jul 13 16:12:41 2011
From: aoz.syn at gmail.com (RB)
Date: Wed, 13 Jul 2011 08:12:41 -0600
Subject: [rsyslog] new config parser out...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com>
Message-ID: <CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com>

On Wed, Jul 13, 2011 at 07:50, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
> I wanted to thank all those users who have commented during the past three
> (!) years on config format questions. I have consolidated all input and hope
> I have come up with a decent solution. Obviously, not everyone will like
> everything, but I hope I could find a good compromise.

You're welcome, and it's exciting to see rsyslog really come through
in the past few years.  I started watching/participating a few years
ago when it became apparent that you guys were the rising star in
actually trying to make a better syslog product, and you've not proven
me wrong.

> I hope the new format is useful and does not upset too many. Sorry for the
> silence on the final selection. Past experience told me that there were too
> many totally conflicting views of what ?the format should look like. I was
> deeply concerned that a broader detail discussion would have derailed this
> effort again. So I used known arguments and my best judgment to create the
> final format. Please all be assured that your arguments were deeply
> considered and extremely useful in getting this done.

The sooner a developer realizes they can't please everyone, the more
quickly they can move on and actually get things done.  I personally
would have preferred a wholesale abandonment of the decades-old
heritage, but can appreciate the utility to others and its simplicity.
 As Torvalds said, "... what's the point of being in charge if you
can't pick the bike shed color without holding a referendum on it? So
I'm just going all alpha-male, and just renumbering it. You'll like
it."

From aoz.syn at gmail.com  Wed Jul 13 16:29:00 2011
From: aoz.syn at gmail.com (RB)
Date: Wed, 13 Jul 2011 08:29:00 -0600
Subject: [rsyslog] new config parser out...
In-Reply-To: <CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com>
	<CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com>
Message-ID: <CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com>

On Wed, Jul 13, 2011 at 08:12, RB <aoz.syn at gmail.com> wrote:
> The sooner a developer realizes they can't please everyone, the more
> quickly they can move on and actually get things done. ?I personally
> would have preferred a wholesale abandonment of the decades-old
> heritage, but can appreciate the utility to others and its simplicity.
> ?As Torvalds said, "... what's the point of being in charge if you
> can't pick the bike shed color without holding a referendum on it? So
> I'm just going all alpha-male, and just renumbering it. You'll like
> it."

Actually, having now read the blog entry fully (and shamelessly
self-responding), I can't say I have any complaints about the new
stuff.  Every item you touch on was a concern of mine, and I frankly
can't think of any others.  The Perl-ish use of '$' to designate
variables still feels odd in a non-programming environment, but it's
not enough to complain about.

The readability quotient is indeed being significantly raised - in
particular, visual and logical blocking have taken a large leap
forward.  Congratulations, and thank you for your continued work!

From david at lang.hm  Wed Jul 13 21:26:53 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 13 Jul 2011 12:26:53 -0700 (PDT)
Subject: [rsyslog] rsyslog 5.8.1 imuxsock failing to detect input format
 correctly
In-Reply-To: <20110713081917.GG29559@majestyk.de>
References: <20110713081917.GG29559@majestyk.de>
Message-ID: <alpine.DEB.2.02.1107131223380.13969@asgard.lang.hm>

On Wed, 13 Jul 2011, Andreas Grosse wrote:

> Hi,
> on my machine I am using rsyslog 5.8.1 for remote syslog, and syslog-ng
> for local log processing and filtering. The syslog-ng is set up to sent
> it's data to the rsyslog daemon. When I have the syslog-ng provide it's
> data to the rsyslog using a tcp connection on localhost, the data I
> receive on the remote end is fine. If I use a datagram socket for the
> communication between syslog-ng and rsyslog (using the imuxsock input
> plugin) the data format is changed into the following:
>
> "<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - -
> exim-out[27081]: 2011-07-12 17:12:00 Start queue run: pid=27081\n"
> "<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - -
> exim-out[27081]: 2011-07-12 17:12:00 End queue run: pid=27081\n"
>
> As you can see, the imuxsock plugin adds it's own timestamps, although
> the documentation says that application-provided timestamps are ignored
> by default. I tried setting the $InputUnixListenSocketIgnoreMsgTimestamp
> configuration value explicitly, but to no avail.
>
> I also tried to change the message format of the syslog-ng which is
> providing the logs. Using the default syslog-ng settings, the logs that
> arrive at the rsyslog daemon look like this:
> <22>Jul 13 09:55:00 agrdevel2 exim-out[25592]: 2011-07-13 09:55:00 Start
> queue run: pid=25592
> <22>Jul 13 09:55:00 agrdevel2 exim-out[25592]: 2011-07-13 09:55:00 End
> queue run: pid=25592

this looks to me as being the same data as you show above, the only 
difference is that the first set is using the new RFC timestamp format and 
protocol tag. The only problem that I see is that it is adding the 
hostname.

> Using the flag 'syslog-protocol' in the syslog-ng configuration, which
> is supposed to have the messages formatted according to the IETF syslog
> protocol standard, the messages arriving at the rsyslog daemon look like
> this:
> <22>1 2011-07-13T09:56:00+02:00 agrdevel2 exim-out 25651 - [meta
> sequenceId="3"] 2011-07-13 09:56:00 Start queue run: pid=25651
> <22>1 2011-07-13T09:56:00+02:00 agrdevel2 exim-out 25651 - [meta
> sequenceId="4"] 2011-07-13 09:56:00 End queue run: pid=25651

Ok, this is the new RFC format.

> Unfortunately, in both cases the result is the same. It looks to me like
> the imuxsock plugin fails to correctly handle the incoming message
> format; date stamps are duplicated, and the fields which are supposed to
> contain application name and pid only contain dashes.

do you have an example of the output that's the problem?

personally, I would use networking over localhost for multiple syslog 
daemons on the same box to talk to each other.

David Lang

> Is there anything I failed to configure correctly, or is this a bug in the
> imuxsock plugin? Is there a better way to hook up a local syslog-ng to a
> local rsyslog?
>
> Best regards,
> Andreas Grosse
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From david at lang.hm  Wed Jul 13 21:35:07 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 13 Jul 2011 12:35:07 -0700 (PDT)
Subject: [rsyslog] new config parser out...
In-Reply-To: <CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com>
	<CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com>
	<CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107131232440.13969@asgard.lang.hm>

On Wed, 13 Jul 2011, RB wrote:

> On Wed, Jul 13, 2011 at 08:12, RB <aoz.syn at gmail.com> wrote:
>> The sooner a developer realizes they can't please everyone, the more
>> quickly they can move on and actually get things done. ?I personally
>> would have preferred a wholesale abandonment of the decades-old
>> heritage, but can appreciate the utility to others and its simplicity.
>> ?As Torvalds said, "... what's the point of being in charge if you
>> can't pick the bike shed color without holding a referendum on it? So
>> I'm just going all alpha-male, and just renumbering it. You'll like
>> it."
>
> Actually, having now read the blog entry fully (and shamelessly
> self-responding), I can't say I have any complaints about the new
> stuff.  Every item you touch on was a concern of mine, and I frankly
> can't think of any others.  The Perl-ish use of '$' to designate
> variables still feels odd in a non-programming environment, but it's
> not enough to complain about.
>
> The readability quotient is indeed being significantly raised - in
> particular, visual and logical blocking have taken a large leap
> forward.  Congratulations, and thank you for your continued work!

looks good to me.

as far as the perl-ish use of '$', in many ways this is becoming a 
programming environment, especially if there is going to be the ability to 
define our own properties (i.e. variables) in the future (something I 
think I remember reading)

most people won't make use of this sort of sophisticated logic, but for 
those that do it sure seems to make things much better than they were 
before.

David Lang

From rgerhards at hq.adiscon.com  Thu Jul 14 09:12:38 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jul 2011 09:12:38 +0200
Subject: [rsyslog] new config parser out...
In-Reply-To: <alpine.DEB.2.02.1107131232440.13969@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com><CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com><CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com>
	<alpine.DEB.2.02.1107131232440.13969@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7280FFE@GRFEXC.intern.adiscon.com>

RB, David,

thanks for the kind words :-). More inline below...

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, July 13, 2011 9:35 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] new config parser out...
> 
> On Wed, 13 Jul 2011, RB wrote:
> 
> > On Wed, Jul 13, 2011 at 08:12, RB <aoz.syn at gmail.com> wrote:
> >> The sooner a developer realizes they can't please everyone, the more
> >> quickly they can move on and actually get things done. ?I personally
> >> would have preferred a wholesale abandonment of the decades-old
> >> heritage, but can appreciate the utility to others and its
> simplicity.
> >> ?As Torvalds said, "... what's the point of being in charge if you
> >> can't pick the bike shed color without holding a referendum on it?
> So
> >> I'm just going all alpha-male, and just renumbering it. You'll like
> >> it."
> >
> > Actually, having now read the blog entry fully (and shamelessly
> > self-responding), I can't say I have any complaints about the new
> > stuff.  Every item you touch on was a concern of mine, and I frankly
> > can't think of any others.  The Perl-ish use of '$' to designate
> > variables still feels odd in a non-programming environment, but it's
> > not enough to complain about.
> >
> > The readability quotient is indeed being significantly raised - in
> > particular, visual and logical blocking have taken a large leap
> > forward.  Congratulations, and thank you for your continued work!
> 
> looks good to me.
> 
> as far as the perl-ish use of '$', in many ways this is becoming a
> programming environment, especially if there is going to be the ability
> to
> define our own properties (i.e. variables) in the future (something I
> think I remember reading)

This is not on my current TODO list, but definitely possible with the new
system. It would make sense to get some actual use cases, so that I can begin
to model it into the longer term todo.

Rainer
> 
> most people won't make use of this sort of sophisticated logic, but for
> those that do it sure seems to make things much better than they were
> before.
> 
> David Lang

From david at lang.hm  Thu Jul 14 09:32:21 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jul 2011 00:32:21 -0700 (PDT)
Subject: [rsyslog] new config parser out...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7280FFE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com><CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com><CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com>
	<alpine.DEB.2.02.1107131232440.13969@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7280FFE@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107140019300.3558@asgard.lang.hm>

On Thu, 14 Jul 2011, Rainer Gerhards wrote:

>> as far as the perl-ish use of '$', in many ways this is becoming a 
>> programming environment, especially if there is going to be the ability 
>> to define our own properties (i.e. variables) in the future (something 
>> I think I remember reading)
>
> This is not on my current TODO list, but definitely possible with the new
> system. It would make sense to get some actual use cases, so that I can begin
> to model it into the longer term todo.

sorry, I thought I had read that you were thinking of this.

a use case would be where you want to take some fairly complex subset of 
an existing property (probably in the message) and then make decisions on 
it, or use it in a future template (potentially using another subset of 
the value). it is a lot cleaner to be able to assign your own property 
name, and some things may not be possible without a multi-step approach


if the message matches X
   the Nth word is the server name
     write the log to a dynafile based on the basename of the server

   the server name is in the form basename[location][farm]-memberid
   where location is numeric, farm and memberid are alpha

   real-world example gaunetlet1a-c

   gauntlet means a particular type of function (in this case a firewall 
doing a particular type of job)

   location indicates what environment it's in (1 = production, 2=DR, 0=QA)

   farm further refines the location (a particular subset of production)

   memberid indicates which system in the farm this is (the 3rd member in 
this case)

this is hard enough to do if gauntlet1c is the hostname for the log entry 
you are looking for, but if you are looking for a log from another system 
referring to gauntlet1a-c in the log message (say a manager system saying 
that it's published rule updates on gauntlet1a-c) that quickly gets beyond 
what you can do today, but would be pretty easy if you can set additional 
variables/properties to hold the intermediate values.

David Lang

From rgerhards at hq.adiscon.com  Thu Jul 14 09:50:29 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jul 2011 09:50:29 +0200
Subject: [rsyslog] new config parser out...
In-Reply-To: <alpine.DEB.2.02.1107140019300.3558@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com><CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com><CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com><alpine.DEB.2.02.1107131232440.13969@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7280FFE@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1107140019300.3558@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281002@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, July 14, 2011 9:32 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] new config parser out...
> 
> On Thu, 14 Jul 2011, Rainer Gerhards wrote:
> 
> >> as far as the perl-ish use of '$', in many ways this is becoming a
> >> programming environment, especially if there is going to be the
> ability
> >> to define our own properties (i.e. variables) in the future
> (something
> >> I think I remember reading)
> >
> > This is not on my current TODO list, but definitely possible with the
> new
> > system. It would make sense to get some actual use cases, so that I
> can begin
> > to model it into the longer term todo.
> 
> sorry, I thought I had read that you were thinking of this.

No need to be sorry. Thinking I was, but not with a real conclusion, just the
overall feeling that it would be useful for some cases.

> 
> a use case would be where you want to take some fairly complex subset
> of
> an existing property (probably in the message) and then make decisions
> on
> it, or use it in a future template (potentially using another subset of
> the value). it is a lot cleaner to be able to assign your own property
> name, and some things may not be possible without a multi-step approach
> 
> 
> if the message matches X
>    the Nth word is the server name
>      write the log to a dynafile based on the basename of the server
> 
>    the server name is in the form basename[location][farm]-memberid
>    where location is numeric, farm and memberid are alpha
> 
>    real-world example gaunetlet1a-c
> 
>    gauntlet means a particular type of function (in this case a
> firewall
> doing a particular type of job)
> 
>    location indicates what environment it's in (1 = production, 2=DR,
> 0=QA)
> 
>    farm further refines the location (a particular subset of
> production)
> 
>    memberid indicates which system in the farm this is (the 3rd member
> in
> this case)
> 
> this is hard enough to do if gauntlet1c is the hostname for the log
> entry
> you are looking for, but if you are looking for a log from another
> system
> referring to gauntlet1a-c in the log message (say a manager system
> saying
> that it's published rule updates on gauntlet1a-c) that quickly gets
> beyond
> what you can do today, but would be pretty easy if you can set
> additional
> variables/properties to hold the intermediate values.

I think I get the idea. To me, it looks much like a performance optimization.
Please correct me if I am wrong, but I think the more important thing is to
have block nesting. That, compared with substr() could probably get you
somewhere. But I agree that user-settable variables would definitely a plus.

I assume a local scope limited to the message being processed is sufficient.
I also assume that there is no need to carry these variables over to an async
action. Or is it? That would complicate and slow down things.

Do you think variables without block nesting make sense (this is the question
if it is a v6 or a v7 feature)?

Rainer 
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From andreas+rsyslog at majestyk.de  Thu Jul 14 10:07:46 2011
From: andreas+rsyslog at majestyk.de (Andreas Grosse)
Date: Thu, 14 Jul 2011 10:07:46 +0200
Subject: [rsyslog] rsyslog 5.8.1 imuxsock failing to detect input	format
	correctly
In-Reply-To: <alpine.DEB.2.02.1107131223380.13969@asgard.lang.hm>
References: <20110713081917.GG29559@majestyk.de>
	<alpine.DEB.2.02.1107131223380.13969@asgard.lang.hm>
Message-ID: <20110714080746.GH29559@majestyk.de>

david at lang.hm <david at lang.hm> [13:07:11 21:27] wrote:
> On Wed, 13 Jul 2011, Andreas Grosse wrote:
>> Unfortunately, in both cases the result is the same. It looks to me like
>> the imuxsock plugin fails to correctly handle the incoming message
>> format; date stamps are duplicated, and the fields which are supposed to
>> contain application name and pid only contain dashes.
>
> do you have an example of the output that's the problem?

Hi David,
thanks for the response.
Actually, the first two log lines I wrote are an example for this, but
probably a bad example because exim has it's own timestamps in the message
part of the log line which adds to confusion:

"<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - -
exim-out[27081]: 2011-07-12 17:12:00 Start queue run: pid=27081\n"
"<22>1 2011-07-12T17:12:00+02:00 agrdevel2 agrdevel2 - - -
exim-out[27081]: 2011-07-12 17:12:00 End queue run: pid=27081\n"

The problem is that the dashes are the places where the program name and
pid should be, but imuxsock fails to correctly parse the input data,
prepends it's own date/hostname header and puts everything in the
message part.
As Rainer explained, imuxsock only supports the syslog() API format.
This put me in the right direction yesterday, and as I found out it is 
possible to tell syslog-ng to log in simple format by just using 
template("<$PRI> $MSGHDR$MSG") as an option in the syslog-ng.conf for the 
socket output. This seems to work well so far.

> personally, I would use networking over localhost for multiple syslog  
> daemons on the same box to talk to each other.
I was just trying to avoid the network overhead by using a socket. Is
there a reason why you would prefer networking for this?

Best regards,
 Andreas

From david at lang.hm  Thu Jul 14 22:41:31 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jul 2011 13:41:31 -0700 (PDT)
Subject: [rsyslog] new config parser out...
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281002@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7280FFA@GRFEXC.intern.adiscon.com><CADkMHCmY+rCXppv9d9JtyS59NnB9D1DmhT6RPCtD_p0cXTsAAA@mail.gmail.com><CADkMHCky-UUkKN8DJvXWVmJ5cP9Dwiefor281UtfZVcJ+3GF=g@mail.gmail.com><alpine.DEB.2.02.1107131232440.13969@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7280FFE@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1107140019300.3558@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7281002@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107141335430.919@asgard.lang.hm>

On Thu, 14 Jul 2011, Rainer Gerhards wrote:

> I think I get the idea. To me, it looks much like a performance optimization.
> Please correct me if I am wrong, but I think the more important thing is to
> have block nesting. That, compared with substr() could probably get you
> somewhere. But I agree that user-settable variables would definitely a plus.

how would you grab everything before a '-' in the Nth word in a message 
today? I'm not sure it's possible (you can do a regex to grab the nth 
word, but then you can't do a substring or regex on that result)

> I assume a local scope limited to the message being processed is sufficient.
> I also assume that there is no need to carry these variables over to an async
> action. Or is it? That would complicate and slow down things.

I think this would be specific to a particular message (i.e. they would be 
zero'd out when processing the next message), I could see this being used 
for an async action (basically in all the same places you would use an 
exiting property)

> Do you think variables without block nesting make sense (this is the question
> if it is a v6 or a v7 feature)?

I don't see any reason why block nesting would be needed to make this 
useful. having these have a global scope, but not be persistant to the 
next message matches every case I can imagine. I don't see any reason you 
would need to limit the scope to a block given that there is only one pass 
through things for each message.

David Lang

From marcin at mejor.pl  Fri Jul 15 15:43:47 2011
From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=)
Date: Fri, 15 Jul 2011 15:43:47 +0200
Subject: [rsyslog] Gentoo's QA notices
Message-ID: <4E204413.70709@mejor.pl>

Hello!
I've just compiled rsyslog git version and i got messages as below.
First and second seems to be present in earlier versions, third is new one.
 * QA Notice: Package has poor programming practices which may compile
 *            fine but exhibit random runtime failures.
 * lexer.l:139:1: warning: implicit declaration of function ?dbgprintf?
[-Wimplicit-function-declaration]


 * QA Notice: Package has poor programming practices which may compile
 *            fine but exhibit random runtime failures.
 * rsconf.c:342:9: warning: array subscript is above array bounds
[-Warray-bounds]


 * QA Notice: Package has poor programming practices which may compile
 *            fine but exhibit random runtime failures.
 * /usr/include/bits/string3.h:52:3: warning: call to
__builtin___memcpy_chk will always overflow destination buffer [enabled
by default]

"Buffer owerflow" doesn't sound good, is it something which can be fixed
or should it be ignored?
Regards.

From rgerhards at hq.adiscon.com  Fri Jul 15 17:04:41 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jul 2011 17:04:41 +0200
Subject: [rsyslog] Gentoo's QA notices
In-Reply-To: <4E204413.70709@mejor.pl>
References: <4E204413.70709@mejor.pl>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>

Thx! Comments inline

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw
> Sent: Friday, July 15, 2011 3:44 PM
> To: rsyslog-users
> Subject: [rsyslog] Gentoo's QA notices
> 
> Hello!
> I've just compiled rsyslog git version and i got messages as below.
> First and second seems to be present in earlier versions, third is new
> one.
>  * QA Notice: Package has poor programming practices which may compile
>  *            fine but exhibit random runtime failures.
>  * lexer.l:139:1: warning: implicit declaration of function ?dbgprintf?
> [-Wimplicit-function-declaration]

Temporay, cosmetic, will go away
> 
> 
>  * QA Notice: Package has poor programming practices which may compile
>  *            fine but exhibit random runtime failures.
>  * rsconf.c:342:9: warning: array subscript is above array bounds
> [-Warray-bounds]
> 
Actual error (!). I just fixed it.
> 
>  * QA Notice: Package has poor programming practices which may compile
>  *            fine but exhibit random runtime failures.
>  * /usr/include/bits/string3.h:52:3: warning: call to
> __builtin___memcpy_chk will always overflow destination buffer [enabled
> by default]

I have no idea where this stems from. Do you have some connect (e.g. which
code calls this lib function?).

Rainer

> 
> "Buffer owerflow" doesn't sound good, is it something which can be
> fixed
> or should it be ignored?
> Regards.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Fri Jul 15 17:11:46 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jul 2011 17:11:46 +0200
Subject: [rsyslog] Introducing Andre
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281023@GRFEXC.intern.adiscon.com>

Hi Folks,

I just wanted to let you know that Andre Lorbach, the main author of Adiscon
LogAnalyzer, is currently lending me a helping hand in rsyslog development.
That way we hope I can concentrate on pushing the new config format while we
are still being able to look at bug reports. Andre is a very experienced C
programmer and I am happy he can spare some time :-)

Rainer

From rgerhards at hq.adiscon.com  Fri Jul 15 17:19:44 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jul 2011 17:19:44 +0200
Subject: [rsyslog] Gentoo's QA notices
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
References: <4E204413.70709@mejor.pl>
	<9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281024@GRFEXC.intern.adiscon.com>

> >  * QA Notice: Package has poor programming practices which may
> compile
> >  *            fine but exhibit random runtime failures.
> >  * rsconf.c:342:9: warning: array subscript is above array bounds
> > [-Warray-bounds]
> >
> Actual error (!). I just fixed it.

BTW: do you know why gcc -Wall does not complain to me? Construct is

 char buf[1024];
 buf[1024] = '\0';

Very obviously wrong...

Rainer

From marcin at mejor.pl  Fri Jul 15 17:36:11 2011
From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=)
Date: Fri, 15 Jul 2011 17:36:11 +0200
Subject: [rsyslog] Gentoo's QA notices
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
References: <4E204413.70709@mejor.pl>
	<9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
Message-ID: <4E205E6B.3060300@mejor.pl>

W dniu 15.07.2011 17:04, Rainer Gerhards pisze:
>>  * rsconf.c:342:9: warning: array subscript is above array bounds
>> [-Warray-bounds]
> Actual error (!). I just fixed it.

Message disappear:)

>>  * QA Notice: Package has poor programming practices which may compile
>>  *            fine but exhibit random runtime failures.
>>  * /usr/include/bits/string3.h:52:3: warning: call to
>> __builtin___memcpy_chk will always overflow destination buffer [enabled
>> by default]
> 
> I have no idea where this stems from. Do you have some connect (e.g. which
> code calls this lib function?).

Probably this is bug in gcc (
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37060 ) ?

I paste warning with context:
net.c: In function ?clearAllowedSenders?:
net.c:580:7: warning: ?pCurr? may be used uninitialized in this function
[-Wuninitialized]
In file included from /usr/include/string.h:642:0,
                 from net.c:50:
In function ?memcpy?,
    inlined from ?AddAllowedSender? at net.c:724:13,
    inlined from ?addAllowedSenderLine? at net.c:872:5:
/usr/include/bits/string3.h:52:3: warning: call to
__builtin___memcpy_chk will always overflow destination buffer [enabled
by default]


From rgerhards at hq.adiscon.com  Fri Jul 15 17:39:18 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jul 2011 17:39:18 +0200
Subject: [rsyslog] Gentoo's QA notices
In-Reply-To: <4E205E6B.3060300@mejor.pl>
References: <4E204413.70709@mejor.pl><9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
	<4E205E6B.3060300@mejor.pl>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281025@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Marcin Miroslaw
> Sent: Friday, July 15, 2011 5:36 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Gentoo's QA notices
> 
> W dniu 15.07.2011 17:04, Rainer Gerhards pisze:
> >>  * rsconf.c:342:9: warning: array subscript is above array bounds
> >> [-Warray-bounds]
> > Actual error (!). I just fixed it.
> 
> Message disappear:)
> 
> >>  * QA Notice: Package has poor programming practices which may
> compile
> >>  *            fine but exhibit random runtime failures.
> >>  * /usr/include/bits/string3.h:52:3: warning: call to
> >> __builtin___memcpy_chk will always overflow destination buffer
> [enabled
> >> by default]
> >
> > I have no idea where this stems from. Do you have some connect (e.g.
> which
> > code calls this lib function?).
> 
> Probably this is bug in gcc (
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37060 ) ?
> 
> I paste warning with context:
> net.c: In function ?clearAllowedSenders?:
> net.c:580:7: warning: ?pCurr? may be used uninitialized in this
> function
> [-Wuninitialized]
> In file included from /usr/include/string.h:642:0,
>                  from net.c:50:
> In function ?memcpy?,
>     inlined from ?AddAllowedSender? at net.c:724:13,
>     inlined from ?addAllowedSenderLine? at net.c:872:5:
> /usr/include/bits/string3.h:52:3: warning: call to
> __builtin___memcpy_chk will always overflow destination buffer [enabled
> by default]

Interesting. I'll see that I dig through it. It's pretty old code, I'd bet I
haven?t touched it for at least two years. But that doesn't mean anything ;)

Rainer

From rgerhards at hq.adiscon.com  Fri Jul 15 18:06:05 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jul 2011 18:06:05 +0200
Subject: [rsyslog] Gentoo's QA notices
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
References: <4E204413.70709@mejor.pl>
	<9B6E2A8877C38245BFB15CC491A11DA7281022@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281026@GRFEXC.intern.adiscon.com>

> > Hello!
> > I've just compiled rsyslog git version and i got messages as below.
> > First and second seems to be present in earlier versions, third is
> new
> > one.
> >  * QA Notice: Package has poor programming practices which may
> compile
> >  *            fine but exhibit random runtime failures.
> >  * lexer.l:139:1: warning: implicit declaration of function
> ?dbgprintf?
> > [-Wimplicit-function-declaration]
> 
> Temporay, cosmetic, will go away

Is gone. Actually, the larger (but still cosmetic) issue was that some
printf's were left ;) See commit:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=63d357862051fd8fea4e26bd
424bdfc0e837a158

Thanks,
Rainer

From rgerhards at hq.adiscon.com  Fri Jul 15 18:28:04 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jul 2011 18:28:04 +0200
Subject: [rsyslog] FW: Re:  Gentoo's QA notices
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281027@GRFEXC.intern.adiscon.com>

All: FYI 

Alex: thanks; that could be. Will try.

Rainer

> -----Original Message-----
> From: Alex Stapleton [mailto:alexs at prol.etari.at]
> Sent: Friday, July 15, 2011 5:57 PM
> To: Rainer Gerhards; rsyslog-owner at lists.adiscon.com
> Subject: Fwd: Re: [rsyslog] Gentoo's QA notices
> 
> On 15/07/2011 16:19, Rainer Gerhards wrote:
> > BTW: do you know why gcc -Wall does not complain to me? Construct is
> 
> -Wall only enables -Warray-bounds when -O2 is also enabled?
> 
> -------- Original Message --------
> Subject: Re: [rsyslog] Gentoo's QA notices
> Date: Fri, 15 Jul 2011 17:56:37 +0200
> From: rsyslog-owner at lists.adiscon.com
> To: alexs at prol.etari.at
> 
> You are not allowed to post to this mailing list, and your message has
> been automatically rejected.  If you think that your messages are
> being rejected in error, contact the mailing list owner at
> rsyslog-owner at lists.adiscon.com.
> 


From taotetek at gmail.com  Fri Jul 15 19:57:08 2011
From: taotetek at gmail.com (Brian Knox)
Date: Fri, 15 Jul 2011 13:57:08 -0400
Subject: [rsyslog] Introducing Andre
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281023@GRFEXC.intern.adiscon.com>
References: <AcxDAYOEgYJeDDqXQAO+BDPc75tLrw==>
	<9B6E2A8877C38245BFB15CC491A11DA7281023@GRFEXC.intern.adiscon.com>
Message-ID: <CAPi=7hPSON5v9anpmmAwkSbt83brTz5oMKaVfMF53iBeCwvttA@mail.gmail.com>

Hello Andre!

Rsyslog rocks!

Great product.  We're using it heavily in some atypical ways and have been
really happ

On Fri, Jul 15, 2011 at 11:11 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com>wrote:

> Hi Folks,
>
> I just wanted to let you know that Andre Lorbach, the main author of
> Adiscon
> LogAnalyzer, is currently lending me a helping hand in rsyslog development.
> That way we hope I can concentrate on pushing the new config format while
> we
> are still being able to look at bug reports. Andre is a very experienced C
> programmer and I am happy he can spare some time :-)
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rodney.mckee at gmail.com  Mon Jul 18 05:59:55 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 18 Jul 2011 13:59:55 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <44013686-20dd-4e34-92a2-a8f49ee7baa3@wsrmckee>
Message-ID: <c79290da-cb0f-49a8-a7ad-5749981e13da@wsrmckee>

What effects the recording of milliseconds when using timereported:::date-rfc3339.
Some log entries get milliseconds and some do not:
The template:
"%TIMESTAMP% %timereported:::date-rfc3339% %timegenerated:::date-rfc3339% %msg%\n"

The output:
Jul 18 13:58:30 2011-07-18T13:58:30+10:00 2011-07-18T13:58:30.723250+10:00  test

Am I missing something.

Rgds
Rodney

From rgerhards at hq.adiscon.com  Mon Jul 18 07:24:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jul 2011 07:24:24 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <c79290da-cb0f-49a8-a7ad-5749981e13da@wsrmckee>
References: <44013686-20dd-4e34-92a2-a8f49ee7baa3@wsrmckee>
	<c79290da-cb0f-49a8-a7ad-5749981e13da@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728102B@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Monday, July 18, 2011 6:00 AM
> To: rsyslog-users
> Subject: [rsyslog] timereported:::date-rfc3339
> 
> What effects the recording of milliseconds when using timereported:::date-
> rfc3339.

This field contains what the sender told us. If the sender sent no ms, we can
not report them. Rather than to pretend "x.000000" they are there, we do not
give them. Note that for the same reason there may be sub-ms resolution, like
us, if that is what the sender reported.

Note that starting with the latest v5-devel version AND a recent Linux
kernel, we can ask the system for more precise timestamps on messages that
come in via the log socket.

Rainer

> Some log entries get milliseconds and some do not:
> The template:
> "%TIMESTAMP% %timereported:::date-rfc3339% %timegenerated:::date-
> rfc3339% %msg%\n"
> 
> The output:
> Jul 18 13:58:30 2011-07-18T13:58:30+10:00 2011-07-18T13:58:30.723250+10:00
> test
> 
> Am I missing something.
> 
> Rgds
> Rodney
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rodney.mckee at gmail.com  Mon Jul 18 07:40:49 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 18 Jul 2011 15:40:49 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728102B@GRFEXC.intern.adiscon.com>
Message-ID: <5826e996-a891-4f95-9566-311bf031f03f@wsrmckee>

Wow, Rainer, thanks for the quick response.

So on a local system some processes actually provide a high res time that rsyslog then logs as %timereported%. Did not realize this would be happening. I guess that most clients then do not provide the hi-res times and this might explain some messages having the time and most not:

Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-18T14:27:10.702529+10:00  The audit daemon is exiting.
Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-18T14:27:10.703673+10:00 audit(1310963230.693:4484770): audit_pid=0 old=1773 by auid=4294967295
Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-18T14:27:10.867738+10:00 audit(1310963230.864:4484771): auid=672 op=remove rule key=(null) list=2 res=1
Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-18T14:27:10.959443+10:00  Warning - freq is non-zero and incremental flushing not selected.
Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd pid: 4794
Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-18T14:27:10.981061+10:00 audit(1310963230.979:4484772): audit_pid=4792 old=0 by auid=672
Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-18T14:27:10.998047+10:00  af_unix plugin initialized




----- Original Message -----
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > Sent: Monday, July 18, 2011 6:00 AM
> > To: rsyslog-users
> > Subject: [rsyslog] timereported:::date-rfc3339
> > 
> > What effects the recording of milliseconds when using
> > timereported:::date-
> > rfc3339.
> 
> This field contains what the sender told us. If the sender sent no
> ms, we can
> not report them. Rather than to pretend "x.000000" they are there, we
> do not
> give them. Note that for the same reason there may be sub-ms
> resolution, like
> us, if that is what the sender reported.
> 
> Note that starting with the latest v5-devel version AND a recent
> Linux
> kernel, we can ask the system for more precise timestamps on messages
> that
> come in via the log socket.
> 
> Rainer
> 
> > Some log entries get milliseconds and some do not:
> > The template:
> > "%TIMESTAMP% %timereported:::date-rfc3339% %timegenerated:::date-
> > rfc3339% %msg%\n"
> > 
> > The output:
> > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > 2011-07-18T13:58:30.723250+10:00
> > test
> > 
> > Am I missing something.
> > 
> > Rgds
> > Rodney
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From rgerhards at hq.adiscon.com  Mon Jul 18 07:49:47 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jul 2011 07:49:47 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <5826e996-a891-4f95-9566-311bf031f03f@wsrmckee>
References: <9B6E2A8877C38245BFB15CC491A11DA728102B@GRFEXC.intern.adiscon.com>
	<5826e996-a891-4f95-9566-311bf031f03f@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728102C@GRFEXC.intern.adiscon.com>



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Monday, July 18, 2011 7:41 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] timereported:::date-rfc3339
> 
> Wow, Rainer, thanks for the quick response.
> 
> So on a local system some processes actually provide a high res time
> that rsyslog then logs as %timereported%. 

As far as the local sockets is concerned, things should be consistent. If
that's not the case, it is best if you provide a debug log -- the log samples
just show the result but now how we arrived there :)

Rainer

Did not realize this would be
> happening. I guess that most clients then do not provide the hi-res
> times and this might explain some messages having the time and most
> not:
> 
> Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> 18T14:27:10.702529+10:00  The audit daemon is exiting.
> Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> 18T14:27:10.703673+10:00 audit(1310963230.693:4484770): audit_pid=0
> old=1773 by auid=4294967295
> Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> 18T14:27:10.867738+10:00 audit(1310963230.864:4484771): auid=672
> op=remove rule key=(null) list=2 res=1
> Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> 18T14:27:10.959443+10:00  Warning - freq is non-zero and incremental
> flushing not selected.
> Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd pid: 4794
> Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> 18T14:27:10.981061+10:00 audit(1310963230.979:4484772): audit_pid=4792
> old=0 by auid=672
> Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> 18T14:27:10.998047+10:00  af_unix plugin initialized
> 
> 
> 
> 
> ----- Original Message -----
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > Sent: Monday, July 18, 2011 6:00 AM
> > > To: rsyslog-users
> > > Subject: [rsyslog] timereported:::date-rfc3339
> > >
> > > What effects the recording of milliseconds when using
> > > timereported:::date-
> > > rfc3339.
> >
> > This field contains what the sender told us. If the sender sent no
> > ms, we can
> > not report them. Rather than to pretend "x.000000" they are there, we
> > do not
> > give them. Note that for the same reason there may be sub-ms
> > resolution, like
> > us, if that is what the sender reported.
> >
> > Note that starting with the latest v5-devel version AND a recent
> > Linux
> > kernel, we can ask the system for more precise timestamps on messages
> > that
> > come in via the log socket.
> >
> > Rainer
> >
> > > Some log entries get milliseconds and some do not:
> > > The template:
> > > "%TIMESTAMP% %timereported:::date-rfc3339% %timegenerated:::date-
> > > rfc3339% %msg%\n"
> > >
> > > The output:
> > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > 2011-07-18T13:58:30.723250+10:00
> > > test
> > >
> > > Am I missing something.
> > >
> > > Rgds
> > > Rodney
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rodney.mckee at gmail.com  Mon Jul 18 08:11:24 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 18 Jul 2011 16:11:24 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728102C@GRFEXC.intern.adiscon.com>
Message-ID: <20265fb2-73cd-49b9-b54c-db8ed444ef2e@wsrmckee>

The following log has a restart of auditd and a ssh connection during the debug run.
http://pastebin.com/cRPuA1Z8

----- Original Message -----
> 
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > Sent: Monday, July 18, 2011 7:41 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > 
> > Wow, Rainer, thanks for the quick response.
> > 
> > So on a local system some processes actually provide a high res
> > time
> > that rsyslog then logs as %timereported%.
> 
> As far as the local sockets is concerned, things should be
> consistent. If
> that's not the case, it is best if you provide a debug log -- the log
> samples
> just show the result but now how we arrived there :)
> 
> Rainer
> 
> Did not realize this would be
> > happening. I guess that most clients then do not provide the hi-res
> > times and this might explain some messages having the time and most
> > not:
> > 
> > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770): audit_pid=0
> > old=1773 by auid=4294967295
> > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771): auid=672
> > op=remove rule key=(null) list=2 res=1
> > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > 18T14:27:10.959443+10:00  Warning - freq is non-zero and
> > incremental
> > flushing not selected.
> > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd pid:
> > 4794
> > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > audit_pid=4792
> > old=0 by auid=672
> > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > 
> > 
> > 
> > 
> > ----- Original Message -----
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > To: rsyslog-users
> > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > >
> > > > What effects the recording of milliseconds when using
> > > > timereported:::date-
> > > > rfc3339.
> > >
> > > This field contains what the sender told us. If the sender sent
> > > no
> > > ms, we can
> > > not report them. Rather than to pretend "x.000000" they are
> > > there, we
> > > do not
> > > give them. Note that for the same reason there may be sub-ms
> > > resolution, like
> > > us, if that is what the sender reported.
> > >
> > > Note that starting with the latest v5-devel version AND a recent
> > > Linux
> > > kernel, we can ask the system for more precise timestamps on
> > > messages
> > > that
> > > come in via the log socket.
> > >
> > > Rainer
> > >
> > > > Some log entries get milliseconds and some do not:
> > > > The template:
> > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > %timegenerated:::date-
> > > > rfc3339% %msg%\n"
> > > >
> > > > The output:
> > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > 2011-07-18T13:58:30.723250+10:00
> > > > test
> > > >
> > > > Am I missing something.
> > > >
> > > > Rgds
> > > > Rodney
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From rgerhards at hq.adiscon.com  Mon Jul 18 08:15:35 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jul 2011 08:15:35 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <20265fb2-73cd-49b9-b54c-db8ed444ef2e@wsrmckee>
References: <9B6E2A8877C38245BFB15CC491A11DA728102C@GRFEXC.intern.adiscon.com>
	<20265fb2-73cd-49b9-b54c-db8ed444ef2e@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728102E@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Monday, July 18, 2011 8:11 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] timereported:::date-rfc3339
> 
> The following log has a restart of auditd and a ssh connection during
> the debug run.
> http://pastebin.com/cRPuA1Z8

Thanks! Unfortunately, the instrumentation does not provide what I am looking
for (maybe because of an older build, maybe it's just not there...). Can you
please also write all messages to a file with RSYSLOG_DebugFormat and post
that file.

With 5.8.0, you should probably never see hires, so I am a bit puzzled. Maybe
auditd does some "interesting" things to the log socket. Note that rsyslog
expects syslog() API format, but older versions (like 5.8.0) did not enforce
that.

Rainer
> 
> ----- Original Message -----
> >
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > Sent: Monday, July 18, 2011 7:41 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > >
> > > Wow, Rainer, thanks for the quick response.
> > >
> > > So on a local system some processes actually provide a high res
> > > time
> > > that rsyslog then logs as %timereported%.
> >
> > As far as the local sockets is concerned, things should be
> > consistent. If
> > that's not the case, it is best if you provide a debug log -- the log
> > samples
> > just show the result but now how we arrived there :)
> >
> > Rainer
> >
> > Did not realize this would be
> > > happening. I guess that most clients then do not provide the hi-res
> > > times and this might explain some messages having the time and most
> > > not:
> > >
> > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770): audit_pid=0
> > > old=1773 by auid=4294967295
> > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771): auid=672
> > > op=remove rule key=(null) list=2 res=1
> > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > 18T14:27:10.959443+10:00  Warning - freq is non-zero and
> > > incremental
> > > flushing not selected.
> > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd pid:
> > > 4794
> > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > audit_pid=4792
> > > old=0 by auid=672
> > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > To: rsyslog-users
> > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > >
> > > > > What effects the recording of milliseconds when using
> > > > > timereported:::date-
> > > > > rfc3339.
> > > >
> > > > This field contains what the sender told us. If the sender sent
> > > > no
> > > > ms, we can
> > > > not report them. Rather than to pretend "x.000000" they are
> > > > there, we
> > > > do not
> > > > give them. Note that for the same reason there may be sub-ms
> > > > resolution, like
> > > > us, if that is what the sender reported.
> > > >
> > > > Note that starting with the latest v5-devel version AND a recent
> > > > Linux
> > > > kernel, we can ask the system for more precise timestamps on
> > > > messages
> > > > that
> > > > come in via the log socket.
> > > >
> > > > Rainer
> > > >
> > > > > Some log entries get milliseconds and some do not:
> > > > > The template:
> > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > %timegenerated:::date-
> > > > > rfc3339% %msg%\n"
> > > > >
> > > > > The output:
> > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > test
> > > > >
> > > > > Am I missing something.
> > > > >
> > > > > Rgds
> > > > > Rodney
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rodney.mckee at gmail.com  Mon Jul 18 08:25:39 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 18 Jul 2011 16:25:39 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728102E@GRFEXC.intern.adiscon.com>
Message-ID: <ed96c389-989e-477e-a73e-8c801a38bc93@wsrmckee>

http://pastebin.com/TzPVzknt
The 2 line I have previously seen with hi-res times are 15 and 23

----- Original Message -----
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > Sent: Monday, July 18, 2011 8:11 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > 
> > The following log has a restart of auditd and a ssh connection
> > during
> > the debug run.
> > http://pastebin.com/cRPuA1Z8
> 
> Thanks! Unfortunately, the instrumentation does not provide what I am
> looking
> for (maybe because of an older build, maybe it's just not there...).
> Can you
> please also write all messages to a file with RSYSLOG_DebugFormat and
> post
> that file.
> 
> With 5.8.0, you should probably never see hires, so I am a bit
> puzzled. Maybe
> auditd does some "interesting" things to the log socket. Note that
> rsyslog
> expects syslog() API format, but older versions (like 5.8.0) did not
> enforce
> that.
> 
> Rainer
> > 
> > ----- Original Message -----
> > >
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > >
> > > > Wow, Rainer, thanks for the quick response.
> > > >
> > > > So on a local system some processes actually provide a high res
> > > > time
> > > > that rsyslog then logs as %timereported%.
> > >
> > > As far as the local sockets is concerned, things should be
> > > consistent. If
> > > that's not the case, it is best if you provide a debug log -- the
> > > log
> > > samples
> > > just show the result but now how we arrived there :)
> > >
> > > Rainer
> > >
> > > Did not realize this would be
> > > > happening. I guess that most clients then do not provide the
> > > > hi-res
> > > > times and this might explain some messages having the time and
> > > > most
> > > > not:
> > > >
> > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> > > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770):
> > > > audit_pid=0
> > > > old=1773 by auid=4294967295
> > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> > > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771):
> > > > auid=672
> > > > op=remove rule key=(null) list=2 res=1
> > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > 18T14:27:10.959443+10:00  Warning - freq is non-zero and
> > > > incremental
> > > > flushing not selected.
> > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd
> > > > pid:
> > > > 4794
> > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> > > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > > audit_pid=4792
> > > > old=0 by auid=672
> > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > > >
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > > -----Original Message-----
> > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > To: rsyslog-users
> > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > >
> > > > > > What effects the recording of milliseconds when using
> > > > > > timereported:::date-
> > > > > > rfc3339.
> > > > >
> > > > > This field contains what the sender told us. If the sender
> > > > > sent
> > > > > no
> > > > > ms, we can
> > > > > not report them. Rather than to pretend "x.000000" they are
> > > > > there, we
> > > > > do not
> > > > > give them. Note that for the same reason there may be sub-ms
> > > > > resolution, like
> > > > > us, if that is what the sender reported.
> > > > >
> > > > > Note that starting with the latest v5-devel version AND a
> > > > > recent
> > > > > Linux
> > > > > kernel, we can ask the system for more precise timestamps on
> > > > > messages
> > > > > that
> > > > > come in via the log socket.
> > > > >
> > > > > Rainer
> > > > >
> > > > > > Some log entries get milliseconds and some do not:
> > > > > > The template:
> > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > %timegenerated:::date-
> > > > > > rfc3339% %msg%\n"
> > > > > >
> > > > > > The output:
> > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > test
> > > > > >
> > > > > > Am I missing something.
> > > > > >
> > > > > > Rgds
> > > > > > Rodney
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From rgerhards at hq.adiscon.com  Mon Jul 18 08:31:05 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jul 2011 08:31:05 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <ed96c389-989e-477e-a73e-8c801a38bc93@wsrmckee>
References: <9B6E2A8877C38245BFB15CC491A11DA728102E@GRFEXC.intern.adiscon.com>
	<ed96c389-989e-477e-a73e-8c801a38bc93@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728102F@GRFEXC.intern.adiscon.com>

Ahh, I now see it. Look at the raw messages. Lines 7 and 31 are correctly
formatted. Lines 15 and 23 have invalid format. With invalid format,
interpretation is not guaranteed. Looks like 5.8.0 in that case uses the
timestamp of message reception. I suggest to use the current stable, I think
it will work somewhat different. Bottom line is that auditd should emit the
proper format.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Monday, July 18, 2011 8:26 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] timereported:::date-rfc3339
> 
> http://pastebin.com/TzPVzknt
> The 2 line I have previously seen with hi-res times are 15 and 23
> 
> ----- Original Message -----
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > Sent: Monday, July 18, 2011 8:11 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > >
> > > The following log has a restart of auditd and a ssh connection
> > > during
> > > the debug run.
> > > http://pastebin.com/cRPuA1Z8
> >
> > Thanks! Unfortunately, the instrumentation does not provide what I am
> > looking
> > for (maybe because of an older build, maybe it's just not there...).
> > Can you
> > please also write all messages to a file with RSYSLOG_DebugFormat and
> > post
> > that file.
> >
> > With 5.8.0, you should probably never see hires, so I am a bit
> > puzzled. Maybe
> > auditd does some "interesting" things to the log socket. Note that
> > rsyslog
> > expects syslog() API format, but older versions (like 5.8.0) did not
> > enforce
> > that.
> >
> > Rainer
> > >
> > > ----- Original Message -----
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > To: rsyslog-users
> > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > >
> > > > > Wow, Rainer, thanks for the quick response.
> > > > >
> > > > > So on a local system some processes actually provide a high res
> > > > > time
> > > > > that rsyslog then logs as %timereported%.
> > > >
> > > > As far as the local sockets is concerned, things should be
> > > > consistent. If
> > > > that's not the case, it is best if you provide a debug log -- the
> > > > log
> > > > samples
> > > > just show the result but now how we arrived there :)
> > > >
> > > > Rainer
> > > >
> > > > Did not realize this would be
> > > > > happening. I guess that most clients then do not provide the
> > > > > hi-res
> > > > > times and this might explain some messages having the time and
> > > > > most
> > > > > not:
> > > > >
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> > > > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770):
> > > > > audit_pid=0
> > > > > old=1773 by auid=4294967295
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> > > > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771):
> > > > > auid=672
> > > > > op=remove rule key=(null) list=2 res=1
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > 18T14:27:10.959443+10:00  Warning - freq is non-zero and
> > > > > incremental
> > > > > flushing not selected.
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd
> > > > > pid:
> > > > > 4794
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> > > > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > > > audit_pid=4792
> > > > > old=0 by auid=672
> > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > > > -----Original Message-----
> > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > To: rsyslog-users
> > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > >
> > > > > > > What effects the recording of milliseconds when using
> > > > > > > timereported:::date-
> > > > > > > rfc3339.
> > > > > >
> > > > > > This field contains what the sender told us. If the sender
> > > > > > sent
> > > > > > no
> > > > > > ms, we can
> > > > > > not report them. Rather than to pretend "x.000000" they are
> > > > > > there, we
> > > > > > do not
> > > > > > give them. Note that for the same reason there may be sub-ms
> > > > > > resolution, like
> > > > > > us, if that is what the sender reported.
> > > > > >
> > > > > > Note that starting with the latest v5-devel version AND a
> > > > > > recent
> > > > > > Linux
> > > > > > kernel, we can ask the system for more precise timestamps on
> > > > > > messages
> > > > > > that
> > > > > > come in via the log socket.
> > > > > >
> > > > > > Rainer
> > > > > >
> > > > > > > Some log entries get milliseconds and some do not:
> > > > > > > The template:
> > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > %timegenerated:::date-
> > > > > > > rfc3339% %msg%\n"
> > > > > > >
> > > > > > > The output:
> > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > test
> > > > > > >
> > > > > > > Am I missing something.
> > > > > > >
> > > > > > > Rgds
> > > > > > > Rodney
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rodney.mckee at gmail.com  Mon Jul 18 08:43:34 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 18 Jul 2011 16:43:34 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728102F@GRFEXC.intern.adiscon.com>
Message-ID: <09bf239f-3af7-44b4-8d5e-69075f4abbb3@wsrmckee>

Rainer,

Thanks for the help, yes it would appear that auditd is dropping it's funky format into syslog. Out of interest restarting rsyslog also triggers the following messages with hi-res times:

Jul 18 16:40:50 2011-07-18T16:40:50.538649+10:00 2011-07-18T16:40:50.538649+10:00 Kernel logging (proc) stopped.
Jul 18 16:40:50 2011-07-18T16:40:50.664222+10:00 2011-07-18T16:40:50.664222+10:00 imklog 5.8.0, log source = /proc/kmsg started.

I'm guessing I'll disable the hi-res for now.


----- Original Message -----
> Ahh, I now see it. Look at the raw messages. Lines 7 and 31 are
> correctly
> formatted. Lines 15 and 23 have invalid format. With invalid format,
> interpretation is not guaranteed. Looks like 5.8.0 in that case uses
> the
> timestamp of message reception. I suggest to use the current stable,
> I think
> it will work somewhat different. Bottom line is that auditd should
> emit the
> proper format.
> 
> Rainer
> 
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > Sent: Monday, July 18, 2011 8:26 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > 
> > http://pastebin.com/TzPVzknt
> > The 2 line I have previously seen with hi-res times are 15 and 23
> > 
> > ----- Original Message -----
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > >
> > > > The following log has a restart of auditd and a ssh connection
> > > > during
> > > > the debug run.
> > > > http://pastebin.com/cRPuA1Z8
> > >
> > > Thanks! Unfortunately, the instrumentation does not provide what
> > > I am
> > > looking
> > > for (maybe because of an older build, maybe it's just not
> > > there...).
> > > Can you
> > > please also write all messages to a file with RSYSLOG_DebugFormat
> > > and
> > > post
> > > that file.
> > >
> > > With 5.8.0, you should probably never see hires, so I am a bit
> > > puzzled. Maybe
> > > auditd does some "interesting" things to the log socket. Note
> > > that
> > > rsyslog
> > > expects syslog() API format, but older versions (like 5.8.0) did
> > > not
> > > enforce
> > > that.
> > >
> > > Rainer
> > > >
> > > > ----- Original Message -----
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > To: rsyslog-users
> > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > >
> > > > > > Wow, Rainer, thanks for the quick response.
> > > > > >
> > > > > > So on a local system some processes actually provide a high
> > > > > > res
> > > > > > time
> > > > > > that rsyslog then logs as %timereported%.
> > > > >
> > > > > As far as the local sockets is concerned, things should be
> > > > > consistent. If
> > > > > that's not the case, it is best if you provide a debug log --
> > > > > the
> > > > > log
> > > > > samples
> > > > > just show the result but now how we arrived there :)
> > > > >
> > > > > Rainer
> > > > >
> > > > > Did not realize this would be
> > > > > > happening. I guess that most clients then do not provide
> > > > > > the
> > > > > > hi-res
> > > > > > times and this might explain some messages having the time
> > > > > > and
> > > > > > most
> > > > > > not:
> > > > > >
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> > > > > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770):
> > > > > > audit_pid=0
> > > > > > old=1773 by auid=4294967295
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> > > > > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771):
> > > > > > auid=672
> > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > 18T14:27:10.959443+10:00  Warning - freq is non-zero and
> > > > > > incremental
> > > > > > flushing not selected.
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd
> > > > > > pid:
> > > > > > 4794
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> > > > > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > > > > audit_pid=4792
> > > > > > old=0 by auid=672
> > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > > > -----Original Message-----
> > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > [mailto:rsyslog-
> > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > To: rsyslog-users
> > > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > > >
> > > > > > > > What effects the recording of milliseconds when using
> > > > > > > > timereported:::date-
> > > > > > > > rfc3339.
> > > > > > >
> > > > > > > This field contains what the sender told us. If the
> > > > > > > sender
> > > > > > > sent
> > > > > > > no
> > > > > > > ms, we can
> > > > > > > not report them. Rather than to pretend "x.000000" they
> > > > > > > are
> > > > > > > there, we
> > > > > > > do not
> > > > > > > give them. Note that for the same reason there may be
> > > > > > > sub-ms
> > > > > > > resolution, like
> > > > > > > us, if that is what the sender reported.
> > > > > > >
> > > > > > > Note that starting with the latest v5-devel version AND a
> > > > > > > recent
> > > > > > > Linux
> > > > > > > kernel, we can ask the system for more precise timestamps
> > > > > > > on
> > > > > > > messages
> > > > > > > that
> > > > > > > come in via the log socket.
> > > > > > >
> > > > > > > Rainer
> > > > > > >
> > > > > > > > Some log entries get milliseconds and some do not:
> > > > > > > > The template:
> > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > %timegenerated:::date-
> > > > > > > > rfc3339% %msg%\n"
> > > > > > > >
> > > > > > > > The output:
> > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > test
> > > > > > > >
> > > > > > > > Am I missing something.
> > > > > > > >
> > > > > > > > Rgds
> > > > > > > > Rodney
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > >
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From rgerhards at hq.adiscon.com  Mon Jul 18 08:46:33 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jul 2011 08:46:33 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <09bf239f-3af7-44b4-8d5e-69075f4abbb3@wsrmckee>
References: <9B6E2A8877C38245BFB15CC491A11DA728102F@GRFEXC.intern.adiscon.com>
	<09bf239f-3af7-44b4-8d5e-69075f4abbb3@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281030@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Monday, July 18, 2011 8:44 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] timereported:::date-rfc3339
> 
> Rainer,
> 
> Thanks for the help, yes it would appear that auditd is dropping it's
> funky format into syslog. Out of interest restarting rsyslog also
> triggers the following messages with hi-res times:
> 
> Jul 18 16:40:50 2011-07-18T16:40:50.538649+10:00 2011-07-
> 18T16:40:50.538649+10:00 Kernel logging (proc) stopped.
> Jul 18 16:40:50 2011-07-18T16:40:50.664222+10:00 2011-07-
> 18T16:40:50.664222+10:00 imklog 5.8.0, log source = /proc/kmsg started.

Yes, of course. You always get the best we have :-)
> 
> I'm guessing I'll disable the hi-res for now.

Can you elaborate why? That would be very interesting to me. I really think
it is a shame that we have hi-res format since 5+ years, but everybody turns
it off...

Rainer
> 
> 
> ----- Original Message -----
> > Ahh, I now see it. Look at the raw messages. Lines 7 and 31 are
> > correctly
> > formatted. Lines 15 and 23 have invalid format. With invalid format,
> > interpretation is not guaranteed. Looks like 5.8.0 in that case uses
> > the
> > timestamp of message reception. I suggest to use the current stable,
> > I think
> > it will work somewhat different. Bottom line is that auditd should
> > emit the
> > proper format.
> >
> > Rainer
> >
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > Sent: Monday, July 18, 2011 8:26 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > >
> > > http://pastebin.com/TzPVzknt
> > > The 2 line I have previously seen with hi-res times are 15 and 23
> > >
> > > ----- Original Message -----
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > > To: rsyslog-users
> > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > >
> > > > > The following log has a restart of auditd and a ssh connection
> > > > > during
> > > > > the debug run.
> > > > > http://pastebin.com/cRPuA1Z8
> > > >
> > > > Thanks! Unfortunately, the instrumentation does not provide what
> > > > I am
> > > > looking
> > > > for (maybe because of an older build, maybe it's just not
> > > > there...).
> > > > Can you
> > > > please also write all messages to a file with RSYSLOG_DebugFormat
> > > > and
> > > > post
> > > > that file.
> > > >
> > > > With 5.8.0, you should probably never see hires, so I am a bit
> > > > puzzled. Maybe
> > > > auditd does some "interesting" things to the log socket. Note
> > > > that
> > > > rsyslog
> > > > expects syslog() API format, but older versions (like 5.8.0) did
> > > > not
> > > > enforce
> > > > that.
> > > >
> > > > Rainer
> > > > >
> > > > > ----- Original Message -----
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > > To: rsyslog-users
> > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > >
> > > > > > > Wow, Rainer, thanks for the quick response.
> > > > > > >
> > > > > > > So on a local system some processes actually provide a high
> > > > > > > res
> > > > > > > time
> > > > > > > that rsyslog then logs as %timereported%.
> > > > > >
> > > > > > As far as the local sockets is concerned, things should be
> > > > > > consistent. If
> > > > > > that's not the case, it is best if you provide a debug log --
> > > > > > the
> > > > > > log
> > > > > > samples
> > > > > > just show the result but now how we arrived there :)
> > > > > >
> > > > > > Rainer
> > > > > >
> > > > > > Did not realize this would be
> > > > > > > happening. I guess that most clients then do not provide
> > > > > > > the
> > > > > > > hi-res
> > > > > > > times and this might explain some messages having the time
> > > > > > > and
> > > > > > > most
> > > > > > > not:
> > > > > > >
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00 2011-07-
> > > > > > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770):
> > > > > > > audit_pid=0
> > > > > > > old=1773 by auid=4294967295
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00 2011-07-
> > > > > > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771):
> > > > > > > auid=672
> > > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > 18T14:27:10.959443+10:00  Warning - freq is non-zero and
> > > > > > > incremental
> > > > > > > flushing not selected.
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > 18T14:27:10.978467+10:00  Started dispatcher: /sbin/audispd
> > > > > > > pid:
> > > > > > > 4794
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00 2011-07-
> > > > > > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > > > > > audit_pid=4792
> > > > > > > old=0 by auid=672
> > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > [mailto:rsyslog-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > > To: rsyslog-users
> > > > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > > > >
> > > > > > > > > What effects the recording of milliseconds when using
> > > > > > > > > timereported:::date-
> > > > > > > > > rfc3339.
> > > > > > > >
> > > > > > > > This field contains what the sender told us. If the
> > > > > > > > sender
> > > > > > > > sent
> > > > > > > > no
> > > > > > > > ms, we can
> > > > > > > > not report them. Rather than to pretend "x.000000" they
> > > > > > > > are
> > > > > > > > there, we
> > > > > > > > do not
> > > > > > > > give them. Note that for the same reason there may be
> > > > > > > > sub-ms
> > > > > > > > resolution, like
> > > > > > > > us, if that is what the sender reported.
> > > > > > > >
> > > > > > > > Note that starting with the latest v5-devel version AND a
> > > > > > > > recent
> > > > > > > > Linux
> > > > > > > > kernel, we can ask the system for more precise timestamps
> > > > > > > > on
> > > > > > > > messages
> > > > > > > > that
> > > > > > > > come in via the log socket.
> > > > > > > >
> > > > > > > > Rainer
> > > > > > > >
> > > > > > > > > Some log entries get milliseconds and some do not:
> > > > > > > > > The template:
> > > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > > %timegenerated:::date-
> > > > > > > > > rfc3339% %msg%\n"
> > > > > > > > >
> > > > > > > > > The output:
> > > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > > test
> > > > > > > > >
> > > > > > > > > Am I missing something.
> > > > > > > > >
> > > > > > > > > Rgds
> > > > > > > > > Rodney
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > >
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rodney.mckee at gmail.com  Mon Jul 18 09:05:45 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 18 Jul 2011 17:05:45 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281030@GRFEXC.intern.adiscon.com>
Message-ID: <6c13128b-e584-42c4-8bd0-08ed54a2c4db@wsrmckee>



----- Original Message -----
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > Sent: Monday, July 18, 2011 8:44 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > 
> > Rainer,
> > 
> > Thanks for the help, yes it would appear that auditd is dropping
> > it's
> > funky format into syslog. Out of interest restarting rsyslog also
> > triggers the following messages with hi-res times:
> > 
> > Jul 18 16:40:50 2011-07-18T16:40:50.538649+10:00 2011-07-
> > 18T16:40:50.538649+10:00 Kernel logging (proc) stopped.
> > Jul 18 16:40:50 2011-07-18T16:40:50.664222+10:00 2011-07-
> > 18T16:40:50.664222+10:00 imklog 5.8.0, log source = /proc/kmsg
> > started.
> 
> Yes, of course. You always get the best we have :-)
> > 
> > I'm guessing I'll disable the hi-res for now.
> 
> Can you elaborate why? That would be very interesting to me. I really
> think
> it is a shame that we have hi-res format since 5+ years, but
> everybody turns
> it off...
> 

It appears that their are a limited number of clients that I'm seeing logging with hi-res so to have it enabled for only a few services logging in hi-res would appear pointless.
If I could enable it in our environment and have all logging hi-res I will certainly be doing it, that's why we have been trying.
The java application that we run WILL certainly be logging in hi-res and this will be centralized using log4j and rsyslog with the JSON module.

Out of interest we are also monitoring the rsyslog stats using pcp and I suspect we will have some modules/details heading your way once we have completed implementation and testing.

> Rainer
> > 
> > 
> > ----- Original Message -----
> > > Ahh, I now see it. Look at the raw messages. Lines 7 and 31 are
> > > correctly
> > > formatted. Lines 15 and 23 have invalid format. With invalid
> > > format,
> > > interpretation is not guaranteed. Looks like 5.8.0 in that case
> > > uses
> > > the
> > > timestamp of message reception. I suggest to use the current
> > > stable,
> > > I think
> > > it will work somewhat different. Bottom line is that auditd
> > > should
> > > emit the
> > > proper format.
> > >
> > > Rainer
> > >
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > Sent: Monday, July 18, 2011 8:26 AM
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > >
> > > > http://pastebin.com/TzPVzknt
> > > > The 2 line I have previously seen with hi-res times are 15 and
> > > > 23
> > > >
> > > > ----- Original Message -----
> > > > > > -----Original Message-----
> > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > > > To: rsyslog-users
> > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > >
> > > > > > The following log has a restart of auditd and a ssh
> > > > > > connection
> > > > > > during
> > > > > > the debug run.
> > > > > > http://pastebin.com/cRPuA1Z8
> > > > >
> > > > > Thanks! Unfortunately, the instrumentation does not provide
> > > > > what
> > > > > I am
> > > > > looking
> > > > > for (maybe because of an older build, maybe it's just not
> > > > > there...).
> > > > > Can you
> > > > > please also write all messages to a file with
> > > > > RSYSLOG_DebugFormat
> > > > > and
> > > > > post
> > > > > that file.
> > > > >
> > > > > With 5.8.0, you should probably never see hires, so I am a
> > > > > bit
> > > > > puzzled. Maybe
> > > > > auditd does some "interesting" things to the log socket. Note
> > > > > that
> > > > > rsyslog
> > > > > expects syslog() API format, but older versions (like 5.8.0)
> > > > > did
> > > > > not
> > > > > enforce
> > > > > that.
> > > > >
> > > > > Rainer
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > [mailto:rsyslog-
> > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > > > To: rsyslog-users
> > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > >
> > > > > > > > Wow, Rainer, thanks for the quick response.
> > > > > > > >
> > > > > > > > So on a local system some processes actually provide a
> > > > > > > > high
> > > > > > > > res
> > > > > > > > time
> > > > > > > > that rsyslog then logs as %timereported%.
> > > > > > >
> > > > > > > As far as the local sockets is concerned, things should
> > > > > > > be
> > > > > > > consistent. If
> > > > > > > that's not the case, it is best if you provide a debug
> > > > > > > log --
> > > > > > > the
> > > > > > > log
> > > > > > > samples
> > > > > > > just show the result but now how we arrived there :)
> > > > > > >
> > > > > > > Rainer
> > > > > > >
> > > > > > > Did not realize this would be
> > > > > > > > happening. I guess that most clients then do not
> > > > > > > > provide
> > > > > > > > the
> > > > > > > > hi-res
> > > > > > > > times and this might explain some messages having the
> > > > > > > > time
> > > > > > > > and
> > > > > > > > most
> > > > > > > > not:
> > > > > > > >
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00
> > > > > > > > 2011-07-
> > > > > > > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770):
> > > > > > > > audit_pid=0
> > > > > > > > old=1773 by auid=4294967295
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00
> > > > > > > > 2011-07-
> > > > > > > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771):
> > > > > > > > auid=672
> > > > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > 18T14:27:10.959443+10:00  Warning - freq is non-zero
> > > > > > > > and
> > > > > > > > incremental
> > > > > > > > flushing not selected.
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > 18T14:27:10.978467+10:00  Started dispatcher:
> > > > > > > > /sbin/audispd
> > > > > > > > pid:
> > > > > > > > 4794
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00
> > > > > > > > 2011-07-
> > > > > > > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > > > > > > audit_pid=4792
> > > > > > > > old=0 by auid=672
> > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > ----- Original Message -----
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > McKee
> > > > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > > > To: rsyslog-users
> > > > > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > >
> > > > > > > > > > What effects the recording of milliseconds when
> > > > > > > > > > using
> > > > > > > > > > timereported:::date-
> > > > > > > > > > rfc3339.
> > > > > > > > >
> > > > > > > > > This field contains what the sender told us. If the
> > > > > > > > > sender
> > > > > > > > > sent
> > > > > > > > > no
> > > > > > > > > ms, we can
> > > > > > > > > not report them. Rather than to pretend "x.000000"
> > > > > > > > > they
> > > > > > > > > are
> > > > > > > > > there, we
> > > > > > > > > do not
> > > > > > > > > give them. Note that for the same reason there may be
> > > > > > > > > sub-ms
> > > > > > > > > resolution, like
> > > > > > > > > us, if that is what the sender reported.
> > > > > > > > >
> > > > > > > > > Note that starting with the latest v5-devel version
> > > > > > > > > AND a
> > > > > > > > > recent
> > > > > > > > > Linux
> > > > > > > > > kernel, we can ask the system for more precise
> > > > > > > > > timestamps
> > > > > > > > > on
> > > > > > > > > messages
> > > > > > > > > that
> > > > > > > > > come in via the log socket.
> > > > > > > > >
> > > > > > > > > Rainer
> > > > > > > > >
> > > > > > > > > > Some log entries get milliseconds and some do not:
> > > > > > > > > > The template:
> > > > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > > > %timegenerated:::date-
> > > > > > > > > > rfc3339% %msg%\n"
> > > > > > > > > >
> > > > > > > > > > The output:
> > > > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > > > test
> > > > > > > > > >
> > > > > > > > > > Am I missing something.
> > > > > > > > > >
> > > > > > > > > > Rgds
> > > > > > > > > > Rodney
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > >
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From rgerhards at hq.adiscon.com  Mon Jul 18 09:28:15 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jul 2011 09:28:15 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <6c13128b-e584-42c4-8bd0-08ed54a2c4db@wsrmckee>
References: <9B6E2A8877C38245BFB15CC491A11DA7281030@GRFEXC.intern.adiscon.com>
	<6c13128b-e584-42c4-8bd0-08ed54a2c4db@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281031@GRFEXC.intern.adiscon.com>

> > Can you elaborate why? That would be very interesting to me. I really
> > think
> > it is a shame that we have hi-res format since 5+ years, but
> > everybody turns
> > it off...
> >
> 
> It appears that their are a limited number of clients that I'm seeing
> logging with hi-res so to have it enabled for only a few services
> logging in hi-res would appear pointless.

I personally think "it depends" because you can correlate the hi-res ones
better. But I see your point. Also let me say that with a sufficiently recent
kernel, 5.8.3 is able to pull a hires timestamp from the system for all local
socket messages.

> If I could enable it in our environment and have all logging hi-res I
> will certainly be doing it, that's why we have been trying.
> The java application that we run WILL certainly be logging in hi-res
> and this will be centralized using log4j and rsyslog with the JSON
> module.
> 
> Out of interest we are also monitoring the rsyslog stats using pcp and
> I suspect we will have some modules/details heading your way once we
> have completed implementation and testing.

Let them flow :)

Rainer
> 
> > Rainer
> > >
> > >
> > > ----- Original Message -----
> > > > Ahh, I now see it. Look at the raw messages. Lines 7 and 31 are
> > > > correctly
> > > > formatted. Lines 15 and 23 have invalid format. With invalid
> > > > format,
> > > > interpretation is not guaranteed. Looks like 5.8.0 in that case
> > > > uses
> > > > the
> > > > timestamp of message reception. I suggest to use the current
> > > > stable,
> > > > I think
> > > > it will work somewhat different. Bottom line is that auditd
> > > > should
> > > > emit the
> > > > proper format.
> > > >
> > > > Rainer
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > Sent: Monday, July 18, 2011 8:26 AM
> > > > > To: rsyslog-users
> > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > >
> > > > > http://pastebin.com/TzPVzknt
> > > > > The 2 line I have previously seen with hi-res times are 15 and
> > > > > 23
> > > > >
> > > > > ----- Original Message -----
> > > > > > > -----Original Message-----
> > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > > > > To: rsyslog-users
> > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > >
> > > > > > > The following log has a restart of auditd and a ssh
> > > > > > > connection
> > > > > > > during
> > > > > > > the debug run.
> > > > > > > http://pastebin.com/cRPuA1Z8
> > > > > >
> > > > > > Thanks! Unfortunately, the instrumentation does not provide
> > > > > > what
> > > > > > I am
> > > > > > looking
> > > > > > for (maybe because of an older build, maybe it's just not
> > > > > > there...).
> > > > > > Can you
> > > > > > please also write all messages to a file with
> > > > > > RSYSLOG_DebugFormat
> > > > > > and
> > > > > > post
> > > > > > that file.
> > > > > >
> > > > > > With 5.8.0, you should probably never see hires, so I am a
> > > > > > bit
> > > > > > puzzled. Maybe
> > > > > > auditd does some "interesting" things to the log socket. Note
> > > > > > that
> > > > > > rsyslog
> > > > > > expects syslog() API format, but older versions (like 5.8.0)
> > > > > > did
> > > > > > not
> > > > > > enforce
> > > > > > that.
> > > > > >
> > > > > > Rainer
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > >
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > [mailto:rsyslog-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > > > > To: rsyslog-users
> > > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > > >
> > > > > > > > > Wow, Rainer, thanks for the quick response.
> > > > > > > > >
> > > > > > > > > So on a local system some processes actually provide a
> > > > > > > > > high
> > > > > > > > > res
> > > > > > > > > time
> > > > > > > > > that rsyslog then logs as %timereported%.
> > > > > > > >
> > > > > > > > As far as the local sockets is concerned, things should
> > > > > > > > be
> > > > > > > > consistent. If
> > > > > > > > that's not the case, it is best if you provide a debug
> > > > > > > > log --
> > > > > > > > the
> > > > > > > > log
> > > > > > > > samples
> > > > > > > > just show the result but now how we arrived there :)
> > > > > > > >
> > > > > > > > Rainer
> > > > > > > >
> > > > > > > > Did not realize this would be
> > > > > > > > > happening. I guess that most clients then do not
> > > > > > > > > provide
> > > > > > > > > the
> > > > > > > > > hi-res
> > > > > > > > > times and this might explain some messages having the
> > > > > > > > > time
> > > > > > > > > and
> > > > > > > > > most
> > > > > > > > > not:
> > > > > > > > >
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > 18T14:27:10.702529+10:00  The audit daemon is exiting.
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00
> > > > > > > > > 2011-07-
> > > > > > > > > 18T14:27:10.703673+10:00 audit(1310963230.693:4484770):
> > > > > > > > > audit_pid=0
> > > > > > > > > old=1773 by auid=4294967295
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00
> > > > > > > > > 2011-07-
> > > > > > > > > 18T14:27:10.867738+10:00 audit(1310963230.864:4484771):
> > > > > > > > > auid=672
> > > > > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > 18T14:27:10.959443+10:00  Warning - freq is non-zero
> > > > > > > > > and
> > > > > > > > > incremental
> > > > > > > > > flushing not selected.
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > 18T14:27:10.978467+10:00  Started dispatcher:
> > > > > > > > > /sbin/audispd
> > > > > > > > > pid:
> > > > > > > > > 4794
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00
> > > > > > > > > 2011-07-
> > > > > > > > > 18T14:27:10.981061+10:00 audit(1310963230.979:4484772):
> > > > > > > > > audit_pid=4792
> > > > > > > > > old=0 by auid=672
> > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > 18T14:27:10.998047+10:00  af_unix plugin initialized
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > ----- Original Message -----
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > > McKee
> > > > > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > > > > To: rsyslog-users
> > > > > > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > > >
> > > > > > > > > > > What effects the recording of milliseconds when
> > > > > > > > > > > using
> > > > > > > > > > > timereported:::date-
> > > > > > > > > > > rfc3339.
> > > > > > > > > >
> > > > > > > > > > This field contains what the sender told us. If the
> > > > > > > > > > sender
> > > > > > > > > > sent
> > > > > > > > > > no
> > > > > > > > > > ms, we can
> > > > > > > > > > not report them. Rather than to pretend "x.000000"
> > > > > > > > > > they
> > > > > > > > > > are
> > > > > > > > > > there, we
> > > > > > > > > > do not
> > > > > > > > > > give them. Note that for the same reason there may be
> > > > > > > > > > sub-ms
> > > > > > > > > > resolution, like
> > > > > > > > > > us, if that is what the sender reported.
> > > > > > > > > >
> > > > > > > > > > Note that starting with the latest v5-devel version
> > > > > > > > > > AND a
> > > > > > > > > > recent
> > > > > > > > > > Linux
> > > > > > > > > > kernel, we can ask the system for more precise
> > > > > > > > > > timestamps
> > > > > > > > > > on
> > > > > > > > > > messages
> > > > > > > > > > that
> > > > > > > > > > come in via the log socket.
> > > > > > > > > >
> > > > > > > > > > Rainer
> > > > > > > > > >
> > > > > > > > > > > Some log entries get milliseconds and some do not:
> > > > > > > > > > > The template:
> > > > > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > > > > %timegenerated:::date-
> > > > > > > > > > > rfc3339% %msg%\n"
> > > > > > > > > > >
> > > > > > > > > > > The output:
> > > > > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > > > > test
> > > > > > > > > > >
> > > > > > > > > > > Am I missing something.
> > > > > > > > > > >
> > > > > > > > > > > Rgds
> > > > > > > > > > > Rodney
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > >
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From alorbach at ro1.adiscon.com  Mon Jul 18 11:00:04 2011
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Mon, 18 Jul 2011 11:00:04 +0200
Subject: [rsyslog] Introducing Andre
Message-ID: <E152A03EC5F8DF4D8567006AF38A282D1ED66C@vmexch1.intern.adiscon.com>

Thanks Rainer. I will do my best to help handling bug reports ;).

Regards,
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Freitag, 15. Juli 2011 17:12
> To: rsyslog-users
> Subject: [rsyslog] Introducing Andre
> 
> Hi Folks,
> 
> I just wanted to let you know that Andre Lorbach, the main author of
Adiscon
> LogAnalyzer, is currently lending me a helping hand in rsyslog development.
> That way we hope I can concentrate on pushing the new config format while
> we are still being able to look at bug reports. Andre is a very experienced
C
> programmer and I am happy he can spare some time :-)
> 
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mad at comsoft.de  Mon Jul 18 15:33:05 2011
From: mad at comsoft.de (mad)
Date: Mon, 18 Jul 2011 15:33:05 +0200
Subject: [rsyslog] Monitoring socket /dev/log
Message-ID: <4E243611.2040902@comsoft.de>

Hi,

I try to monitor /dev/log with monit. I had some problems with rsyslog
still running but /dev/log did no longer exist.

Monitoring /dev/log directly did not work because it is no regular file.

Trying to monitor /dev/log did not work either. I tried to monitor
/dev/log as a stream based and a datagram socket. Both times monit could
not open a connection to the unix socket.

What type of socket is /dev/log? Did anyone already configured monit to
monitor rsyslog?

Thanks in advance for any help,
mad

From rodney.mckee at gmail.com  Tue Jul 19 00:02:38 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Tue, 19 Jul 2011 08:02:38 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281031@GRFEXC.intern.adiscon.com>
Message-ID: <dc96a4b8-7f83-4fbb-958d-0167f641b50b@wsrmckee>

I've been looking further into this and even on my Fedora 15 system with 2.6.38.8-35 and rsyslog 5.8.2 I'm only seeing low-res times for local services but for instance, iptables is logging with high-res times.

Do the services themselves need to support the use of hi-res timing, if that's the case then surely the usability of the hi-res timing is going to be reduced.
Is it likely to impact log analyzers having a mix of hi-res and low-res times with-in the logs.

I'd be interested to hear your thoughts on this.


----- Original Message -----
> > > Can you elaborate why? That would be very interesting to me. I
> > > really
> > > think
> > > it is a shame that we have hi-res format since 5+ years, but
> > > everybody turns
> > > it off...
> > >
> > 
> > It appears that their are a limited number of clients that I'm
> > seeing
> > logging with hi-res so to have it enabled for only a few services
> > logging in hi-res would appear pointless.
> 
> I personally think "it depends" because you can correlate the hi-res
> ones
> better. But I see your point. Also let me say that with a
> sufficiently recent
> kernel, 5.8.3 is able to pull a hires timestamp from the system for
> all local
> socket messages.
> 
> > If I could enable it in our environment and have all logging hi-res
> > I
> > will certainly be doing it, that's why we have been trying.
> > The java application that we run WILL certainly be logging in
> > hi-res
> > and this will be centralized using log4j and rsyslog with the JSON
> > module.
> > 
> > Out of interest we are also monitoring the rsyslog stats using pcp
> > and
> > I suspect we will have some modules/details heading your way once
> > we
> > have completed implementation and testing.
> 
> Let them flow :)
> 
> Rainer
> > 
> > > Rainer
> > > >
> > > >
> > > > ----- Original Message -----
> > > > > Ahh, I now see it. Look at the raw messages. Lines 7 and 31
> > > > > are
> > > > > correctly
> > > > > formatted. Lines 15 and 23 have invalid format. With invalid
> > > > > format,
> > > > > interpretation is not guaranteed. Looks like 5.8.0 in that
> > > > > case
> > > > > uses
> > > > > the
> > > > > timestamp of message reception. I suggest to use the current
> > > > > stable,
> > > > > I think
> > > > > it will work somewhat different. Bottom line is that auditd
> > > > > should
> > > > > emit the
> > > > > proper format.
> > > > >
> > > > > Rainer
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > Sent: Monday, July 18, 2011 8:26 AM
> > > > > > To: rsyslog-users
> > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > >
> > > > > > http://pastebin.com/TzPVzknt
> > > > > > The 2 line I have previously seen with hi-res times are 15
> > > > > > and
> > > > > > 23
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > > > -----Original Message-----
> > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > [mailto:rsyslog-
> > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > > > > > To: rsyslog-users
> > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > >
> > > > > > > > The following log has a restart of auditd and a ssh
> > > > > > > > connection
> > > > > > > > during
> > > > > > > > the debug run.
> > > > > > > > http://pastebin.com/cRPuA1Z8
> > > > > > >
> > > > > > > Thanks! Unfortunately, the instrumentation does not
> > > > > > > provide
> > > > > > > what
> > > > > > > I am
> > > > > > > looking
> > > > > > > for (maybe because of an older build, maybe it's just not
> > > > > > > there...).
> > > > > > > Can you
> > > > > > > please also write all messages to a file with
> > > > > > > RSYSLOG_DebugFormat
> > > > > > > and
> > > > > > > post
> > > > > > > that file.
> > > > > > >
> > > > > > > With 5.8.0, you should probably never see hires, so I am
> > > > > > > a
> > > > > > > bit
> > > > > > > puzzled. Maybe
> > > > > > > auditd does some "interesting" things to the log socket.
> > > > > > > Note
> > > > > > > that
> > > > > > > rsyslog
> > > > > > > expects syslog() API format, but older versions (like
> > > > > > > 5.8.0)
> > > > > > > did
> > > > > > > not
> > > > > > > enforce
> > > > > > > that.
> > > > > > >
> > > > > > > Rainer
> > > > > > > >
> > > > > > > > ----- Original Message -----
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > McKee
> > > > > > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > > > > > To: rsyslog-users
> > > > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > >
> > > > > > > > > > Wow, Rainer, thanks for the quick response.
> > > > > > > > > >
> > > > > > > > > > So on a local system some processes actually
> > > > > > > > > > provide a
> > > > > > > > > > high
> > > > > > > > > > res
> > > > > > > > > > time
> > > > > > > > > > that rsyslog then logs as %timereported%.
> > > > > > > > >
> > > > > > > > > As far as the local sockets is concerned, things
> > > > > > > > > should
> > > > > > > > > be
> > > > > > > > > consistent. If
> > > > > > > > > that's not the case, it is best if you provide a
> > > > > > > > > debug
> > > > > > > > > log --
> > > > > > > > > the
> > > > > > > > > log
> > > > > > > > > samples
> > > > > > > > > just show the result but now how we arrived there :)
> > > > > > > > >
> > > > > > > > > Rainer
> > > > > > > > >
> > > > > > > > > Did not realize this would be
> > > > > > > > > > happening. I guess that most clients then do not
> > > > > > > > > > provide
> > > > > > > > > > the
> > > > > > > > > > hi-res
> > > > > > > > > > times and this might explain some messages having
> > > > > > > > > > the
> > > > > > > > > > time
> > > > > > > > > > and
> > > > > > > > > > most
> > > > > > > > > > not:
> > > > > > > > > >
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > 18T14:27:10.702529+10:00  The audit daemon is
> > > > > > > > > > exiting.
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00
> > > > > > > > > > 2011-07-
> > > > > > > > > > 18T14:27:10.703673+10:00
> > > > > > > > > > audit(1310963230.693:4484770):
> > > > > > > > > > audit_pid=0
> > > > > > > > > > old=1773 by auid=4294967295
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00
> > > > > > > > > > 2011-07-
> > > > > > > > > > 18T14:27:10.867738+10:00
> > > > > > > > > > audit(1310963230.864:4484771):
> > > > > > > > > > auid=672
> > > > > > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > 18T14:27:10.959443+10:00  Warning - freq is
> > > > > > > > > > non-zero
> > > > > > > > > > and
> > > > > > > > > > incremental
> > > > > > > > > > flushing not selected.
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > 18T14:27:10.978467+10:00  Started dispatcher:
> > > > > > > > > > /sbin/audispd
> > > > > > > > > > pid:
> > > > > > > > > > 4794
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00
> > > > > > > > > > 2011-07-
> > > > > > > > > > 18T14:27:10.981061+10:00
> > > > > > > > > > audit(1310963230.979:4484772):
> > > > > > > > > > audit_pid=4792
> > > > > > > > > > old=0 by auid=672
> > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > 18T14:27:10.998047+10:00  af_unix plugin
> > > > > > > > > > initialized
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ----- Original Message -----
> > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > > > McKee
> > > > > > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > > > > > To: rsyslog-users
> > > > > > > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > > > >
> > > > > > > > > > > > What effects the recording of milliseconds when
> > > > > > > > > > > > using
> > > > > > > > > > > > timereported:::date-
> > > > > > > > > > > > rfc3339.
> > > > > > > > > > >
> > > > > > > > > > > This field contains what the sender told us. If
> > > > > > > > > > > the
> > > > > > > > > > > sender
> > > > > > > > > > > sent
> > > > > > > > > > > no
> > > > > > > > > > > ms, we can
> > > > > > > > > > > not report them. Rather than to pretend
> > > > > > > > > > > "x.000000"
> > > > > > > > > > > they
> > > > > > > > > > > are
> > > > > > > > > > > there, we
> > > > > > > > > > > do not
> > > > > > > > > > > give them. Note that for the same reason there
> > > > > > > > > > > may be
> > > > > > > > > > > sub-ms
> > > > > > > > > > > resolution, like
> > > > > > > > > > > us, if that is what the sender reported.
> > > > > > > > > > >
> > > > > > > > > > > Note that starting with the latest v5-devel
> > > > > > > > > > > version
> > > > > > > > > > > AND a
> > > > > > > > > > > recent
> > > > > > > > > > > Linux
> > > > > > > > > > > kernel, we can ask the system for more precise
> > > > > > > > > > > timestamps
> > > > > > > > > > > on
> > > > > > > > > > > messages
> > > > > > > > > > > that
> > > > > > > > > > > come in via the log socket.
> > > > > > > > > > >
> > > > > > > > > > > Rainer
> > > > > > > > > > >
> > > > > > > > > > > > Some log entries get milliseconds and some do
> > > > > > > > > > > > not:
> > > > > > > > > > > > The template:
> > > > > > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > > > > > %timegenerated:::date-
> > > > > > > > > > > > rfc3339% %msg%\n"
> > > > > > > > > > > >
> > > > > > > > > > > > The output:
> > > > > > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > > > > > test
> > > > > > > > > > > >
> > > > > > > > > > > > Am I missing something.
> > > > > > > > > > > >
> > > > > > > > > > > > Rgds
> > > > > > > > > > > > Rodney
> > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > >
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From rgerhards at hq.adiscon.com  Tue Jul 19 07:28:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jul 2011 07:28:37 +0200
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <dc96a4b8-7f83-4fbb-958d-0167f641b50b@wsrmckee>
References: <9B6E2A8877C38245BFB15CC491A11DA7281031@GRFEXC.intern.adiscon.com>
	<dc96a4b8-7f83-4fbb-958d-0167f641b50b@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281042@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Tuesday, July 19, 2011 12:03 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] timereported:::date-rfc3339
> 
> I've been looking further into this and even on my Fedora 15 system
> with 2.6.38.8-35 and rsyslog 5.8.2 I'm only seeing low-res times for
> local services but for instance, iptables is logging with high-res
> times.

Can you provide me a debug format example? I know I can set up another lab
for that, but that ties up some resources I don't have during reimplementing
the config format.

I've checked the ChangeLog. You need at least 5.9.1 to obtain timestamps from
the kernel.

> 
> Do the services themselves need to support the use of hi-res timing,

for all but imuxsock, that's for sure true, because the apps emit the format.
But of course, for imuxsock 5.9.1+ can pull from the kernel (iff the kernel
is recent enough -- there was a patch to the kernel to support this - at
least SUSE already ships it and I guess F15, too).

> if
> that's the case then surely the usability of the hi-res timing is going
> to be reduced.
> Is it likely to impact log analyzers having a mix of hi-res and low-res
> times with-in the logs.

I really think that log analyzers need to be fixed. After all, what's the
problem with parsing an ISO date with different time resolution? I think it's
5 to 10 lines of code in rsyslog. Not a big problem, really. It takes more
time to write this post than to code that ;)

BUT: if that would be a solution, I could always write milliseconds, even if
they are unknown. I could simply write them as "s.000000". However, this
gives a false impression of correctness. Because when you see "s.000000", you
don't know any longer if it were actually at "s.000000" or even at
"s.999999". In order to differentiate between the cases, where we really have
"s.000000" vs. where we have just "s", the timestamp is written with the
resolution provided. This is also as of RFC recommendation. Please note that
this actually is an *aid* to (sufficiently well-written) log analyzers.

Rainer
> 
> I'd be interested to hear your thoughts on this.
> 
> 
> ----- Original Message -----
> > > > Can you elaborate why? That would be very interesting to me. I
> > > > really
> > > > think
> > > > it is a shame that we have hi-res format since 5+ years, but
> > > > everybody turns
> > > > it off...
> > > >
> > >
> > > It appears that their are a limited number of clients that I'm
> > > seeing
> > > logging with hi-res so to have it enabled for only a few services
> > > logging in hi-res would appear pointless.
> >
> > I personally think "it depends" because you can correlate the hi-res
> > ones
> > better. But I see your point. Also let me say that with a
> > sufficiently recent
> > kernel, 5.8.3 is able to pull a hires timestamp from the system for
> > all local
> > socket messages.
> >
> > > If I could enable it in our environment and have all logging hi-res
> > > I
> > > will certainly be doing it, that's why we have been trying.
> > > The java application that we run WILL certainly be logging in
> > > hi-res
> > > and this will be centralized using log4j and rsyslog with the JSON
> > > module.
> > >
> > > Out of interest we are also monitoring the rsyslog stats using pcp
> > > and
> > > I suspect we will have some modules/details heading your way once
> > > we
> > > have completed implementation and testing.
> >
> > Let them flow :)
> >
> > Rainer
> > >
> > > > Rainer
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > > Ahh, I now see it. Look at the raw messages. Lines 7 and 31
> > > > > > are
> > > > > > correctly
> > > > > > formatted. Lines 15 and 23 have invalid format. With invalid
> > > > > > format,
> > > > > > interpretation is not guaranteed. Looks like 5.8.0 in that
> > > > > > case
> > > > > > uses
> > > > > > the
> > > > > > timestamp of message reception. I suggest to use the current
> > > > > > stable,
> > > > > > I think
> > > > > > it will work somewhat different. Bottom line is that auditd
> > > > > > should
> > > > > > emit the
> > > > > > proper format.
> > > > > >
> > > > > > Rainer
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > Sent: Monday, July 18, 2011 8:26 AM
> > > > > > > To: rsyslog-users
> > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > >
> > > > > > > http://pastebin.com/TzPVzknt
> > > > > > > The 2 line I have previously seen with hi-res times are 15
> > > > > > > and
> > > > > > > 23
> > > > > > >
> > > > > > > ----- Original Message -----
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > [mailto:rsyslog-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > > > > > > To: rsyslog-users
> > > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > > >
> > > > > > > > > The following log has a restart of auditd and a ssh
> > > > > > > > > connection
> > > > > > > > > during
> > > > > > > > > the debug run.
> > > > > > > > > http://pastebin.com/cRPuA1Z8
> > > > > > > >
> > > > > > > > Thanks! Unfortunately, the instrumentation does not
> > > > > > > > provide
> > > > > > > > what
> > > > > > > > I am
> > > > > > > > looking
> > > > > > > > for (maybe because of an older build, maybe it's just not
> > > > > > > > there...).
> > > > > > > > Can you
> > > > > > > > please also write all messages to a file with
> > > > > > > > RSYSLOG_DebugFormat
> > > > > > > > and
> > > > > > > > post
> > > > > > > > that file.
> > > > > > > >
> > > > > > > > With 5.8.0, you should probably never see hires, so I am
> > > > > > > > a
> > > > > > > > bit
> > > > > > > > puzzled. Maybe
> > > > > > > > auditd does some "interesting" things to the log socket.
> > > > > > > > Note
> > > > > > > > that
> > > > > > > > rsyslog
> > > > > > > > expects syslog() API format, but older versions (like
> > > > > > > > 5.8.0)
> > > > > > > > did
> > > > > > > > not
> > > > > > > > enforce
> > > > > > > > that.
> > > > > > > >
> > > > > > > > Rainer
> > > > > > > > >
> > > > > > > > > ----- Original Message -----
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > > McKee
> > > > > > > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > > > > > > To: rsyslog-users
> > > > > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > > >
> > > > > > > > > > > Wow, Rainer, thanks for the quick response.
> > > > > > > > > > >
> > > > > > > > > > > So on a local system some processes actually
> > > > > > > > > > > provide a
> > > > > > > > > > > high
> > > > > > > > > > > res
> > > > > > > > > > > time
> > > > > > > > > > > that rsyslog then logs as %timereported%.
> > > > > > > > > >
> > > > > > > > > > As far as the local sockets is concerned, things
> > > > > > > > > > should
> > > > > > > > > > be
> > > > > > > > > > consistent. If
> > > > > > > > > > that's not the case, it is best if you provide a
> > > > > > > > > > debug
> > > > > > > > > > log --
> > > > > > > > > > the
> > > > > > > > > > log
> > > > > > > > > > samples
> > > > > > > > > > just show the result but now how we arrived there :)
> > > > > > > > > >
> > > > > > > > > > Rainer
> > > > > > > > > >
> > > > > > > > > > Did not realize this would be
> > > > > > > > > > > happening. I guess that most clients then do not
> > > > > > > > > > > provide
> > > > > > > > > > > the
> > > > > > > > > > > hi-res
> > > > > > > > > > > times and this might explain some messages having
> > > > > > > > > > > the
> > > > > > > > > > > time
> > > > > > > > > > > and
> > > > > > > > > > > most
> > > > > > > > > > > not:
> > > > > > > > > > >
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > > 18T14:27:10.702529+10:00  The audit daemon is
> > > > > > > > > > > exiting.
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.703673+10:00
> > > > > > > > > > > 2011-07-
> > > > > > > > > > > 18T14:27:10.703673+10:00
> > > > > > > > > > > audit(1310963230.693:4484770):
> > > > > > > > > > > audit_pid=0
> > > > > > > > > > > old=1773 by auid=4294967295
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.867738+10:00
> > > > > > > > > > > 2011-07-
> > > > > > > > > > > 18T14:27:10.867738+10:00
> > > > > > > > > > > audit(1310963230.864:4484771):
> > > > > > > > > > > auid=672
> > > > > > > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > > 18T14:27:10.959443+10:00  Warning - freq is
> > > > > > > > > > > non-zero
> > > > > > > > > > > and
> > > > > > > > > > > incremental
> > > > > > > > > > > flushing not selected.
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > > 18T14:27:10.978467+10:00  Started dispatcher:
> > > > > > > > > > > /sbin/audispd
> > > > > > > > > > > pid:
> > > > > > > > > > > 4794
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10.981061+10:00
> > > > > > > > > > > 2011-07-
> > > > > > > > > > > 18T14:27:10.981061+10:00
> > > > > > > > > > > audit(1310963230.979:4484772):
> > > > > > > > > > > audit_pid=4792
> > > > > > > > > > > old=0 by auid=672
> > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00 2011-07-
> > > > > > > > > > > 18T14:27:10.998047+10:00  af_unix plugin
> > > > > > > > > > > initialized
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > ----- Original Message -----
> > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > > > > McKee
> > > > > > > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > > > > > > To: rsyslog-users
> > > > > > > > > > > > > Subject: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > > > > >
> > > > > > > > > > > > > What effects the recording of milliseconds when
> > > > > > > > > > > > > using
> > > > > > > > > > > > > timereported:::date-
> > > > > > > > > > > > > rfc3339.
> > > > > > > > > > > >
> > > > > > > > > > > > This field contains what the sender told us. If
> > > > > > > > > > > > the
> > > > > > > > > > > > sender
> > > > > > > > > > > > sent
> > > > > > > > > > > > no
> > > > > > > > > > > > ms, we can
> > > > > > > > > > > > not report them. Rather than to pretend
> > > > > > > > > > > > "x.000000"
> > > > > > > > > > > > they
> > > > > > > > > > > > are
> > > > > > > > > > > > there, we
> > > > > > > > > > > > do not
> > > > > > > > > > > > give them. Note that for the same reason there
> > > > > > > > > > > > may be
> > > > > > > > > > > > sub-ms
> > > > > > > > > > > > resolution, like
> > > > > > > > > > > > us, if that is what the sender reported.
> > > > > > > > > > > >
> > > > > > > > > > > > Note that starting with the latest v5-devel
> > > > > > > > > > > > version
> > > > > > > > > > > > AND a
> > > > > > > > > > > > recent
> > > > > > > > > > > > Linux
> > > > > > > > > > > > kernel, we can ask the system for more precise
> > > > > > > > > > > > timestamps
> > > > > > > > > > > > on
> > > > > > > > > > > > messages
> > > > > > > > > > > > that
> > > > > > > > > > > > come in via the log socket.
> > > > > > > > > > > >
> > > > > > > > > > > > Rainer
> > > > > > > > > > > >
> > > > > > > > > > > > > Some log entries get milliseconds and some do
> > > > > > > > > > > > > not:
> > > > > > > > > > > > > The template:
> > > > > > > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > > > > > > %timegenerated:::date-
> > > > > > > > > > > > > rfc3339% %msg%\n"
> > > > > > > > > > > > >
> > > > > > > > > > > > > The output:
> > > > > > > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > > > > > > test
> > > > > > > > > > > > >
> > > > > > > > > > > > > Am I missing something.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Rgds
> > > > > > > > > > > > > Rodney
> > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > > >
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > > >
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > >
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > >
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > >
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Tue Jul 19 08:58:30 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jul 2011 08:58:30 +0200
Subject: [rsyslog] linking from dlloaded modules to global symbols
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281046@GRFEXC.intern.adiscon.com>

Hi all,

I have a question to which some of you may have at least an opinion.

A couple of years ago, I read in several guides and textbooks that there
potentially is a problem in this situation:

Let a, b be dlloaded modules. Let a.a and b.b be public symbols in these
modules, and g.g a public symbol inside rsyslog core (which loads a and b).
Now assume code in b wants to access a.a and g.g (so the dlloaded module
needs to resolve public symbols inside the core and other dlloaded modules
during its load time). Linux handles this without problems, and as it looks
Solaris and BSD do as well. However, I was told that there are some platforms
which do not support this (AIX? HP-UX?).

In order to keep portable, I created a scheme where b (the dlloaded module)
queries pointers to all other methods in core and dlloaded modules it needs.
This works well, but there is some overhead associated with that. Obviously,
there is a (mild) runtime overhead. But it also causes development overhead,
as quite some code needs to be written to support that.

Looking backward, I have never seen any platform that actually requires that
scheme. Even "better", I have not followed it in all cases (some legacy
calls) and I never got into any troubles. May be that's just because I don't
have AIX and HP-UX and ...

The big question is if it really makes sense to continue that way. If we can
conclude that the problem this code intends to solve is actually no problem,
I could clean up a lot of code and also get some better development
productivity in the future. Also, as far as I know, libtool permits static
module linking for such cases if a very exotic platform would actually not
support the dynaload functionality.

So I am pretty tempted to drop that approach and use the Linux dynalinker's
capability. Does anyone have concerns about this? Does it sound reasonable?

Feedback would deeply be appreciated.

Thanks,
Rainer 

From david at lang.hm  Tue Jul 19 09:06:48 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jul 2011 00:06:48 -0700 (PDT)
Subject: [rsyslog] linking from dlloaded modules to global symbols
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281046@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7281046@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107190005360.3558@asgard.lang.hm>

I have access to SIX (and one of these day's I'll get a chance to try to 
get rsyslog to compile there again)

can you make a short test program for me to compile there? I know I have 
AIX 5.3 and 6.1 available, and I think I have 5.2 available.

David Lang

On Tue, 19 Jul 2011, Rainer Gerhards wrote:

> Date: Tue, 19 Jul 2011 08:58:30 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: [rsyslog] linking from dlloaded modules to global symbols
> 
> Hi all,
>
> I have a question to which some of you may have at least an opinion.
>
> A couple of years ago, I read in several guides and textbooks that there
> potentially is a problem in this situation:
>
> Let a, b be dlloaded modules. Let a.a and b.b be public symbols in these
> modules, and g.g a public symbol inside rsyslog core (which loads a and b).
> Now assume code in b wants to access a.a and g.g (so the dlloaded module
> needs to resolve public symbols inside the core and other dlloaded modules
> during its load time). Linux handles this without problems, and as it looks
> Solaris and BSD do as well. However, I was told that there are some platforms
> which do not support this (AIX? HP-UX?).
>
> In order to keep portable, I created a scheme where b (the dlloaded module)
> queries pointers to all other methods in core and dlloaded modules it needs.
> This works well, but there is some overhead associated with that. Obviously,
> there is a (mild) runtime overhead. But it also causes development overhead,
> as quite some code needs to be written to support that.
>
> Looking backward, I have never seen any platform that actually requires that
> scheme. Even "better", I have not followed it in all cases (some legacy
> calls) and I never got into any troubles. May be that's just because I don't
> have AIX and HP-UX and ...
>
> The big question is if it really makes sense to continue that way. If we can
> conclude that the problem this code intends to solve is actually no problem,
> I could clean up a lot of code and also get some better development
> productivity in the future. Also, as far as I know, libtool permits static
> module linking for such cases if a very exotic platform would actually not
> support the dynaload functionality.
>
> So I am pretty tempted to drop that approach and use the Linux dynalinker's
> capability. Does anyone have concerns about this? Does it sound reasonable?
>
> Feedback would deeply be appreciated.
>
> Thanks,
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Tue Jul 19 09:09:41 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jul 2011 09:09:41 +0200
Subject: [rsyslog] linking from dlloaded modules to global symbols
In-Reply-To: <alpine.DEB.2.02.1107190005360.3558@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA7281046@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1107190005360.3558@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281047@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, July 19, 2011 9:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] linking from dlloaded modules to global symbols
> 
> I have access to SIX (and one of these day's I'll get a chance to try
> to
> get rsyslog to compile there again)
> 
> can you make a short test program for me to compile there? I know I
> have
> AIX 5.3 and 6.1 available, and I think I have 5.2 available.

Thanks - I'll try ASAP, but it may take me few days as I think it is not
totally trivial (need to fiddle with autotools, create new project etc -- not
just a simple program).

Rainer

> 
> David Lang
> 
> On Tue, 19 Jul 2011, Rainer Gerhards wrote:
> 
> > Date: Tue, 19 Jul 2011 08:58:30 +0200
> > From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog-users <rsyslog at lists.adiscon.com>
> > Subject: [rsyslog] linking from dlloaded modules to global symbols
> >
> > Hi all,
> >
> > I have a question to which some of you may have at least an opinion.
> >
> > A couple of years ago, I read in several guides and textbooks that
> there
> > potentially is a problem in this situation:
> >
> > Let a, b be dlloaded modules. Let a.a and b.b be public symbols in
> these
> > modules, and g.g a public symbol inside rsyslog core (which loads a
> and b).
> > Now assume code in b wants to access a.a and g.g (so the dlloaded
> module
> > needs to resolve public symbols inside the core and other dlloaded
> modules
> > during its load time). Linux handles this without problems, and as it
> looks
> > Solaris and BSD do as well. However, I was told that there are some
> platforms
> > which do not support this (AIX? HP-UX?).
> >
> > In order to keep portable, I created a scheme where b (the dlloaded
> module)
> > queries pointers to all other methods in core and dlloaded modules it
> needs.
> > This works well, but there is some overhead associated with that.
> Obviously,
> > there is a (mild) runtime overhead. But it also causes development
> overhead,
> > as quite some code needs to be written to support that.
> >
> > Looking backward, I have never seen any platform that actually
> requires that
> > scheme. Even "better", I have not followed it in all cases (some
> legacy
> > calls) and I never got into any troubles. May be that's just because
> I don't
> > have AIX and HP-UX and ...
> >
> > The big question is if it really makes sense to continue that way. If
> we can
> > conclude that the problem this code intends to solve is actually no
> problem,
> > I could clean up a lot of code and also get some better development
> > productivity in the future. Also, as far as I know, libtool permits
> static
> > module linking for such cases if a very exotic platform would
> actually not
> > support the dynaload functionality.
> >
> > So I am pretty tempted to drop that approach and use the Linux
> dynalinker's
> > capability. Does anyone have concerns about this? Does it sound
> reasonable?
> >
> > Feedback would deeply be appreciated.
> >
> > Thanks,
> > Rainer
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Tue Jul 19 09:39:24 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jul 2011 00:39:24 -0700 (PDT)
Subject: [rsyslog] linking from dlloaded modules to global symbols
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281047@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA7281046@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1107190005360.3558@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7281047@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1107190038550.3558@asgard.lang.hm>

getting autotools setup properly is what has stalled me on doing the 
rsyslog AIX compile.

David Lang

On Tue, 19 Jul 2011, Rainer Gerhards wrote:

> Date: Tue, 19 Jul 2011 09:09:41 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: [rsyslog] linking from dlloaded modules to global symbols
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Tuesday, July 19, 2011 9:07 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] linking from dlloaded modules to global symbols
>>
>> I have access to SIX (and one of these day's I'll get a chance to try
>> to
>> get rsyslog to compile there again)
>>
>> can you make a short test program for me to compile there? I know I
>> have
>> AIX 5.3 and 6.1 available, and I think I have 5.2 available.
>
> Thanks - I'll try ASAP, but it may take me few days as I think it is not
> totally trivial (need to fiddle with autotools, create new project etc -- not
> just a simple program).
>
> Rainer
>
>>
>> David Lang
>>
>> On Tue, 19 Jul 2011, Rainer Gerhards wrote:
>>
>>> Date: Tue, 19 Jul 2011 08:58:30 +0200
>>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
>>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> To: rsyslog-users <rsyslog at lists.adiscon.com>
>>> Subject: [rsyslog] linking from dlloaded modules to global symbols
>>>
>>> Hi all,
>>>
>>> I have a question to which some of you may have at least an opinion.
>>>
>>> A couple of years ago, I read in several guides and textbooks that
>> there
>>> potentially is a problem in this situation:
>>>
>>> Let a, b be dlloaded modules. Let a.a and b.b be public symbols in
>> these
>>> modules, and g.g a public symbol inside rsyslog core (which loads a
>> and b).
>>> Now assume code in b wants to access a.a and g.g (so the dlloaded
>> module
>>> needs to resolve public symbols inside the core and other dlloaded
>> modules
>>> during its load time). Linux handles this without problems, and as it
>> looks
>>> Solaris and BSD do as well. However, I was told that there are some
>> platforms
>>> which do not support this (AIX? HP-UX?).
>>>
>>> In order to keep portable, I created a scheme where b (the dlloaded
>> module)
>>> queries pointers to all other methods in core and dlloaded modules it
>> needs.
>>> This works well, but there is some overhead associated with that.
>> Obviously,
>>> there is a (mild) runtime overhead. But it also causes development
>> overhead,
>>> as quite some code needs to be written to support that.
>>>
>>> Looking backward, I have never seen any platform that actually
>> requires that
>>> scheme. Even "better", I have not followed it in all cases (some
>> legacy
>>> calls) and I never got into any troubles. May be that's just because
>> I don't
>>> have AIX and HP-UX and ...
>>>
>>> The big question is if it really makes sense to continue that way. If
>> we can
>>> conclude that the problem this code intends to solve is actually no
>> problem,
>>> I could clean up a lot of code and also get some better development
>>> productivity in the future. Also, as far as I know, libtool permits
>> static
>>> module linking for such cases if a very exotic platform would
>> actually not
>>> support the dynaload functionality.
>>>
>>> So I am pretty tempted to drop that approach and use the Linux
>> dynalinker's
>>> capability. Does anyone have concerns about this? Does it sound
>> reasonable?
>>>
>>> Feedback would deeply be appreciated.
>>>
>>> Thanks,
>>> Rainer
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Tue Jul 19 11:59:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jul 2011 11:59:24 +0200
Subject: [rsyslog] linking from dlloaded modules to global symbols
In-Reply-To: <alpine.DEB.2.02.1107190038550.3558@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA7281046@GRFEXC.intern.adiscon.com><alpine.DEB.2.02.1107190005360.3558@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7281047@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1107190038550.3558@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728104C@GRFEXC.intern.adiscon.com>

OK, I placed some rudimentary files at

http://git.adiscon.com/?p=rsyslog.git;a=tree;f=dlopen;h=cb58031408718339c86a6
6608aa053591eeb949f;hb=dlopen-test

As far as I know, they should be sufficient to prove the point. Obviously,
the makefile is far from clean, but I guess you can handle that ;) It's plain
C, no autotools, no libtool, so we can not prove if libtool would solve a
problem. So let's first see if it exists (and if so, let's talk if it is
worth fixing ;)).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, July 19, 2011 9:39 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] linking from dlloaded modules to global symbols
> 
> getting autotools setup properly is what has stalled me on doing the
> rsyslog AIX compile.
> 
> David Lang
> 
> On Tue, 19 Jul 2011, Rainer Gerhards wrote:
> 
> > Date: Tue, 19 Jul 2011 09:09:41 +0200
> > From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog-users <rsyslog at lists.adiscon.com>
> > Subject: Re: [rsyslog] linking from dlloaded modules to global
> symbols
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >> Sent: Tuesday, July 19, 2011 9:07 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] linking from dlloaded modules to global
> symbols
> >>
> >> I have access to SIX (and one of these day's I'll get a chance to
> try
> >> to
> >> get rsyslog to compile there again)
> >>
> >> can you make a short test program for me to compile there? I know I
> >> have
> >> AIX 5.3 and 6.1 available, and I think I have 5.2 available.
> >
> > Thanks - I'll try ASAP, but it may take me few days as I think it is
> not
> > totally trivial (need to fiddle with autotools, create new project
> etc -- not
> > just a simple program).
> >
> > Rainer
> >
> >>
> >> David Lang
> >>
> >> On Tue, 19 Jul 2011, Rainer Gerhards wrote:
> >>
> >>> Date: Tue, 19 Jul 2011 08:58:30 +0200
> >>> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> >>> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>> To: rsyslog-users <rsyslog at lists.adiscon.com>
> >>> Subject: [rsyslog] linking from dlloaded modules to global symbols
> >>>
> >>> Hi all,
> >>>
> >>> I have a question to which some of you may have at least an
> opinion.
> >>>
> >>> A couple of years ago, I read in several guides and textbooks that
> >> there
> >>> potentially is a problem in this situation:
> >>>
> >>> Let a, b be dlloaded modules. Let a.a and b.b be public symbols in
> >> these
> >>> modules, and g.g a public symbol inside rsyslog core (which loads a
> >> and b).
> >>> Now assume code in b wants to access a.a and g.g (so the dlloaded
> >> module
> >>> needs to resolve public symbols inside the core and other dlloaded
> >> modules
> >>> during its load time). Linux handles this without problems, and as
> it
> >> looks
> >>> Solaris and BSD do as well. However, I was told that there are some
> >> platforms
> >>> which do not support this (AIX? HP-UX?).
> >>>
> >>> In order to keep portable, I created a scheme where b (the dlloaded
> >> module)
> >>> queries pointers to all other methods in core and dlloaded modules
> it
> >> needs.
> >>> This works well, but there is some overhead associated with that.
> >> Obviously,
> >>> there is a (mild) runtime overhead. But it also causes development
> >> overhead,
> >>> as quite some code needs to be written to support that.
> >>>
> >>> Looking backward, I have never seen any platform that actually
> >> requires that
> >>> scheme. Even "better", I have not followed it in all cases (some
> >> legacy
> >>> calls) and I never got into any troubles. May be that's just
> because
> >> I don't
> >>> have AIX and HP-UX and ...
> >>>
> >>> The big question is if it really makes sense to continue that way.
> If
> >> we can
> >>> conclude that the problem this code intends to solve is actually no
> >> problem,
> >>> I could clean up a lot of code and also get some better development
> >>> productivity in the future. Also, as far as I know, libtool permits
> >> static
> >>> module linking for such cases if a very exotic platform would
> >> actually not
> >>> support the dynaload functionality.
> >>>
> >>> So I am pretty tempted to drop that approach and use the Linux
> >> dynalinker's
> >>> capability. Does anyone have concerns about this? Does it sound
> >> reasonable?
> >>>
> >>> Feedback would deeply be appreciated.
> >>>
> >>> Thanks,
> >>> Rainer
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Wed Jul 20 03:27:09 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jul 2011 18:27:09 -0700 (PDT)
Subject: [rsyslog] Monitoring socket /dev/log
In-Reply-To: <4E243611.2040902@comsoft.de>
References: <4E243611.2040902@comsoft.de>
Message-ID: <alpine.DEB.2.02.1107191825310.1582@asgard.lang.hm>

how does monit monitor other syslog implementations (specifically thinking 
of syslog-ng here)

David Lang

On Mon, 18 Jul 2011, mad wrote:

> Hi,
>
> I try to monitor /dev/log with monit. I had some problems with rsyslog
> still running but /dev/log did no longer exist.
>
> Monitoring /dev/log directly did not work because it is no regular file.
>
> Trying to monitor /dev/log did not work either. I tried to monitor
> /dev/log as a stream based and a datagram socket. Both times monit could
> not open a connection to the unix socket.
>
> What type of socket is /dev/log? Did anyone already configured monit to
> monitor rsyslog?
>
> Thanks in advance for any help,
> mad
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rodney.mckee at gmail.com  Wed Jul 20 05:55:15 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Wed, 20 Jul 2011 13:55:15 +1000 (EST)
Subject: [rsyslog] remote logging from multiple timezones
In-Reply-To: <0586adea-14b8-4d93-a360-b6ebb0fcaf02@wsrmckee>
Message-ID: <8ab3022e-b959-4b70-8011-152b8045d3e6@wsrmckee>

Collecting logs from multiple timezones and rotating on a central log server with a template like:
$template DAILYSYSLOG,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log"

means that logs from each timezone still get rotated at midnight on the localhost, would something like the following work to rotate logs for each timezones change of day

$template TEST,"/var/log/logdir/%TIMESTAMP:1:4:date-rfc3339%/%TIMESTAMP:6:7:date-rfc3339%/%TIMESTAMP:9:10:date-rfc3339%/%HOSTNAME%/messages.log"

or would it cause other issues like excessive load etc or some other unforeseen issue. I guess it the date format gets screwed it could have some impact.

Rgds
Rodney

From rodney.mckee at gmail.com  Wed Jul 20 06:00:39 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Wed, 20 Jul 2011 14:00:39 +1000 (EST)
Subject: [rsyslog] timereported:::date-rfc3339
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281042@GRFEXC.intern.adiscon.com>
Message-ID: <9392fb58-64b3-4d1b-a916-8c0a2dfc1321@wsrmckee>



----- Original Message -----
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > Sent: Tuesday, July 19, 2011 12:03 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > 
> > I've been looking further into this and even on my Fedora 15 system
> > with 2.6.38.8-35 and rsyslog 5.8.2 I'm only seeing low-res times
> > for
> > local services but for instance, iptables is logging with high-res
> > times.
> 
> Can you provide me a debug format example? I know I can set up
> another lab
> for that, but that ties up some resources I don't have during
> reimplementing
> the config format.
> 
> I've checked the ChangeLog. You need at least 5.9.1 to obtain
> timestamps from
> the kernel.

This pretty much cover your request for the above debug!
Our prod systems are no where need these kernel/rsyslog levels anyway.

> 
> > 
> > Do the services themselves need to support the use of hi-res
> > timing,
> 
> for all but imuxsock, that's for sure true, because the apps emit the
> format.
> But of course, for imuxsock 5.9.1+ can pull from the kernel (iff the
> kernel
> is recent enough -- there was a patch to the kernel to support this -
> at
> least SUSE already ships it and I guess F15, too).
> 
> > if
> > that's the case then surely the usability of the hi-res timing is
> > going
> > to be reduced.
> > Is it likely to impact log analyzers having a mix of hi-res and
> > low-res
> > times with-in the logs.
> 
> I really think that log analyzers need to be fixed. After all, what's
> the
> problem with parsing an ISO date with different time resolution? I
> think it's
> 5 to 10 lines of code in rsyslog. Not a big problem, really. It takes
> more
> time to write this post than to code that ;)
> 
> BUT: if that would be a solution, I could always write milliseconds,
> even if
> they are unknown. I could simply write them as "s.000000". However,
> this
> gives a false impression of correctness. Because when you see
> "s.000000", you
> don't know any longer if it were actually at "s.000000" or even at
> "s.999999". In order to differentiate between the cases, where we
> really have
> "s.000000" vs. where we have just "s", the timestamp is written with
> the
> resolution provided. This is also as of RFC recommendation. Please
> note that
> this actually is an *aid* to (sufficiently well-written) log
> analyzers.
> 
> Rainer
> > 
> > I'd be interested to hear your thoughts on this.
> > 
> > 
> > ----- Original Message -----
> > > > > Can you elaborate why? That would be very interesting to me.
> > > > > I
> > > > > really
> > > > > think
> > > > > it is a shame that we have hi-res format since 5+ years, but
> > > > > everybody turns
> > > > > it off...
> > > > >
> > > >
> > > > It appears that their are a limited number of clients that I'm
> > > > seeing
> > > > logging with hi-res so to have it enabled for only a few
> > > > services
> > > > logging in hi-res would appear pointless.
> > >
> > > I personally think "it depends" because you can correlate the
> > > hi-res
> > > ones
> > > better. But I see your point. Also let me say that with a
> > > sufficiently recent
> > > kernel, 5.8.3 is able to pull a hires timestamp from the system
> > > for
> > > all local
> > > socket messages.
> > >
> > > > If I could enable it in our environment and have all logging
> > > > hi-res
> > > > I
> > > > will certainly be doing it, that's why we have been trying.
> > > > The java application that we run WILL certainly be logging in
> > > > hi-res
> > > > and this will be centralized using log4j and rsyslog with the
> > > > JSON
> > > > module.
> > > >
> > > > Out of interest we are also monitoring the rsyslog stats using
> > > > pcp
> > > > and
> > > > I suspect we will have some modules/details heading your way
> > > > once
> > > > we
> > > > have completed implementation and testing.
> > >
> > > Let them flow :)
> > >
> > > Rainer
> > > >
> > > > > Rainer
> > > > > >
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > > Ahh, I now see it. Look at the raw messages. Lines 7 and
> > > > > > > 31
> > > > > > > are
> > > > > > > correctly
> > > > > > > formatted. Lines 15 and 23 have invalid format. With
> > > > > > > invalid
> > > > > > > format,
> > > > > > > interpretation is not guaranteed. Looks like 5.8.0 in
> > > > > > > that
> > > > > > > case
> > > > > > > uses
> > > > > > > the
> > > > > > > timestamp of message reception. I suggest to use the
> > > > > > > current
> > > > > > > stable,
> > > > > > > I think
> > > > > > > it will work somewhat different. Bottom line is that
> > > > > > > auditd
> > > > > > > should
> > > > > > > emit the
> > > > > > > proper format.
> > > > > > >
> > > > > > > Rainer
> > > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > [mailto:rsyslog-
> > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> > > > > > > > Sent: Monday, July 18, 2011 8:26 AM
> > > > > > > > To: rsyslog-users
> > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > >
> > > > > > > > http://pastebin.com/TzPVzknt
> > > > > > > > The 2 line I have previously seen with hi-res times are
> > > > > > > > 15
> > > > > > > > and
> > > > > > > > 23
> > > > > > > >
> > > > > > > > ----- Original Message -----
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > McKee
> > > > > > > > > > Sent: Monday, July 18, 2011 8:11 AM
> > > > > > > > > > To: rsyslog-users
> > > > > > > > > > Subject: Re: [rsyslog] timereported:::date-rfc3339
> > > > > > > > > >
> > > > > > > > > > The following log has a restart of auditd and a ssh
> > > > > > > > > > connection
> > > > > > > > > > during
> > > > > > > > > > the debug run.
> > > > > > > > > > http://pastebin.com/cRPuA1Z8
> > > > > > > > >
> > > > > > > > > Thanks! Unfortunately, the instrumentation does not
> > > > > > > > > provide
> > > > > > > > > what
> > > > > > > > > I am
> > > > > > > > > looking
> > > > > > > > > for (maybe because of an older build, maybe it's just
> > > > > > > > > not
> > > > > > > > > there...).
> > > > > > > > > Can you
> > > > > > > > > please also write all messages to a file with
> > > > > > > > > RSYSLOG_DebugFormat
> > > > > > > > > and
> > > > > > > > > post
> > > > > > > > > that file.
> > > > > > > > >
> > > > > > > > > With 5.8.0, you should probably never see hires, so I
> > > > > > > > > am
> > > > > > > > > a
> > > > > > > > > bit
> > > > > > > > > puzzled. Maybe
> > > > > > > > > auditd does some "interesting" things to the log
> > > > > > > > > socket.
> > > > > > > > > Note
> > > > > > > > > that
> > > > > > > > > rsyslog
> > > > > > > > > expects syslog() API format, but older versions (like
> > > > > > > > > 5.8.0)
> > > > > > > > > did
> > > > > > > > > not
> > > > > > > > > enforce
> > > > > > > > > that.
> > > > > > > > >
> > > > > > > > > Rainer
> > > > > > > > > >
> > > > > > > > > > ----- Original Message -----
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Rodney
> > > > > > > > > > > > McKee
> > > > > > > > > > > > Sent: Monday, July 18, 2011 7:41 AM
> > > > > > > > > > > > To: rsyslog-users
> > > > > > > > > > > > Subject: Re: [rsyslog]
> > > > > > > > > > > > timereported:::date-rfc3339
> > > > > > > > > > > >
> > > > > > > > > > > > Wow, Rainer, thanks for the quick response.
> > > > > > > > > > > >
> > > > > > > > > > > > So on a local system some processes actually
> > > > > > > > > > > > provide a
> > > > > > > > > > > > high
> > > > > > > > > > > > res
> > > > > > > > > > > > time
> > > > > > > > > > > > that rsyslog then logs as %timereported%.
> > > > > > > > > > >
> > > > > > > > > > > As far as the local sockets is concerned, things
> > > > > > > > > > > should
> > > > > > > > > > > be
> > > > > > > > > > > consistent. If
> > > > > > > > > > > that's not the case, it is best if you provide a
> > > > > > > > > > > debug
> > > > > > > > > > > log --
> > > > > > > > > > > the
> > > > > > > > > > > log
> > > > > > > > > > > samples
> > > > > > > > > > > just show the result but now how we arrived there
> > > > > > > > > > > :)
> > > > > > > > > > >
> > > > > > > > > > > Rainer
> > > > > > > > > > >
> > > > > > > > > > > Did not realize this would be
> > > > > > > > > > > > happening. I guess that most clients then do
> > > > > > > > > > > > not
> > > > > > > > > > > > provide
> > > > > > > > > > > > the
> > > > > > > > > > > > hi-res
> > > > > > > > > > > > times and this might explain some messages
> > > > > > > > > > > > having
> > > > > > > > > > > > the
> > > > > > > > > > > > time
> > > > > > > > > > > > and
> > > > > > > > > > > > most
> > > > > > > > > > > > not:
> > > > > > > > > > > >
> > > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.702529+10:00  The audit daemon is
> > > > > > > > > > > > exiting.
> > > > > > > > > > > > Jul 18 14:27:10
> > > > > > > > > > > > 2011-07-18T14:27:10.703673+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.703673+10:00
> > > > > > > > > > > > audit(1310963230.693:4484770):
> > > > > > > > > > > > audit_pid=0
> > > > > > > > > > > > old=1773 by auid=4294967295
> > > > > > > > > > > > Jul 18 14:27:10
> > > > > > > > > > > > 2011-07-18T14:27:10.867738+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.867738+10:00
> > > > > > > > > > > > audit(1310963230.864:4484771):
> > > > > > > > > > > > auid=672
> > > > > > > > > > > > op=remove rule key=(null) list=2 res=1
> > > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.959443+10:00  Warning - freq is
> > > > > > > > > > > > non-zero
> > > > > > > > > > > > and
> > > > > > > > > > > > incremental
> > > > > > > > > > > > flushing not selected.
> > > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.978467+10:00  Started dispatcher:
> > > > > > > > > > > > /sbin/audispd
> > > > > > > > > > > > pid:
> > > > > > > > > > > > 4794
> > > > > > > > > > > > Jul 18 14:27:10
> > > > > > > > > > > > 2011-07-18T14:27:10.981061+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.981061+10:00
> > > > > > > > > > > > audit(1310963230.979:4484772):
> > > > > > > > > > > > audit_pid=4792
> > > > > > > > > > > > old=0 by auid=672
> > > > > > > > > > > > Jul 18 14:27:10 2011-07-18T14:27:10+10:00
> > > > > > > > > > > > 2011-07-
> > > > > > > > > > > > 18T14:27:10.998047+10:00  af_unix plugin
> > > > > > > > > > > > initialized
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > ----- Original Message -----
> > > > > > > > > > > > > > -----Original Message-----
> > > > > > > > > > > > > > From: rsyslog-bounces at lists.adiscon.com
> > > > > > > > > > > > > > [mailto:rsyslog-
> > > > > > > > > > > > > > bounces at lists.adiscon.com] On Behalf Of
> > > > > > > > > > > > > > Rodney
> > > > > > > > > > > > > > McKee
> > > > > > > > > > > > > > Sent: Monday, July 18, 2011 6:00 AM
> > > > > > > > > > > > > > To: rsyslog-users
> > > > > > > > > > > > > > Subject: [rsyslog]
> > > > > > > > > > > > > > timereported:::date-rfc3339
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > What effects the recording of milliseconds
> > > > > > > > > > > > > > when
> > > > > > > > > > > > > > using
> > > > > > > > > > > > > > timereported:::date-
> > > > > > > > > > > > > > rfc3339.
> > > > > > > > > > > > >
> > > > > > > > > > > > > This field contains what the sender told us.
> > > > > > > > > > > > > If
> > > > > > > > > > > > > the
> > > > > > > > > > > > > sender
> > > > > > > > > > > > > sent
> > > > > > > > > > > > > no
> > > > > > > > > > > > > ms, we can
> > > > > > > > > > > > > not report them. Rather than to pretend
> > > > > > > > > > > > > "x.000000"
> > > > > > > > > > > > > they
> > > > > > > > > > > > > are
> > > > > > > > > > > > > there, we
> > > > > > > > > > > > > do not
> > > > > > > > > > > > > give them. Note that for the same reason
> > > > > > > > > > > > > there
> > > > > > > > > > > > > may be
> > > > > > > > > > > > > sub-ms
> > > > > > > > > > > > > resolution, like
> > > > > > > > > > > > > us, if that is what the sender reported.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Note that starting with the latest v5-devel
> > > > > > > > > > > > > version
> > > > > > > > > > > > > AND a
> > > > > > > > > > > > > recent
> > > > > > > > > > > > > Linux
> > > > > > > > > > > > > kernel, we can ask the system for more
> > > > > > > > > > > > > precise
> > > > > > > > > > > > > timestamps
> > > > > > > > > > > > > on
> > > > > > > > > > > > > messages
> > > > > > > > > > > > > that
> > > > > > > > > > > > > come in via the log socket.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Rainer
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Some log entries get milliseconds and some
> > > > > > > > > > > > > > do
> > > > > > > > > > > > > > not:
> > > > > > > > > > > > > > The template:
> > > > > > > > > > > > > > "%TIMESTAMP% %timereported:::date-rfc3339%
> > > > > > > > > > > > > > %timegenerated:::date-
> > > > > > > > > > > > > > rfc3339% %msg%\n"
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > The output:
> > > > > > > > > > > > > > Jul 18 13:58:30 2011-07-18T13:58:30+10:00
> > > > > > > > > > > > > > 2011-07-18T13:58:30.723250+10:00
> > > > > > > > > > > > > > test
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Am I missing something.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Rgds
> > > > > > > > > > > > > > Rodney
> > > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > > > >
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > > > >
> > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > rsyslog mailing list
> > > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > rsyslog mailing list
> > > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > > http://www.rsyslog.com
> > > > > > > > > _______________________________________________
> > > > > > > > > rsyslog mailing list
> > > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > > http://www.rsyslog.com
> > > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > rsyslog mailing list
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > > http://www.rsyslog.com
> > > > > > > _______________________________________________
> > > > > > > rsyslog mailing list
> > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > > http://www.rsyslog.com
> > > > > > >
> > > > > > _______________________________________________
> > > > > > rsyslog mailing list
> > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > > http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From david at lang.hm  Wed Jul 20 06:03:11 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jul 2011 21:03:11 -0700 (PDT)
Subject: [rsyslog] remote logging from multiple timezones
In-Reply-To: <8ab3022e-b959-4b70-8011-152b8045d3e6@wsrmckee>
References: <8ab3022e-b959-4b70-8011-152b8045d3e6@wsrmckee>
Message-ID: <alpine.DEB.2.02.1107192057360.5355@asgard.lang.hm>

On Wed, 20 Jul 2011, Rodney McKee wrote:

> Collecting logs from multiple timezones and rotating on a central log server with a template like:
> $template DAILYSYSLOG,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log"
>
> means that logs from each timezone still get rotated at midnight on the localhost, would something like the following work to rotate logs for each timezones change of day
>
> $template TEST,"/var/log/logdir/%TIMESTAMP:1:4:date-rfc3339%/%TIMESTAMP:6:7:date-rfc3339%/%TIMESTAMP:9:10:date-rfc3339%/%HOSTNAME%/messages.log"
>
> or would it cause other issues like excessive load etc or some other unforeseen issue. I guess it the date format gets screwed it could have some impact.

I would be surprised if the difference in format made much of a difference 
in load, unless you are dealing with very high traffic volumes.

I suspect your bigger problem will be devices that don't send their 
timezone as part of their syslog message. up until a couple of years ago 
when they new RFC came out, the RFC for syslog said that the timestamp did 
not have a timezone, and even today, most systems don't send a timezone in 
their syslog messages.

At my company, we decided that the the timezone for all the servers in the 
company should be the same, no matter where in the world the system 
happened to live right now. In retrospect we should have gone one step 
further and set everything to GMT to completely avoid daylight savings 
headaches. But as we've purchased competitors over the years and seen them 
with different time zones on different servers in different offices (or 
worse, different time zones on different servers in the same office, 
because the people connecting to that server were expected to be in a 
particular timezone), the value of keeping every system we manage on the 
same timezone setting has been shown to be a _very_ good idea.

if you want to disply different times to the users, have your application 
adjust the time (and then it can adjust it based on where the user is, not 
where the server is)

David Lang

From rodney.mckee at gmail.com  Wed Jul 20 06:25:05 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Wed, 20 Jul 2011 14:25:05 +1000 (EST)
Subject: [rsyslog] remote logging from multiple timezones
In-Reply-To: <alpine.DEB.2.02.1107192057360.5355@asgard.lang.hm>
Message-ID: <f1b03072-9877-4f8a-9581-85c792db99c4@wsrmckee>

David,

----- Original Message -----
> On Wed, 20 Jul 2011, Rodney McKee wrote:
> 
> > Collecting logs from multiple timezones and rotating on a central
> > log server with a template like:
> > $template
> > DAILYSYSLOG,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log"
> >
> > means that logs from each timezone still get rotated at midnight on
> > the localhost, would something like the following work to rotate
> > logs for each timezones change of day
> >
> > $template
> > TEST,"/var/log/logdir/%TIMESTAMP:1:4:date-rfc3339%/%TIMESTAMP:6:7:date-rfc3339%/%TIMESTAMP:9:10:date-rfc3339%/%HOSTNAME%/messages.log"
> >
> > or would it cause other issues like excessive load etc or some
> > other unforeseen issue. I guess it the date format gets screwed it
> > could have some impact.
> 
> I would be surprised if the difference in format made much of a
> difference
> in load, unless you are dealing with very high traffic volumes.

I suspect once we start collecting our full application logs the traffic WILL be high, that's a bridge yet to be crossed.

> 
> I suspect your bigger problem will be devices that don't send their
> timezone as part of their syslog message. up until a couple of years
> ago
> when they new RFC came out, the RFC for syslog said that the
> timestamp did
> not have a timezone, and even today, most systems don't send a
> timezone in
> their syslog messages.

I must admit, have not started sending logs from "other" devices except for a token windows server and it looks fine. Considering the template is only breaking up the date component and not looking at time or timezone, I'm -) reasonably confidant. But I heed your warnings.

> 
> At my company, we decided that the the timezone for all the servers
> in the
> company should be the same, no matter where in the world the system
> happened to live right now. In retrospect we should have gone one
> step
> further and set everything to GMT to completely avoid daylight
> savings
> headaches. But as we've purchased competitors over the years and seen
> them
> with different time zones on different servers in different offices
> (or
> worse, different time zones on different servers in the same office,
> because the people connecting to that server were expected to be in a
> particular timezone), the value of keeping every system we manage on
> the
> same timezone setting has been shown to be a _very_ good idea.
> 
> if you want to disply different times to the users, have your
> application
> adjust the time (and then it can adjust it based on where the user
> is, not
> where the server is)

Interesting approach, comes back to good application developers in most cases I'm sure. We see similar issues but it's sorta locked in now.

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 

From david at lang.hm  Wed Jul 20 06:52:47 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jul 2011 21:52:47 -0700 (PDT)
Subject: [rsyslog] remote logging from multiple timezones
In-Reply-To: <f1b03072-9877-4f8a-9581-85c792db99c4@wsrmckee>
References: <f1b03072-9877-4f8a-9581-85c792db99c4@wsrmckee>
Message-ID: <alpine.DEB.2.02.1107192134170.5355@asgard.lang.hm>

On Wed, 20 Jul 2011, Rodney McKee wrote:

> David,
>
> ----- Original Message -----
>> On Wed, 20 Jul 2011, Rodney McKee wrote:
>>
>>> Collecting logs from multiple timezones and rotating on a central
>>> log server with a template like:
>>> $template
>>> DAILYSYSLOG,"/var/log/logdir/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%/messages.log"
>>>
>>> means that logs from each timezone still get rotated at midnight on
>>> the localhost, would something like the following work to rotate
>>> logs for each timezones change of day
>>>
>>> $template
>>> TEST,"/var/log/logdir/%TIMESTAMP:1:4:date-rfc3339%/%TIMESTAMP:6:7:date-rfc3339%/%TIMESTAMP:9:10:date-rfc3339%/%HOSTNAME%/messages.log"
>>>
>>> or would it cause other issues like excessive load etc or some
>>> other unforeseen issue. I guess it the date format gets screwed it
>>> could have some impact.
>>
>> I would be surprised if the difference in format made much of a
>> difference
>> in load, unless you are dealing with very high traffic volumes.
>
> I suspect once we start collecting our full application logs the traffic WILL be high, that's a bridge yet to be crossed.

by high I am talking about tens of thousands of logs per second.

If you are getting log volumes that high, you probably don't want to use 
dynafiles, no matter what the timestamp version.

>> At my company, we decided that the the timezone for all the servers
>> in the
>> company should be the same, no matter where in the world the system
>> happened to live right now. In retrospect we should have gone one
>> step
>> further and set everything to GMT to completely avoid daylight
>> savings
>> headaches. But as we've purchased competitors over the years and seen
>> them
>> with different time zones on different servers in different offices
>> (or
>> worse, different time zones on different servers in the same office,
>> because the people connecting to that server were expected to be in a
>> particular timezone), the value of keeping every system we manage on
>> the
>> same timezone setting has been shown to be a _very_ good idea.
>>
>> if you want to disply different times to the users, have your
>> application
>> adjust the time (and then it can adjust it based on where the user
>> is, not
>> where the server is)
>
> Interesting approach, comes back to good application developers in most 
> cases I'm sure. We see similar issues but it's sorta locked in now.

even if you have this problem now, make a big stink about the problems it 
causes and get the fix on the roadmap for the next release.

if you think about it, with the Internet, what are the odds that the only 
people who are hitting your server live in the same timezone that the 
server happens to be in. at $work, we provide services for Credit Unions, 
and even the smallest Credit Union has people who joined when they were 
local, but have since moved away. These people really are happier with the 
ability to set the timezone to where they live.

David Lang

From rodney.mckee at gmail.com  Thu Jul 21 06:52:26 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Thu, 21 Jul 2011 14:52:26 +1000 (EST)
Subject: [rsyslog] impstats details
In-Reply-To: <fa288e56-5bcc-435a-aa0a-f40b9cbe3c98@wsrmckee>
Message-ID: <2fd47f00-166d-4ff7-8e69-32ece420163d@wsrmckee>

Just wondering what doco is out their that will let me understand the metrics being emitted by impstats

imuxsock: submitted=1140 ratelimit.discarded=0 ratelimit.numratelimiters=432 
action 5 queue[DA]: size=0 enqueued=0 full=0 maxqsize=0 
action 5 queue: size=0 enqueued=947834 full=0 maxqsize=724 
main Q: size=3 enqueued=952342 full=0 maxqsize=5087 

imuxsock:
 submitted=                 # of log lines submitted via the Unix socket between the given sample period (instant value)
 ratelimit.discarded=       # of log lines discarded due to rate limiting between the given sample period (instant value)
 ratelimit.numratelimiters= ? Not sure about this one, I'm seeing a steadily increasing value for this.


action 5 queue[DA]:
or
action 5 queue:
or
main Q:
 size=      
 enqueued= 
 full=
 maxqsize=



Rgds
Rodney

From parthachowdhury78 at gmail.com  Sun Jul 24 15:09:09 2011
From: parthachowdhury78 at gmail.com (Partha Chowdhury)
Date: Sun, 24 Jul 2011 18:39:09 +0530
Subject: [rsyslog] To customize rsyslog timestamp format to show messages in
 12 hour format with am/pm
Message-ID: <4E2C1975.6010309@gmail.com>

hallo to everyone on the list.

I am using rsyslog version 5.8.0.

My /etc/rsyslog file is at [1]

With the above config, a typical message in message log looks like 
following:

> Jul 24 16:35:53 localhost ntpd[1279]: synchronized to 89.221.207.113, stratum 2

However i would like to modify it so that it will look like the following:


> Jul 24 04:35:53 PM localhost ntpd[1279]: synchronized to 89.221.207.113, stratum 2

I have read the rsyslog documentation and looked at different rsyslog 
config files via google, but i cannot seem to find any example nor can i 
find  any property replacer which does the above.

So, am i missing any obvious hint or is it not possible to do this ?

[1]http://pastebin.com/kQwR7taZ



From rgerhards at hq.adiscon.com  Sun Jul 24 16:36:51 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 24 Jul 2011 16:36:51 +0200
Subject: [rsyslog] To customize rsyslog timestamp format to show
	messages in 12 hour format with am/pm
In-Reply-To: <4E2C1975.6010309@gmail.com>
References: <4E2C1975.6010309@gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281089@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Partha Chowdhury
> Sent: Sunday, July 24, 2011 3:09 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] To customize rsyslog timestamp format to show
> messages in 12 hour format with am/pm
> 
> hallo to everyone on the list.
> 
> I am using rsyslog version 5.8.0.
> 
> My /etc/rsyslog file is at [1]
> 
> With the above config, a typical message in message log looks like
> following:
> 
> > Jul 24 16:35:53 localhost ntpd[1279]: synchronized to 89.221.207.113,
> stratum 2
> 
> However i would like to modify it so that it will look like the
> following:
> 
> 
> > Jul 24 04:35:53 PM localhost ntpd[1279]: synchronized to
> 89.221.207.113, stratum 2

There is no such format available as a canned format. Why is this important?

Rainer

> 
> I have read the rsyslog documentation and looked at different rsyslog
> config files via google, but i cannot seem to find any example nor can
> i
> find  any property replacer which does the above.
> 
> So, am i missing any obvious hint or is it not possible to do this ?
> 
> [1]http://pastebin.com/kQwR7taZ
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From parthachowdhury78 at gmail.com  Mon Jul 25 02:46:16 2011
From: parthachowdhury78 at gmail.com (Partha Chowdhury)
Date: Mon, 25 Jul 2011 06:16:16 +0530
Subject: [rsyslog] To customize rsyslog timestamp format to show
 messages in 12 hour format with am/pm
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281089@GRFEXC.intern.adiscon.com>
References: <4E2C1975.6010309@gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281089@GRFEXC.intern.adiscon.com>
Message-ID: <4E2CBCD8.80306@gmail.com>

On 24/07/11 08:06 PM, Rainer Gerhards wrote:
>>
>> However i would like to modify it so that it will look like the
>> following:
>>
>>
>>> Jul 24 04:35:53 PM localhost ntpd[1279]: synchronized to
>> 89.221.207.113, stratum 2
>
> There is no such format available as a canned format.

Yesterday I came across the syslog protocol at link at [1] via google 
while searching. After reading through the draft, I could not find my 
desired format.So then is it not possible to have a 12-hour format in 
the timestamp field ?

[1] http://tools.ietf.org/html/rfc5424

>  Why is this important?

Just my personal preference.For me it makes reading the log more 
comfortable and quicker.

From aoz.syn at gmail.com  Mon Jul 25 16:47:24 2011
From: aoz.syn at gmail.com (RB)
Date: Mon, 25 Jul 2011 08:47:24 -0600
Subject: [rsyslog] To customize rsyslog timestamp format to show
 messages in 12 hour format with am/pm
In-Reply-To: <4E2CBCD8.80306@gmail.com>
References: <4E2C1975.6010309@gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281089@GRFEXC.intern.adiscon.com>
	<4E2CBCD8.80306@gmail.com>
Message-ID: <CADkMHCkGoz=neAzF=9x8iTkDXd=L_K+R5Raq6mVTj4U9i09oRg@mail.gmail.com>

On Sun, Jul 24, 2011 at 18:46, Partha Chowdhury
<parthachowdhury78 at gmail.com> wrote:
> Just my personal preference.For me it makes reading the log more comfortable
> and quicker.

24-hour timestamps are less ambiguous and far more easily sorted.  If
you're really that uncomfortable with the 24-hour timestamp, you
should find a presentation layer (splunk, simple Perl filter, etc.)
that will show you what you want to see while leaving the stored data
in the more precise format.

From a.chapellon at horoa.net  Tue Jul 26 11:12:43 2011
From: a.chapellon at horoa.net (Alexandre Chapellon)
Date: Tue, 26 Jul 2011 11:12:43 +0200
Subject: [rsyslog] forwarding malformated syslog messages.
Message-ID: <4E2E850B.8090907@horoa.net>

Hello,

I have to relay syslog messages from some locked-up/proprietary boxes 
(Mirapoint mail servers). To achieve this I am using rsyslog from the 
debian squeeze packages: rsyslogd 4.6.4.
Messages are sent by the boxes using UDP protocol (no choice here), and 
must be relayed to a "home server" using RELP.
My problem is that messages from the boxes are fairly malformed. The 
syslog-tag field largely exceed the 32 chars defined in the RFC 3164 (i 
did not checked if this has been updated), and so I belive they just 
started the put the MSG in the syslog TAG field.
As a consequence, when forwarding thoose ugly messages, rsyslogd 
truncate the syslogtag (which is in fact the message itself) to fit in 
32 char field.

Does newer version of rsyslog would handle thoose kind of messages?
Is there any way to use template to properly "re-construct" the mesages 
prior to forwarding?

If you want to deeper look at it, I attached 2 dumps of such packets. 
First one (mirapoint.pcap) is the yslog packet as sent by the boxes, 
second one (rsyslog.pcap) is the message as forwarded to "home server" 
by rsyslog.

-- 
<http://www.horoa.net>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: syslog-forward.tar.gz
Type: application/x-gzip
Size: 1434 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110726/110e0346/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a_chapellon.vcf
Type: text/x-vcard
Size: 373 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110726/110e0346/attachment.vcf>

From david at lang.hm  Wed Jul 27 02:47:05 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 26 Jul 2011 17:47:05 -0700 (PDT)
Subject: [rsyslog] forwarding malformated syslog messages.
In-Reply-To: <4E2E850B.8090907@horoa.net>
References: <4E2E850B.8090907@horoa.net>
Message-ID: <alpine.DEB.2.02.1107261746010.24595@asgard.lang.hm>

I don't think there's anything that can be done in the templates to fix 
this, but I think that it would not be that hard to create a parser module 
that would clean things up.

David Lang

  On Tue, 26 Jul 2011, Alexandre Chapellon 
wrote:

> Hello,
>
> I have to relay syslog messages from some locked-up/proprietary boxes 
> (Mirapoint mail servers). To achieve this I am using rsyslog from the debian 
> squeeze packages: rsyslogd 4.6.4.
> Messages are sent by the boxes using UDP protocol (no choice here), and must 
> be relayed to a "home server" using RELP.
> My problem is that messages from the boxes are fairly malformed. The 
> syslog-tag field largely exceed the 32 chars defined in the RFC 3164 (i did 
> not checked if this has been updated), and so I belive they just started the 
> put the MSG in the syslog TAG field.
> As a consequence, when forwarding thoose ugly messages, rsyslogd truncate the 
> syslogtag (which is in fact the message itself) to fit in 32 char field.
>
> Does newer version of rsyslog would handle thoose kind of messages?
> Is there any way to use template to properly "re-construct" the mesages prior 
> to forwarding?
>
> If you want to deeper look at it, I attached 2 dumps of such packets. First 
> one (mirapoint.pcap) is the yslog packet as sent by the boxes, second one 
> (rsyslog.pcap) is the message as forwarded to "home server" by rsyslog.
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: syslog-forward.tar.gz
Type: application/x-gzip
Size: 1434 bytes
Desc: 
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110726/f3d310db/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a_chapellon.vcf
Type: text/x-vcard
Size: 373 bytes
Desc: 
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110726/f3d310db/attachment.vcf>
-------------- next part --------------
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Wed Jul 27 07:24:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jul 2011 07:24:24 +0200
Subject: [rsyslog] forwarding malformated syslog messages.
In-Reply-To: <alpine.DEB.2.02.1107261746010.24595@asgard.lang.hm>
References: <4E2E850B.8090907@horoa.net>
	<alpine.DEB.2.02.1107261746010.24595@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72810A9@GRFEXC.intern.adiscon.com>

Depending on the neeeds, it may be sufficient to use %rawmsg% for
forwarding...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, July 27, 2011 2:47 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] forwarding malformated syslog messages.
> 
> I don't think there's anything that can be done in the templates to fix
> this, but I think that it would not be that hard to create a parser
> module
> that would clean things up.
> 
> David Lang
> 
>   On Tue, 26 Jul 2011, Alexandre Chapellon
> wrote:
> 
> > Hello,
> >
> > I have to relay syslog messages from some locked-up/proprietary boxes
> > (Mirapoint mail servers). To achieve this I am using rsyslog from the
> debian
> > squeeze packages: rsyslogd 4.6.4.
> > Messages are sent by the boxes using UDP protocol (no choice here),
> and must
> > be relayed to a "home server" using RELP.
> > My problem is that messages from the boxes are fairly malformed. The
> > syslog-tag field largely exceed the 32 chars defined in the RFC 3164
> (i did
> > not checked if this has been updated), and so I belive they just
> started the
> > put the MSG in the syslog TAG field.
> > As a consequence, when forwarding thoose ugly messages, rsyslogd
> truncate the
> > syslogtag (which is in fact the message itself) to fit in 32 char
> field.
> >
> > Does newer version of rsyslog would handle thoose kind of messages?
> > Is there any way to use template to properly "re-construct" the
> mesages prior
> > to forwarding?
> >
> > If you want to deeper look at it, I attached 2 dumps of such packets.
> First
> > one (mirapoint.pcap) is the yslog packet as sent by the boxes, second
> one
> > (rsyslog.pcap) is the message as forwarded to "home server" by
> rsyslog.
> >
> >

From a.chapellon at horoa.net  Wed Jul 27 09:56:11 2011
From: a.chapellon at horoa.net (Alexandre Chapellon)
Date: Wed, 27 Jul 2011 09:56:11 +0200
Subject: [rsyslog] forwarding malformated syslog messages.
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72810A9@GRFEXC.intern.adiscon.com>
References: <4E2E850B.8090907@horoa.net>
	<alpine.DEB.2.02.1107261746010.24595@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA72810A9@GRFEXC.intern.adiscon.com>
Message-ID: <4E2FC49B.70907@horoa.net>

I finally got it working by using a modified version of the default 
forwarding template:

/$template buggyMirapointSyslogtag, "<%PRI%>%TIMESTAMP:::date-rfc3339% 
%HOSTNAME% %syslogtag% %msg%" /

Initially I used the '/sp-if-no-1st-sp/' options as shown in the default 
template but removed it as it produced really weired behaviour.
As I use this template only for messages from mirapoint boxes, I 
supposed it is not that important... Am I right?

regards.

Le 27/07/2011 07:24, Rainer Gerhards a ?crit :
> Depending on the neeeds, it may be sufficient to use %rawmsg% for
> forwarding...
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Wednesday, July 27, 2011 2:47 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] forwarding malformated syslog messages.
>>
>> I don't think there's anything that can be done in the templates to fix
>> this, but I think that it would not be that hard to create a parser
>> module
>> that would clean things up.
>>
>> David Lang
>>
>>    On Tue, 26 Jul 2011, Alexandre Chapellon
>> wrote:
>>
>>> Hello,
>>>
>>> I have to relay syslog messages from some locked-up/proprietary boxes
>>> (Mirapoint mail servers). To achieve this I am using rsyslog from the
>> debian
>>> squeeze packages: rsyslogd 4.6.4.
>>> Messages are sent by the boxes using UDP protocol (no choice here),
>> and must
>>> be relayed to a "home server" using RELP.
>>> My problem is that messages from the boxes are fairly malformed. The
>>> syslog-tag field largely exceed the 32 chars defined in the RFC 3164
>> (i did
>>> not checked if this has been updated), and so I belive they just
>> started the
>>> put the MSG in the syslog TAG field.
>>> As a consequence, when forwarding thoose ugly messages, rsyslogd
>> truncate the
>>> syslogtag (which is in fact the message itself) to fit in 32 char
>> field.
>>> Does newer version of rsyslog would handle thoose kind of messages?
>>> Is there any way to use template to properly "re-construct" the
>> mesages prior
>>> to forwarding?
>>>
>>> If you want to deeper look at it, I attached 2 dumps of such packets.
>> First
>>> one (mirapoint.pcap) is the yslog packet as sent by the boxes, second
>> one
>>> (rsyslog.pcap) is the message as forwarded to "home server" by
>> rsyslog.
>>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

-- 
<http://www.horoa.net>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: a_chapellon.vcf
Type: text/x-vcard
Size: 373 bytes
Desc: not available
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20110727/b0cec025/attachment.vcf>

From alanwevans at gmail.com  Thu Jul 28 21:21:10 2011
From: alanwevans at gmail.com (Alan Evans)
Date: Thu, 28 Jul 2011 15:21:10 -0400
Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
Message-ID: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>

I am attempting to configure rsyslog to queue messages while the
destination server is down but I am finding it is only queuing approx
1 message per second and we receive a whole lot more than that.

We have many hosts that send messages to a 'proxy' if you will at a
location which then in turn forwards it to our central log server for
archival.  We are going to have an outage on the central log server
soon so I am trying to replace syslog-ng with rsyslog so the
intermediates can queue messages while the central server is down.

I am using 3.22.1 packaged with RHEL 5 which I understand is old but I
will not be able to use a newer version.  I have followed the setup at
http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html but am not
getting the expected results.

Here's my config.

# Globals
$MaxMessageSize 8k
$MainMsgQueueType LinkedList

# Use traditional timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Provides kernel logging support (previously done by rklogd)
$ModLoad imklog
# Provides support for local system logging (e.g. via logger command)
$ModLoad imuxsock
$ModLoad imtcp.so

$InputTCPMaxSessions 1024
$InputTCPServerRun 5142

$WorkDirectory /var/spool/rsyslog # where to place spool files
$ActionQueueType LinkedList   # run asynchronously
$ActionQueueFileName remotqeque # unique name prefix for spool files
$ActionResumeRetryCount -1    # infinite retries if host is down
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueMaxDiskSpace 2g   # 1gb space limit (use as much as possible)
$ActionQueueDequeueSlowdown 0
*.*                                                     @@redacted:51422

To get logs to rsyslog I have configured syslog-ng to deliver to both
the central syslog server and to rsyslog on the intermediate.  Rsyslog
is sending to redacted:51422 which is not actually listening (since I
don't really want to double my messages to the central server) which
simulates the central server being offline.

In practice I am seeing in the debug rsyslog process about one log
entry per second which it is sending to my disk assisted queue.
Syslog-ng however is receiving thousands of messages per minute.

If I get rid of the action queue and just log to a file rsyslog keeps
up with syslog-ng no problem.

Here's a snippet from the debug log.

2402.830836385:main queue:Reg/w0: action 1 queue: enqueueMsg: cond
timeout, dropping message!
2402.830862669:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
2402.830868074:main queue:Reg/w0: action 1 queue: EnqueueMsg advised
worker start
2402.830879658:main queue:Reg/w0: main queue: entering rate limiter
2402.830887901:main queue:Reg/w0: main queue: entry deleted, state 0,
size now 7385 entries
2402.830897655:main queue:Reg/w0: Called action, logging to builtin-fwd
2402.830903570:main queue:Reg/w0: action 1 queue: enqueueMsg:
LightDelay mark reached for light delayable message - blocking a bit.
2403.059393630:imuxsock.c: Message from UNIX socket: #6
2403.059418695:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
message content redacted
2403.059424142:imuxsock.c: Message has legacy syslog format.
2403.059434733:imuxsock.c: main queue: entry added, size now 7386 entries
2403.059441696:imuxsock.c: wtpAdviseMaxWorkers signals busy
2403.059446737:imuxsock.c: main queue: EnqueueMsg advised worker start
2403.059453751:imuxsock.c: --------imuxsock calling select, active
file descriptors (max 6): 6
2403.087373110:imuxsock.c: Message from UNIX socket: #6
2403.087416859:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
message content redacted
2403.087422226:imuxsock.c: Message has legacy syslog format.
2403.087432941:imuxsock.c: main queue: entry added, size now 7387 entries
2403.087439847:imuxsock.c: wtpAdviseMaxWorkers signals busy
2403.087444749:imuxsock.c: main queue: EnqueueMsg advised worker start
2403.087451626:imuxsock.c: --------imuxsock calling select, active
file descriptors (max 6): 6
2403.088771548:imuxsock.c: Message from UNIX socket: #6
2403.088785375:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
message content redacted
2403.088790500:imuxsock.c: Message has legacy syslog format.
2403.088797998:imuxsock.c: main queue: entry added, size now 7388 entries
2403.088804299:imuxsock.c: wtpAdviseMaxWorkers signals busy
2403.088809079:imuxsock.c: main queue: EnqueueMsg advised worker start
2403.088826047:imuxsock.c: --------imuxsock calling select, active
file descriptors (max 6): 6
2403.113614170:imuxsock.c: Message from UNIX socket: #6
2403.113659621:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
message content redacted
2403.113675071:imuxsock.c: main queue: entry added, size now 7389 entries
2403.113681913:imuxsock.c: wtpAdviseMaxWorkers signals busy
2403.113687091:imuxsock.c: main queue: EnqueueMsg advised worker start
2403.113693974:imuxsock.c: --------imuxsock calling select, active
file descriptors (max 6): 6
2403.709635421:imtcp.c: main queue: entry added, size now 7390 entries
2403.709660157:imtcp.c: wtpAdviseMaxWorkers signals busy
2403.709665565:imtcp.c: main queue: EnqueueMsg advised worker start
2403.709683772:imtcp.c: logmsg: flags 0, from 'hostnameredacted',
message content redacted
2403.709688841:imtcp.c: Message has legacy syslog format.
2403.709697186:imtcp.c: main queue: enqueueMsg: LightDelay mark
reached for light delayable message - blocking a bit.
2403.832590271:main queue:Reg/w0: action 1 queue: enqueueMsg: queue
FULL - waiting to drain.
2404.712385305:imtcp.c: main queue: entry added, size now 7391 entries
2404.712414709:imtcp.c: wtpAdviseMaxWorkers signals busy

Lines containing "cond timeout, dropping message!" and "queue FULL -
waiting to drain." have me concerned.  But it doesn't look like all of
my logs are making it to the main queue anyway.  Is imtcp dropping
messages and the debug level is not high enough to see it maybe?

Please help!  How do I get rsyslog to queue all the messages bound for
the simulated down central log host?

Regards,
-Alan

From david at lang.hm  Thu Jul 28 21:28:06 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 28 Jul 2011 12:28:06 -0700 (PDT)
Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
In-Reply-To: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>
References: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1107281224590.25283@asgard.lang.hm>

If you are sticking with rsyslog 3.x because that's the only thing 
supported by redhat, I think that you will need to ask redhat for support. 
(or buy a support agreement). v3 isn't supported anymore due to it's age 
and how much ahs changed (they are up to version 6 now)

there have been problems in the queuing, even in far more recent versions, 
so I suspect that that version is just buggy in that area.

note that this isn't an official answer (I don't work for adiscon), but 
supporting such an old version is not easy.

David Lang



  On Thu, 28 Jul 2011, Alan Evans wrote:

> Date: Thu, 28 Jul 2011 15:21:10 -0400
> From: Alan Evans <alanwevans at gmail.com>
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
> 
> I am attempting to configure rsyslog to queue messages while the
> destination server is down but I am finding it is only queuing approx
> 1 message per second and we receive a whole lot more than that.
>
> We have many hosts that send messages to a 'proxy' if you will at a
> location which then in turn forwards it to our central log server for
> archival.  We are going to have an outage on the central log server
> soon so I am trying to replace syslog-ng with rsyslog so the
> intermediates can queue messages while the central server is down.
>
> I am using 3.22.1 packaged with RHEL 5 which I understand is old but I
> will not be able to use a newer version.  I have followed the setup at
> http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html but am not
> getting the expected results.
>
> Here's my config.
>
> # Globals
> $MaxMessageSize 8k
> $MainMsgQueueType LinkedList
>
> # Use traditional timestamp format
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>
> # Provides kernel logging support (previously done by rklogd)
> $ModLoad imklog
> # Provides support for local system logging (e.g. via logger command)
> $ModLoad imuxsock
> $ModLoad imtcp.so
>
> $InputTCPMaxSessions 1024
> $InputTCPServerRun 5142
>
> $WorkDirectory /var/spool/rsyslog # where to place spool files
> $ActionQueueType LinkedList   # run asynchronously
> $ActionQueueFileName remotqeque # unique name prefix for spool files
> $ActionResumeRetryCount -1    # infinite retries if host is down
> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
> $ActionQueueMaxDiskSpace 2g   # 1gb space limit (use as much as possible)
> $ActionQueueDequeueSlowdown 0
> *.*                                                     @@redacted:51422
>
> To get logs to rsyslog I have configured syslog-ng to deliver to both
> the central syslog server and to rsyslog on the intermediate.  Rsyslog
> is sending to redacted:51422 which is not actually listening (since I
> don't really want to double my messages to the central server) which
> simulates the central server being offline.
>
> In practice I am seeing in the debug rsyslog process about one log
> entry per second which it is sending to my disk assisted queue.
> Syslog-ng however is receiving thousands of messages per minute.
>
> If I get rid of the action queue and just log to a file rsyslog keeps
> up with syslog-ng no problem.
>
> Here's a snippet from the debug log.
>
> 2402.830836385:main queue:Reg/w0: action 1 queue: enqueueMsg: cond
> timeout, dropping message!
> 2402.830862669:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
> 2402.830868074:main queue:Reg/w0: action 1 queue: EnqueueMsg advised
> worker start
> 2402.830879658:main queue:Reg/w0: main queue: entering rate limiter
> 2402.830887901:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 7385 entries
> 2402.830897655:main queue:Reg/w0: Called action, logging to builtin-fwd
> 2402.830903570:main queue:Reg/w0: action 1 queue: enqueueMsg:
> LightDelay mark reached for light delayable message - blocking a bit.
> 2403.059393630:imuxsock.c: Message from UNIX socket: #6
> 2403.059418695:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.059424142:imuxsock.c: Message has legacy syslog format.
> 2403.059434733:imuxsock.c: main queue: entry added, size now 7386 entries
> 2403.059441696:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.059446737:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.059453751:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.087373110:imuxsock.c: Message from UNIX socket: #6
> 2403.087416859:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.087422226:imuxsock.c: Message has legacy syslog format.
> 2403.087432941:imuxsock.c: main queue: entry added, size now 7387 entries
> 2403.087439847:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.087444749:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.087451626:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.088771548:imuxsock.c: Message from UNIX socket: #6
> 2403.088785375:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.088790500:imuxsock.c: Message has legacy syslog format.
> 2403.088797998:imuxsock.c: main queue: entry added, size now 7388 entries
> 2403.088804299:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.088809079:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.088826047:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.113614170:imuxsock.c: Message from UNIX socket: #6
> 2403.113659621:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.113675071:imuxsock.c: main queue: entry added, size now 7389 entries
> 2403.113681913:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.113687091:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.113693974:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.709635421:imtcp.c: main queue: entry added, size now 7390 entries
> 2403.709660157:imtcp.c: wtpAdviseMaxWorkers signals busy
> 2403.709665565:imtcp.c: main queue: EnqueueMsg advised worker start
> 2403.709683772:imtcp.c: logmsg: flags 0, from 'hostnameredacted',
> message content redacted
> 2403.709688841:imtcp.c: Message has legacy syslog format.
> 2403.709697186:imtcp.c: main queue: enqueueMsg: LightDelay mark
> reached for light delayable message - blocking a bit.
> 2403.832590271:main queue:Reg/w0: action 1 queue: enqueueMsg: queue
> FULL - waiting to drain.
> 2404.712385305:imtcp.c: main queue: entry added, size now 7391 entries
> 2404.712414709:imtcp.c: wtpAdviseMaxWorkers signals busy
>
> Lines containing "cond timeout, dropping message!" and "queue FULL -
> waiting to drain." have me concerned.  But it doesn't look like all of
> my logs are making it to the main queue anyway.  Is imtcp dropping
> messages and the debug level is not high enough to see it maybe?
>
> Please help!  How do I get rsyslog to queue all the messages bound for
> the simulated down central log host?
>
> Regards,
> -Alan
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From alanwevans at gmail.com  Fri Jul 29 00:35:09 2011
From: alanwevans at gmail.com (Alan Evans)
Date: Thu, 28 Jul 2011 18:35:09 -0400
Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
In-Reply-To: <alpine.DEB.2.02.1107281224590.25283@asgard.lang.hm>
References: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>
	<alpine.DEB.2.02.1107281224590.25283@asgard.lang.hm>
Message-ID: <CAMFVOoXgT4JAp3GK0YgL9PLYmn+2puHf-qyY3xi9pu1FjPk9bg@mail.gmail.com>

I've made some progress.

It seems it has to do with the main queue and the action queue and
their interaction.

If I set the mainqueue to something large, say 20,000 or 100,000
everything seems to be spooling to disk and the main queue floats at
about 8000 entries (the default high water mark IIRC).  If I set the
main queue to 10,000 it floats at about 7,000 entries but messages
seem to be getting dropped.

Does this make any sense to anyone?

-Alan

On Thu, Jul 28, 2011 at 3:28 PM,  <david at lang.hm> wrote:
> If you are sticking with rsyslog 3.x because that's the only thing supported
> by redhat, I think that you will need to ask redhat for support. (or buy a
> support agreement). v3 isn't supported anymore due to it's age and how much
> ahs changed (they are up to version 6 now)
>
> there have been problems in the queuing, even in far more recent versions,
> so I suspect that that version is just buggy in that area.
>
> note that this isn't an official answer (I don't work for adiscon), but
> supporting such an old version is not easy.
>
> David Lang
>
>
>
> ?On Thu, 28 Jul 2011, Alan Evans wrote:
>
>> Date: Thu, 28 Jul 2011 15:21:10 -0400
>> From: Alan Evans <alanwevans at gmail.com>
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
>>
>> I am attempting to configure rsyslog to queue messages while the
>> destination server is down but I am finding it is only queuing approx
>> 1 message per second and we receive a whole lot more than that.
>>
>> We have many hosts that send messages to a 'proxy' if you will at a
>> location which then in turn forwards it to our central log server for
>> archival. ?We are going to have an outage on the central log server
>> soon so I am trying to replace syslog-ng with rsyslog so the
>> intermediates can queue messages while the central server is down.
>>
>> I am using 3.22.1 packaged with RHEL 5 which I understand is old but I
>> will not be able to use a newer version. ?I have followed the setup at
>> http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html but am not
>> getting the expected results.
>>
>> Here's my config.
>>
>> # Globals
>> $MaxMessageSize 8k
>> $MainMsgQueueType LinkedList
>>
>> # Use traditional timestamp format
>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>>
>> # Provides kernel logging support (previously done by rklogd)
>> $ModLoad imklog
>> # Provides support for local system logging (e.g. via logger command)
>> $ModLoad imuxsock
>> $ModLoad imtcp.so
>>
>> $InputTCPMaxSessions 1024
>> $InputTCPServerRun 5142
>>
>> $WorkDirectory /var/spool/rsyslog # where to place spool files
>> $ActionQueueType LinkedList ? # run asynchronously
>> $ActionQueueFileName remotqeque # unique name prefix for spool files
>> $ActionResumeRetryCount -1 ? ?# infinite retries if host is down
>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
>> $ActionQueueMaxDiskSpace 2g ? # 1gb space limit (use as much as possible)
>> $ActionQueueDequeueSlowdown 0
>> *.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? @@redacted:51422
>>
>> To get logs to rsyslog I have configured syslog-ng to deliver to both
>> the central syslog server and to rsyslog on the intermediate. ?Rsyslog
>> is sending to redacted:51422 which is not actually listening (since I
>> don't really want to double my messages to the central server) which
>> simulates the central server being offline.
>>
>> In practice I am seeing in the debug rsyslog process about one log
>> entry per second which it is sending to my disk assisted queue.
>> Syslog-ng however is receiving thousands of messages per minute.
>>
>> If I get rid of the action queue and just log to a file rsyslog keeps
>> up with syslog-ng no problem.
>>
>> Here's a snippet from the debug log.
>>
>> 2402.830836385:main queue:Reg/w0: action 1 queue: enqueueMsg: cond
>> timeout, dropping message!
>> 2402.830862669:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
>> 2402.830868074:main queue:Reg/w0: action 1 queue: EnqueueMsg advised
>> worker start
>> 2402.830879658:main queue:Reg/w0: main queue: entering rate limiter
>> 2402.830887901:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 7385 entries
>> 2402.830897655:main queue:Reg/w0: Called action, logging to builtin-fwd
>> 2402.830903570:main queue:Reg/w0: action 1 queue: enqueueMsg:
>> LightDelay mark reached for light delayable message - blocking a bit.
>> 2403.059393630:imuxsock.c: Message from UNIX socket: #6
>> 2403.059418695:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>> message content redacted
>> 2403.059424142:imuxsock.c: Message has legacy syslog format.
>> 2403.059434733:imuxsock.c: main queue: entry added, size now 7386 entries
>> 2403.059441696:imuxsock.c: wtpAdviseMaxWorkers signals busy
>> 2403.059446737:imuxsock.c: main queue: EnqueueMsg advised worker start
>> 2403.059453751:imuxsock.c: --------imuxsock calling select, active
>> file descriptors (max 6): 6
>> 2403.087373110:imuxsock.c: Message from UNIX socket: #6
>> 2403.087416859:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>> message content redacted
>> 2403.087422226:imuxsock.c: Message has legacy syslog format.
>> 2403.087432941:imuxsock.c: main queue: entry added, size now 7387 entries
>> 2403.087439847:imuxsock.c: wtpAdviseMaxWorkers signals busy
>> 2403.087444749:imuxsock.c: main queue: EnqueueMsg advised worker start
>> 2403.087451626:imuxsock.c: --------imuxsock calling select, active
>> file descriptors (max 6): 6
>> 2403.088771548:imuxsock.c: Message from UNIX socket: #6
>> 2403.088785375:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>> message content redacted
>> 2403.088790500:imuxsock.c: Message has legacy syslog format.
>> 2403.088797998:imuxsock.c: main queue: entry added, size now 7388 entries
>> 2403.088804299:imuxsock.c: wtpAdviseMaxWorkers signals busy
>> 2403.088809079:imuxsock.c: main queue: EnqueueMsg advised worker start
>> 2403.088826047:imuxsock.c: --------imuxsock calling select, active
>> file descriptors (max 6): 6
>> 2403.113614170:imuxsock.c: Message from UNIX socket: #6
>> 2403.113659621:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>> message content redacted
>> 2403.113675071:imuxsock.c: main queue: entry added, size now 7389 entries
>> 2403.113681913:imuxsock.c: wtpAdviseMaxWorkers signals busy
>> 2403.113687091:imuxsock.c: main queue: EnqueueMsg advised worker start
>> 2403.113693974:imuxsock.c: --------imuxsock calling select, active
>> file descriptors (max 6): 6
>> 2403.709635421:imtcp.c: main queue: entry added, size now 7390 entries
>> 2403.709660157:imtcp.c: wtpAdviseMaxWorkers signals busy
>> 2403.709665565:imtcp.c: main queue: EnqueueMsg advised worker start
>> 2403.709683772:imtcp.c: logmsg: flags 0, from 'hostnameredacted',
>> message content redacted
>> 2403.709688841:imtcp.c: Message has legacy syslog format.
>> 2403.709697186:imtcp.c: main queue: enqueueMsg: LightDelay mark
>> reached for light delayable message - blocking a bit.
>> 2403.832590271:main queue:Reg/w0: action 1 queue: enqueueMsg: queue
>> FULL - waiting to drain.
>> 2404.712385305:imtcp.c: main queue: entry added, size now 7391 entries
>> 2404.712414709:imtcp.c: wtpAdviseMaxWorkers signals busy
>>
>> Lines containing "cond timeout, dropping message!" and "queue FULL -
>> waiting to drain." have me concerned. ?But it doesn't look like all of
>> my logs are making it to the main queue anyway. ?Is imtcp dropping
>> messages and the debug level is not high enough to see it maybe?
>>
>> Please help! ?How do I get rsyslog to queue all the messages bound for
>> the simulated down central log host?
>>
>> Regards,
>> -Alan
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>

From alanwevans at gmail.com  Fri Jul 29 01:51:26 2011
From: alanwevans at gmail.com (Alan Evans)
Date: Thu, 28 Jul 2011 19:51:26 -0400
Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
In-Reply-To: <CAMFVOoXgT4JAp3GK0YgL9PLYmn+2puHf-qyY3xi9pu1FjPk9bg@mail.gmail.com>
References: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>
	<alpine.DEB.2.02.1107281224590.25283@asgard.lang.hm>
	<CAMFVOoXgT4JAp3GK0YgL9PLYmn+2puHf-qyY3xi9pu1FjPk9bg@mail.gmail.com>
Message-ID: <CAMFVOoXB2FdK=j3Wi6Z_+cTWiftgD5Jn4aE=iQSnPkqFFRK35g@mail.gmail.com>

Ok even more details.

The below statements are only true if I change the main message queue
to a disk backed LinkedList queue.

The following config gives me a working setup.  I get log entries
spooled to disk and if the destination becomes available everything is
dequeued nicely.  BUT the mainqueue is bound to disk now...  What
gives?

Here's the config.

$MaxMessageSize 8k
$RepeatedMsgReduction off

# Use traditional timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Provides kernel logging support (previously done by rklogd)
$ModLoad imklog
# Provides support for local system logging (e.g. via logger command)
$ModLoad imuxsock
$ModLoad imtcp.so
$InputTCPFlowControl off
#$OMFileAsyncWriting on

$MainMsgQueueFileName mainqueue
$MainMsgQueueType LinkedList
$MainMsgQueueSaveOnShutdown on
$MainMsgQueueSize 100000

$InputTCPMaxSessions 1024
$InputTCPFlowControl off
$InputTCPServerRun 5142

$WorkDirectory /var/spool/rsyslog # where to place spool files
$ActionQueueType LinkedList   # run asynchronously
$ActionQueueFileName remotqeque # unique name prefix for spool files
$ActionResumeRetryCount -1    # infinite retries if host is down
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueMaxFileSize 100m
$ActionQueueMaxDiskSpace 19g  # 1gb space limit (use as much as possible)
$ActionQueueDequeueSlowdown 0
$ActionQueueWorkerThreads 4
*.*     @@127.0.0.1:51422

If I watch the debug with something like: tail -f
/tmp/rsyslog-debug.log | awk  '/main queue: entry added, size now/ {
print $8 }'

I see that the queue has right around 8000 messages in it.  Then if I
look on disk I am getting 100M remotequeue.00000000 files but my
mainqueue.0000000 files fluctuate between 0 and 700K on average I
would say.

Regards,
-Alan

On Thu, Jul 28, 2011 at 6:35 PM, Alan Evans <alanwevans at gmail.com> wrote:
> I've made some progress.
>
> It seems it has to do with the main queue and the action queue and
> their interaction.
>
> If I set the mainqueue to something large, say 20,000 or 100,000
> everything seems to be spooling to disk and the main queue floats at
> about 8000 entries (the default high water mark IIRC). ?If I set the
> main queue to 10,000 it floats at about 7,000 entries but messages
> seem to be getting dropped.
>
> Does this make any sense to anyone?
>
> -Alan
>
> On Thu, Jul 28, 2011 at 3:28 PM, ?<david at lang.hm> wrote:
>> If you are sticking with rsyslog 3.x because that's the only thing supported
>> by redhat, I think that you will need to ask redhat for support. (or buy a
>> support agreement). v3 isn't supported anymore due to it's age and how much
>> ahs changed (they are up to version 6 now)
>>
>> there have been problems in the queuing, even in far more recent versions,
>> so I suspect that that version is just buggy in that area.
>>
>> note that this isn't an official answer (I don't work for adiscon), but
>> supporting such an old version is not easy.
>>
>> David Lang
>>
>>
>>
>> ?On Thu, 28 Jul 2011, Alan Evans wrote:
>>
>>> Date: Thu, 28 Jul 2011 15:21:10 -0400
>>> From: Alan Evans <alanwevans at gmail.com>
>>> To: rsyslog at lists.adiscon.com
>>> Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
>>>
>>> I am attempting to configure rsyslog to queue messages while the
>>> destination server is down but I am finding it is only queuing approx
>>> 1 message per second and we receive a whole lot more than that.
>>>
>>> We have many hosts that send messages to a 'proxy' if you will at a
>>> location which then in turn forwards it to our central log server for
>>> archival. ?We are going to have an outage on the central log server
>>> soon so I am trying to replace syslog-ng with rsyslog so the
>>> intermediates can queue messages while the central server is down.
>>>
>>> I am using 3.22.1 packaged with RHEL 5 which I understand is old but I
>>> will not be able to use a newer version. ?I have followed the setup at
>>> http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html but am not
>>> getting the expected results.
>>>
>>> Here's my config.
>>>
>>> # Globals
>>> $MaxMessageSize 8k
>>> $MainMsgQueueType LinkedList
>>>
>>> # Use traditional timestamp format
>>> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
>>>
>>> # Provides kernel logging support (previously done by rklogd)
>>> $ModLoad imklog
>>> # Provides support for local system logging (e.g. via logger command)
>>> $ModLoad imuxsock
>>> $ModLoad imtcp.so
>>>
>>> $InputTCPMaxSessions 1024
>>> $InputTCPServerRun 5142
>>>
>>> $WorkDirectory /var/spool/rsyslog # where to place spool files
>>> $ActionQueueType LinkedList ? # run asynchronously
>>> $ActionQueueFileName remotqeque # unique name prefix for spool files
>>> $ActionResumeRetryCount -1 ? ?# infinite retries if host is down
>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
>>> $ActionQueueMaxDiskSpace 2g ? # 1gb space limit (use as much as possible)
>>> $ActionQueueDequeueSlowdown 0
>>> *.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? @@redacted:51422
>>>
>>> To get logs to rsyslog I have configured syslog-ng to deliver to both
>>> the central syslog server and to rsyslog on the intermediate. ?Rsyslog
>>> is sending to redacted:51422 which is not actually listening (since I
>>> don't really want to double my messages to the central server) which
>>> simulates the central server being offline.
>>>
>>> In practice I am seeing in the debug rsyslog process about one log
>>> entry per second which it is sending to my disk assisted queue.
>>> Syslog-ng however is receiving thousands of messages per minute.
>>>
>>> If I get rid of the action queue and just log to a file rsyslog keeps
>>> up with syslog-ng no problem.
>>>
>>> Here's a snippet from the debug log.
>>>
>>> 2402.830836385:main queue:Reg/w0: action 1 queue: enqueueMsg: cond
>>> timeout, dropping message!
>>> 2402.830862669:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
>>> 2402.830868074:main queue:Reg/w0: action 1 queue: EnqueueMsg advised
>>> worker start
>>> 2402.830879658:main queue:Reg/w0: main queue: entering rate limiter
>>> 2402.830887901:main queue:Reg/w0: main queue: entry deleted, state 0,
>>> size now 7385 entries
>>> 2402.830897655:main queue:Reg/w0: Called action, logging to builtin-fwd
>>> 2402.830903570:main queue:Reg/w0: action 1 queue: enqueueMsg:
>>> LightDelay mark reached for light delayable message - blocking a bit.
>>> 2403.059393630:imuxsock.c: Message from UNIX socket: #6
>>> 2403.059418695:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>>> message content redacted
>>> 2403.059424142:imuxsock.c: Message has legacy syslog format.
>>> 2403.059434733:imuxsock.c: main queue: entry added, size now 7386 entries
>>> 2403.059441696:imuxsock.c: wtpAdviseMaxWorkers signals busy
>>> 2403.059446737:imuxsock.c: main queue: EnqueueMsg advised worker start
>>> 2403.059453751:imuxsock.c: --------imuxsock calling select, active
>>> file descriptors (max 6): 6
>>> 2403.087373110:imuxsock.c: Message from UNIX socket: #6
>>> 2403.087416859:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>>> message content redacted
>>> 2403.087422226:imuxsock.c: Message has legacy syslog format.
>>> 2403.087432941:imuxsock.c: main queue: entry added, size now 7387 entries
>>> 2403.087439847:imuxsock.c: wtpAdviseMaxWorkers signals busy
>>> 2403.087444749:imuxsock.c: main queue: EnqueueMsg advised worker start
>>> 2403.087451626:imuxsock.c: --------imuxsock calling select, active
>>> file descriptors (max 6): 6
>>> 2403.088771548:imuxsock.c: Message from UNIX socket: #6
>>> 2403.088785375:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>>> message content redacted
>>> 2403.088790500:imuxsock.c: Message has legacy syslog format.
>>> 2403.088797998:imuxsock.c: main queue: entry added, size now 7388 entries
>>> 2403.088804299:imuxsock.c: wtpAdviseMaxWorkers signals busy
>>> 2403.088809079:imuxsock.c: main queue: EnqueueMsg advised worker start
>>> 2403.088826047:imuxsock.c: --------imuxsock calling select, active
>>> file descriptors (max 6): 6
>>> 2403.113614170:imuxsock.c: Message from UNIX socket: #6
>>> 2403.113659621:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
>>> message content redacted
>>> 2403.113675071:imuxsock.c: main queue: entry added, size now 7389 entries
>>> 2403.113681913:imuxsock.c: wtpAdviseMaxWorkers signals busy
>>> 2403.113687091:imuxsock.c: main queue: EnqueueMsg advised worker start
>>> 2403.113693974:imuxsock.c: --------imuxsock calling select, active
>>> file descriptors (max 6): 6
>>> 2403.709635421:imtcp.c: main queue: entry added, size now 7390 entries
>>> 2403.709660157:imtcp.c: wtpAdviseMaxWorkers signals busy
>>> 2403.709665565:imtcp.c: main queue: EnqueueMsg advised worker start
>>> 2403.709683772:imtcp.c: logmsg: flags 0, from 'hostnameredacted',
>>> message content redacted
>>> 2403.709688841:imtcp.c: Message has legacy syslog format.
>>> 2403.709697186:imtcp.c: main queue: enqueueMsg: LightDelay mark
>>> reached for light delayable message - blocking a bit.
>>> 2403.832590271:main queue:Reg/w0: action 1 queue: enqueueMsg: queue
>>> FULL - waiting to drain.
>>> 2404.712385305:imtcp.c: main queue: entry added, size now 7391 entries
>>> 2404.712414709:imtcp.c: wtpAdviseMaxWorkers signals busy
>>>
>>> Lines containing "cond timeout, dropping message!" and "queue FULL -
>>> waiting to drain." have me concerned. ?But it doesn't look like all of
>>> my logs are making it to the main queue anyway. ?Is imtcp dropping
>>> messages and the debug level is not high enough to see it maybe?
>>>
>>> Please help! ?How do I get rsyslog to queue all the messages bound for
>>> the simulated down central log host?
>>>
>>> Regards,
>>> -Alan
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>
>

From rgerhards at hq.adiscon.com  Fri Jul 29 12:38:59 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jul 2011 12:38:59 +0200
Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
In-Reply-To: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>
References: <CAMFVOoXNJ1wjo0UVPXZH=TFow40wZOs6Vkr+2kO_H2GhZqLneQ@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72810BB@GRFEXC.intern.adiscon.com>

Looking at the debug log, this reminds me on an issue with rate limiting. We
initially tried to rate-limit the local log socket as well as e.g. TCP
connections. However, that turned out to be a bad idea in many (most) cases.
So this was made conditional. I don't remember exactly when this was done. I
can probably dig you out the relevant patch - BUT... if you can not build
from source, there is no sense in doing this, because you need to build a new
binary in any case. Assuming the problem is what I have on my mind, you don't
have any other options but patching with the version you use. So if you need
to use the official Red Hat package, you actually need to file a bug report
with them.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Alan Evans
> Sent: Thursday, July 28, 2011 9:21 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Rsyslog as intermediate syslog not queueing messages
> 
> I am attempting to configure rsyslog to queue messages while the
> destination server is down but I am finding it is only queuing approx
> 1 message per second and we receive a whole lot more than that.
> 
> We have many hosts that send messages to a 'proxy' if you will at a
> location which then in turn forwards it to our central log server for
> archival.  We are going to have an outage on the central log server
> soon so I am trying to replace syslog-ng with rsyslog so the
> intermediates can queue messages while the central server is down.
> 
> I am using 3.22.1 packaged with RHEL 5 which I understand is old but I
> will not be able to use a newer version.  I have followed the setup at
> http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html but am not
> getting the expected results.
> 
> Here's my config.
> 
> # Globals
> $MaxMessageSize 8k
> $MainMsgQueueType LinkedList
> 
> # Use traditional timestamp format
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
> 
> # Provides kernel logging support (previously done by rklogd)
> $ModLoad imklog
> # Provides support for local system logging (e.g. via logger command)
> $ModLoad imuxsock
> $ModLoad imtcp.so
> 
> $InputTCPMaxSessions 1024
> $InputTCPServerRun 5142
> 
> $WorkDirectory /var/spool/rsyslog # where to place spool files
> $ActionQueueType LinkedList   # run asynchronously
> $ActionQueueFileName remotqeque # unique name prefix for spool files
> $ActionResumeRetryCount -1    # infinite retries if host is down
> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
> $ActionQueueMaxDiskSpace 2g   # 1gb space limit (use as much as
> possible)
> $ActionQueueDequeueSlowdown 0
> *.*
> @@redacted:51422
> 
> To get logs to rsyslog I have configured syslog-ng to deliver to both
> the central syslog server and to rsyslog on the intermediate.  Rsyslog
> is sending to redacted:51422 which is not actually listening (since I
> don't really want to double my messages to the central server) which
> simulates the central server being offline.
> 
> In practice I am seeing in the debug rsyslog process about one log
> entry per second which it is sending to my disk assisted queue.
> Syslog-ng however is receiving thousands of messages per minute.
> 
> If I get rid of the action queue and just log to a file rsyslog keeps
> up with syslog-ng no problem.
> 
> Here's a snippet from the debug log.
> 
> 2402.830836385:main queue:Reg/w0: action 1 queue: enqueueMsg: cond
> timeout, dropping message!
> 2402.830862669:main queue:Reg/w0: wtpAdviseMaxWorkers signals busy
> 2402.830868074:main queue:Reg/w0: action 1 queue: EnqueueMsg advised
> worker start
> 2402.830879658:main queue:Reg/w0: main queue: entering rate limiter
> 2402.830887901:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 7385 entries
> 2402.830897655:main queue:Reg/w0: Called action, logging to builtin-fwd
> 2402.830903570:main queue:Reg/w0: action 1 queue: enqueueMsg:
> LightDelay mark reached for light delayable message - blocking a bit.
> 2403.059393630:imuxsock.c: Message from UNIX socket: #6
> 2403.059418695:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.059424142:imuxsock.c: Message has legacy syslog format.
> 2403.059434733:imuxsock.c: main queue: entry added, size now 7386
> entries
> 2403.059441696:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.059446737:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.059453751:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.087373110:imuxsock.c: Message from UNIX socket: #6
> 2403.087416859:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.087422226:imuxsock.c: Message has legacy syslog format.
> 2403.087432941:imuxsock.c: main queue: entry added, size now 7387
> entries
> 2403.087439847:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.087444749:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.087451626:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.088771548:imuxsock.c: Message from UNIX socket: #6
> 2403.088785375:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.088790500:imuxsock.c: Message has legacy syslog format.
> 2403.088797998:imuxsock.c: main queue: entry added, size now 7388
> entries
> 2403.088804299:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.088809079:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.088826047:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.113614170:imuxsock.c: Message from UNIX socket: #6
> 2403.113659621:imuxsock.c: logmsg: flags 4, from 'hostnameredacted',
> message content redacted
> 2403.113675071:imuxsock.c: main queue: entry added, size now 7389
> entries
> 2403.113681913:imuxsock.c: wtpAdviseMaxWorkers signals busy
> 2403.113687091:imuxsock.c: main queue: EnqueueMsg advised worker start
> 2403.113693974:imuxsock.c: --------imuxsock calling select, active
> file descriptors (max 6): 6
> 2403.709635421:imtcp.c: main queue: entry added, size now 7390 entries
> 2403.709660157:imtcp.c: wtpAdviseMaxWorkers signals busy
> 2403.709665565:imtcp.c: main queue: EnqueueMsg advised worker start
> 2403.709683772:imtcp.c: logmsg: flags 0, from 'hostnameredacted',
> message content redacted
> 2403.709688841:imtcp.c: Message has legacy syslog format.
> 2403.709697186:imtcp.c: main queue: enqueueMsg: LightDelay mark
> reached for light delayable message - blocking a bit.
> 2403.832590271:main queue:Reg/w0: action 1 queue: enqueueMsg: queue
> FULL - waiting to drain.
> 2404.712385305:imtcp.c: main queue: entry added, size now 7391 entries
> 2404.712414709:imtcp.c: wtpAdviseMaxWorkers signals busy
> 
> Lines containing "cond timeout, dropping message!" and "queue FULL -
> waiting to drain." have me concerned.  But it doesn't look like all of
> my logs are making it to the main queue anyway.  Is imtcp dropping
> messages and the debug level is not high enough to see it maybe?
> 
> Please help!  How do I get rsyslog to queue all the messages bound for
> the simulated down central log host?
> 
> Regards,
> -Alan
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From toddmichael at gmail.com  Sat Jul 30 07:27:56 2011
From: toddmichael at gmail.com (Todd Michael Bushnell)
Date: Fri, 29 Jul 2011 22:27:56 -0700
Subject: [rsyslog] rsyslog 5.8.0 dies at startup on Centos 5.6 box (error
	during class init)
Message-ID: <16356064-B4F2-49B1-8E0F-A749FBA65FFA@gmail.com>

Been using rsyslog quite nicely since the day 5.8.0 came out.  Install all systems the same way: same config and same RPM.  Recently installed on newly build CentOS 5.6 box and got this when I fired it up:

Starting system logger: Error during class init for object 'rsyslog runtime' - failing...
rsyslogd initializiation failed - global classes could not be initialized.
Did you do a "make install"?
Suggested action: run rsyslogd with -d -n options to see what exactly fails.
rsyslogd run failed with error 1 (see rsyslog.h or try http://www.rsyslog.com/e/-1 to learn what that number means)
                                                           [  OK  ]


ps aux does not reveal a running (r)syslog process, but /var/lock/subsys/rsyslog does get created.  

I assumed that perhaps something broke with 5.8.0 on CentOS 5.6 so I pulled down the latest version - 5.8.3 - built on a similarly configured CentOS 5.6 box and attempted redeployment.  Same issue.  I followed instructions by running rsyslog -d -n along with sysconfig options to see what the problem is; however, when I launch in the foreground it launches and stays up perfectly.  I've tried with both my known working configuration as well as the default config that comes with the system.  Because we're talking about the log engine and because the only way it fails is when launched in the background via init, I'm not really sure where to go from here.  I suspect a missing dependency, but if that were the case, why would it run in the foreground?  Any troubleshooting guidance appreciated.  Happy to provide more info upon request.

todd





From up at 3.am  Sat Jul 30 20:42:32 2011
From: up at 3.am (up at 3.am)
Date: Sat, 30 Jul 2011 14:42:32 -0400
Subject: [rsyslog] sharing (r)syslog facilities
Message-ID: <56a11b8160af52e3a050019589a83ae9.squirrel@ssl.pil.net>

I've been doing a few basic remote rsyslog services for a few months with mostly
good results.

Now we want to have dozens of servers all log many different services to a central
log server.  Each service has its own set of challenges due to varying levels of
syslog compatibility/compliance, but my main, simple (stupid?) question is...what
do you do about the fact that there aren't really enough different, unique
facilities to go around for all the different logs you want to keep?

I thought I had found a way around this, while trying to get apache to log
remotely (mixed success):

http://wiki.rsyslog.com/index.php/Working_Apache_and_Rsyslog_configuration

In this example, it shows:
---
Now for rsyslog.conf. It's possible that other applications are logging under the
local6 and local7 facilities, so we want to log based on both facility and program
name. Moreover, having these logs included in multiple places would not be good,
so we'll just dump them after we've pulled them out.

if $syslogfacility-text == 'local6' and $programname == 'httpd' then
/var/log/httpd-access_log
if $syslogfacility-text == 'local6' and $programname == 'httpd' then ~
if $syslogfacility-text == 'local7' and $programname == 'httpd' then
/var/log/httpd-error_log
if $syslogfacility-text == 'local7' and $programname == 'httpd' then ~
----
I literally copied and pasted (changed the log name only) the above into both the
client host's rsyslog.conf and the logging server's rsyslog.conf, but what did log
at all (errors only - separate issue), logged into /var/log/messages of the local
server, which looks like a facility conflict to me.

I just have one forwarding rule at the end of the client's ryslog.conf as follows
(works for most services):

*.* @@192.0.0.22:514

Lastly, I have a rule to keep listed facilities from posting to /var/log/messages
on the rsyslog server:

*.info;mail.none;authpriv.none;cron.none;local7.none;local6.none;local5.none;loc
al1.none      /var/log/messages

What am I missing?

From david at lang.hm  Sun Jul 31 03:20:30 2011
From: david at lang.hm (david at lang.hm)
Date: Sat, 30 Jul 2011 18:20:30 -0700 (PDT)
Subject: [rsyslog] sharing (r)syslog facilities
In-Reply-To: <56a11b8160af52e3a050019589a83ae9.squirrel@ssl.pil.net>
References: <56a11b8160af52e3a050019589a83ae9.squirrel@ssl.pil.net>
Message-ID: <alpine.DEB.2.02.1107301813530.32706@asgard.lang.hm>

On Sat, 30 Jul 2011, up at 3.am wrote:

> I've been doing a few basic remote rsyslog services for a few months with mostly
> good results.
>
> Now we want to have dozens of servers all log many different services to a central
> log server.  Each service has its own set of challenges due to varying levels of
> syslog compatibility/compliance, but my main, simple (stupid?) question is...what
> do you do about the fact that there aren't really enough different, unique
> facilities to go around for all the different logs you want to keep?

facility based logging is insufficient for just about any serious logging 
project. personally I act as if facility doesn't exist at all (and 
frequently act as if severity doesn't exist)

> I thought I had found a way around this, while trying to get apache to log
> remotely (mixed success):
>
> http://wiki.rsyslog.com/index.php/Working_Apache_and_Rsyslog_configuration
>
> In this example, it shows:
> ---
> Now for rsyslog.conf. It's possible that other applications are logging under the
> local6 and local7 facilities, so we want to log based on both facility and program
> name. Moreover, having these logs included in multiple places would not be good,
> so we'll just dump them after we've pulled them out.
>
> if $syslogfacility-text == 'local6' and $programname == 'httpd' then
> /var/log/httpd-access_log
> if $syslogfacility-text == 'local6' and $programname == 'httpd' then ~
> if $syslogfacility-text == 'local7' and $programname == 'httpd' then
> /var/log/httpd-error_log
> if $syslogfacility-text == 'local7' and $programname == 'httpd' then ~

a simpler way to do this is:

if $syslogfacility-text == "local6" and $programname == "httpd" then/var/log/httpd-access_log
& ~
if $syslogfacility-text == "local7" and $programname == "httpd" then/var/log/httpd-error_log
& ~

one important thing to note is that current versions of rsyslog don't 
allow strings to be delimited by ' only by "  this is being fixed in the 
6.3 branch, but will not be backported.

> ----
> I literally copied and pasted (changed the log name only) the above into both the
> client host's rsyslog.conf and the logging server's rsyslog.conf, but what did log
> at all (errors only - separate issue), logged into /var/log/messages of the local
> server, which looks like a facility conflict to me.
>
> I just have one forwarding rule at the end of the client's ryslog.conf as follows
> (works for most services):
>
> *.* @@192.0.0.22:514
>
> Lastly, I have a rule to keep listed facilities from posting to /var/log/messages
> on the rsyslog server:
>
> *.info;mail.none;authpriv.none;cron.none;local7.none;local6.none;local5.none;loc
> al1.none      /var/log/messages
>
> What am I missing?

I think your problem is probably ' vs "

David Lang

From sivan at omniqueue.com  Sun Jul 31 14:50:55 2011
From: sivan at omniqueue.com (Sivan Greenberg)
Date: Sun, 31 Jul 2011 15:50:55 +0300
Subject: [rsyslog] sharing (r)syslog facilities
In-Reply-To: <alpine.DEB.2.02.1107301813530.32706@asgard.lang.hm>
References: <56a11b8160af52e3a050019589a83ae9.squirrel@ssl.pil.net>
	<alpine.DEB.2.02.1107301813530.32706@asgard.lang.hm>
Message-ID: <CAJNcf-ZE38AkfsSo3q1rHQW79=89EbQWWW0BrivGSVYthQbF=Q@mail.gmail.com>

On Sun, Jul 31, 2011 at 4:20 AM,  <david at lang.hm> wrote:
> On Sat, 30 Jul 2011, up at 3.am wrote:
>
>> I've been doing a few basic remote rsyslog services for a few months with
>> mostly
>> good results.
>>
>> Now we want to have dozens of servers all log many different services to a
>> central
>> log server. ?Each service has its own set of challenges due to varying
>> levels of
>> syslog compatibility/compliance, but my main, simple (stupid?) question
>> is...what
>> do you do about the fact that there aren't really enough different, unique
>> facilities to go around for all the different logs you want to keep?
>
> facility based logging is insufficient for just about any serious logging
> project. personally I act as if facility doesn't exist at all (and
> frequently act as if severity doesn't exist)
>
That is the same approach I've taken in my logging projects, I share
your opinion on this David.


I used tags and filters on the other hand to achieve that, completely
ignoring facilities and severity.

-Sivan

From up at 3.am  Sun Jul 31 16:25:24 2011
From: up at 3.am (up at 3.am)
Date: Sun, 31 Jul 2011 10:25:24 -0400
Subject: [rsyslog] sharing (r)syslog facilities
Message-ID: <07e4c6213540112bb209bdfc25e918a3.squirrel@ssl.pil.net>

> On Sun, Jul 31, 2011 at 4:20 AM,  <david at lang.hm> wrote:
>> On Sat, 30 Jul 2011, up at 3.am wrote:
>>> I've been doing a few basic remote rsyslog services for a few months with mostly
>>> good results.
>>> Now we want to have dozens of servers all log many different services to a
central
>>> log server. ?Each service has its own set of challenges due to varying levels of
>>> syslog compatibility/compliance, but my main, simple (stupid?) question is...what
>>> do you do about the fact that there aren't really enough different, unique
facilities to go around for all the different logs you want to keep?
>> facility based logging is insufficient for just about any serious logging
project. personally I act as if facility doesn't exist at all (and frequently
act as if severity doesn't exist)
> That is the same approach I've taken in my logging projects, I share your
opinion on this David.
>
>
> I used tags and filters on the other hand to achieve that, completely ignoring
facilities and severity.

I didn't even know you could do (r)syslog without facilities.  Every client app,
when configured to log to syslog, seems to require a facility, even if not a
severity.  I'd love to get rid of this albatross.  Is this a good start?:

http://www.rsyslog.com/doc/rsyslog_conf_filter.html

Are you basically saying that one can ignore true "syslog" and just have a client
server's rsyslogd remotely log any text log file to the log server, based on these
filters?

Is it safe to assume that Adiscon's own LogAnalyzer can digest these
non-syslog-like logs without much trouble?

Thanks again!




From sivan at omniqueue.com  Sun Jul 31 16:30:16 2011
From: sivan at omniqueue.com (Sivan Greenberg)
Date: Sun, 31 Jul 2011 17:30:16 +0300
Subject: [rsyslog] sharing (r)syslog facilities
In-Reply-To: <07e4c6213540112bb209bdfc25e918a3.squirrel@ssl.pil.net>
References: <07e4c6213540112bb209bdfc25e918a3.squirrel@ssl.pil.net>
Message-ID: <CAJNcf-ZwxVOobHeL3qW-mqR2cYELynmkgeUWxMp0J7rxDYeVbA@mail.gmail.com>

On Sun, Jul 31, 2011 at 5:25 PM,  <up at 3.am> wrote:
>
> I didn't even know you could do (r)syslog without facilities. ?Every client app,
> when configured to log to syslog, seems to require a facility, even if not a
> severity. ?I'd love to get rid of this albatross. ?Is this a good start?:
>
> http://www.rsyslog.com/doc/rsyslog_conf_filter.html
>

Not for nothing was rsyslog chosen as the de factor standard logger on
most of the linuxes.

> Are you basically saying that one can ignore true "syslog" and just have a client
> server's rsyslogd remotely log any text log file to the log server, based on these
> filters?
>

This is how I used it, yes.

> Is it safe to assume that Adiscon's own LogAnalyzer can digest these
> non-syslog-like logs without much trouble?
>
YMMV. But I think it might not have problems it'd just list an empty
field, but in some cases I saw it puts the arbitrary used tag as an
severity or facility, again YMMV.

> Thanks again!
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>



-- 
-Sivan

From up at 3.am  Sun Jul 31 16:33:25 2011
From: up at 3.am (up at 3.am)
Date: Sun, 31 Jul 2011 10:33:25 -0400
Subject: [rsyslog] sharing (r)syslog facilities
In-Reply-To: <alpine.DEB.2.02.1107301813530.32706@asgard.lang.hm>
References: <56a11b8160af52e3a050019589a83ae9.squirrel@ssl.pil.net>
	<alpine.DEB.2.02.1107301813530.32706@asgard.lang.hm>
Message-ID: <a4118357c3ab436daede98bc818219b5.squirrel@ssl.pil.net>

> On Sat, 30 Jul 2011, up at 3.am wrote:

> if $syslogfacility-text == "local7" and $programname == "httpd"
> then/var/log/httpd-error_log
> & ~
>
> one important thing to note is that current versions of rsyslog don't
> allow strings to be delimited by ' only by "  this is being fixed in the
> 6.3 branch, but will not be backported.

6.3??  Yikes, we're running 3.22...installed on centos via yum a few months ago
using standard repositories.  The only optional repositories we use are rpmforge. 
They prefer I use yum for consistency on so many servers, but should I just yum
uninstall these things and re-build 6.latest from source on all clients and the
log server, or is there a better repo for this?

Sorry for the noobness and TIA!


From sivan at omniqueue.com  Sun Jul 31 16:37:25 2011
From: sivan at omniqueue.com (Sivan Greenberg)
Date: Sun, 31 Jul 2011 17:37:25 +0300
Subject: [rsyslog] sharing (r)syslog facilities
In-Reply-To: <a4118357c3ab436daede98bc818219b5.squirrel@ssl.pil.net>
References: <56a11b8160af52e3a050019589a83ae9.squirrel@ssl.pil.net>
	<alpine.DEB.2.02.1107301813530.32706@asgard.lang.hm>
	<a4118357c3ab436daede98bc818219b5.squirrel@ssl.pil.net>
Message-ID: <CAJNcf-aFffrfC-KLjb70MP6cA-kr89P=HoryuzCGH2uuOUBoUQ@mail.gmail.com>

On Sun, Jul 31, 2011 at 5:33 PM,  <up at 3.am> wrote:
> 6.3?? ?Yikes, we're running 3.22...installed on centos via yum a few months ago
> using standard repositories. ?The only optional repositories we use are rpmforge.
> They prefer I use yum for consistency on so many servers, but should I just yum
> uninstall these things and re-build 6.latest from source on all clients and the
> log server, or is there a better repo for this?

I just built from source and it works well, with many fixes compared
to 3.2. However, YMMV again, and make sure you guys don't rely on
stuff from 3.2, e.g. do a good research before switching.

No problem, you're welcome.

-Sivan

From toddmichael at gmail.com  Sun Jul 31 21:22:19 2011
From: toddmichael at gmail.com (Todd Michael Bushnell)
Date: Sun, 31 Jul 2011 12:22:19 -0700
Subject: [rsyslog] rsyslog 5.8.0 dies at startup on Centos 5.6 box
	(error during class init)
In-Reply-To: <16356064-B4F2-49B1-8E0F-A749FBA65FFA@gmail.com>
References: <16356064-B4F2-49B1-8E0F-A749FBA65FFA@gmail.com>
Message-ID: <E40DB44B-E796-417A-9005-76F17F6C0759@gmail.com>

Issue resolved.  Turns out, this box had SELinux set to enforced mode.  Would normally discover this via log perusal, but a bit difficult when the issue is with the logging engine :-)  Anyway, all good now.  Thx.

todd



On Jul 29, 2011, at 10:27 PM, Todd Michael Bushnell wrote:

> Been using rsyslog quite nicely since the day 5.8.0 came out.  Install all systems the same way: same config and same RPM.  Recently installed on newly build CentOS 5.6 box and got this when I fired it up:
> 
> Starting system logger: Error during class init for object 'rsyslog runtime' - failing...
> rsyslogd initializiation failed - global classes could not be initialized.
> Did you do a "make install"?
> Suggested action: run rsyslogd with -d -n options to see what exactly fails.
> rsyslogd run failed with error 1 (see rsyslog.h or try http://www.rsyslog.com/e/-1 to learn what that number means)
>                                                            [  OK  ]
> 
> 
> ps aux does not reveal a running (r)syslog process, but /var/lock/subsys/rsyslog does get created.  
> 
> I assumed that perhaps something broke with 5.8.0 on CentOS 5.6 so I pulled down the latest version - 5.8.3 - built on a similarly configured CentOS 5.6 box and attempted redeployment.  Same issue.  I followed instructions by running rsyslog -d -n along with sysconfig options to see what the problem is; however, when I launch in the foreground it launches and stays up perfectly.  I've tried with both my known working configuration as well as the default config that comes with the system.  Because we're talking about the log engine and because the only way it fails is when launched in the background via init, I'm not really sure where to go from here.  I suspect a missing dependency, but if that were the case, why would it run in the foreground?  Any troubleshooting guidance appreciated.  Happy to provide more info upon request.
> 
> todd
> 
> 
> 
>