From pchacin at inteleye.com  Tue Oct  4 18:05:47 2011
From: pchacin at inteleye.com (Pablo Chacin)
Date: Tue, 04 Oct 2011 18:05:47 +0200
Subject: [rsyslog] How to generate RFC 5424 log data from my application
Message-ID: <4E8B2EDB.3010208@inteleye.com>

Hi all

Is there a way to generate log data using the RFC 5424 format?
If not, would an standard syslog message on which I add structured data 
according the RFC 5424
be recognised and processed properly by rsyslog?

Thanks in advance

From david at lang.hm  Tue Oct  4 22:30:57 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 4 Oct 2011 13:30:57 -0700 (PDT)
Subject: [rsyslog] How to generate RFC 5424 log data from my application
In-Reply-To: <4E8B2EDB.3010208@inteleye.com>
References: <4E8B2EDB.3010208@inteleye.com>
Message-ID: <alpine.DEB.2.02.1110041328570.7797@asgard.lang.hm>

On Tue, 4 Oct 2011, Pablo Chacin wrote:

> Hi all
>
> Is there a way to generate log data using the RFC 5424 format?

remember that rsyslog transports and outputs log messages, with very few 
exceptions it doesn't create them. you need the software that creates them 
to do the formatting

> If not, would an standard syslog message on which I add structured data 
> according the RFC 5424
> be recognised and processed properly by rsyslog?

yes, rsyslog handles RFC5424, you may note that the author of that RFC is 
the primary author of rsyslog.

David Lang

> Thanks in advance
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From vladg at illinois.edu  Wed Oct  5 00:55:34 2011
From: vladg at illinois.edu (Grigorescu, Vlad)
Date: Tue, 4 Oct 2011 22:55:34 +0000
Subject: [rsyslog] omoracle on rsyslog v6?
Message-ID: <CAB0F916.5132%vladg@illinois.edu>

Hello,

Has anyone else tried building omoracle on rsyslog 6? It worked fine for
me in rsyslog-5.8.4, but it's failing in rsyslog-6.1.12:

<snip> (For full log file, see: <http://pastebin.ca/2086975>)
omoracle.c: In function 'queryEtryPt':
omoracle.c:575: error: 'newScope' undeclared (first use in this function)
omoracle.c:575: error: (Each undeclared identifier is reported only once
omoracle.c:575: error: for each function it appears in.)
omoracle.c:575: error: 'restoreScope' undeclared (first use in this
function)
omoracle.c: In function 'modInit':
omoracle.c:607: error: too few arguments to function 'omsdRegCFSLineHdlr'
omoracle.c:611: error: too few arguments to function 'omsdRegCFSLineHdlr'
omoracle.c:614: error: too few arguments to function 'omsdRegCFSLineHdlr'
omoracle.c:617: error: too few arguments to function 'omsdRegCFSLineHdlr'
omoracle.c:620: error: too few arguments to function 'omsdRegCFSLineHdlr'
omoracle.c:628: error: too few arguments to function 'omsdRegCFSLineHdlr'
omoracle.c:632: error: too few arguments to function 'omsdRegCFSLineHdlr'
make[2]: *** [omoracle_la-omoracle.lo] Error 1
make[2]: Leaving directory `/home/user/src/rsyslog-6.1.12/plugins/omoracle'
</snip>


I've tried using progressively older versions of rsyslog6, going as far
back as 6.1.0, the earliest I could find.

It looks like the v6 branch introduces scoping, which seems to be related
to the error messages I see. Does the plugin need to be updated? I'll do
some more poking around...

Thanks,

  --Vlad Grigorescu
  IT Security Engineer
  University of Illinois


From rgerhards at hq.adiscon.com  Wed Oct  5 11:50:34 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 5 Oct 2011 11:50:34 +0200
Subject: [rsyslog] omoracle on rsyslog v6?
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72812E1@GRFEXC.intern.adiscon.com>

Hi,

I just checked. I received a potential patch some time ago from Maik K?ndig,
find it attached. I did not yet merge it because I cannot even compile
omoracle in my environment.If it solves the issue, please let me know and
I'll apply it.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Grigorescu, Vlad
> Sent: Wednesday, October 05, 2011 12:56 AM
> To: rsyslog-users
> Subject: [rsyslog] omoracle on rsyslog v6?
> 
> Hello,
> 
> Has anyone else tried building omoracle on rsyslog 6? It worked fine for me
in
> rsyslog-5.8.4, but it's failing in rsyslog-6.1.12:
> 
> <snip> (For full log file, see: <http://pastebin.ca/2086975>)
> omoracle.c: In function 'queryEtryPt':
> omoracle.c:575: error: 'newScope' undeclared (first use in this function)
> omoracle.c:575: error: (Each undeclared identifier is reported only once
> omoracle.c:575: error: for each function it appears in.)
> omoracle.c:575: error: 'restoreScope' undeclared (first use in this
> function)
> omoracle.c: In function 'modInit':
> omoracle.c:607: error: too few arguments to function 'omsdRegCFSLineHdlr'
> omoracle.c:611: error: too few arguments to function 'omsdRegCFSLineHdlr'
> omoracle.c:614: error: too few arguments to function 'omsdRegCFSLineHdlr'
> omoracle.c:617: error: too few arguments to function 'omsdRegCFSLineHdlr'
> omoracle.c:620: error: too few arguments to function 'omsdRegCFSLineHdlr'
> omoracle.c:628: error: too few arguments to function 'omsdRegCFSLineHdlr'
> omoracle.c:632: error: too few arguments to function 'omsdRegCFSLineHdlr'
> make[2]: *** [omoracle_la-omoracle.lo] Error 1
> make[2]: Leaving directory `/home/user/src/rsyslog-
> 6.1.12/plugins/omoracle'
> </snip>
> 
> 
> I've tried using progressively older versions of rsyslog6, going as far
back as
> 6.1.0, the earliest I could find.
> 
> It looks like the v6 branch introduces scoping, which seems to be related
to
> the error messages I see. Does the plugin need to be updated? I'll do some
> more poking around...
> 
> Thanks,
> 
>   --Vlad Grigorescu
>   IT Security Engineer
>   University of Illinois
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: omoracle.c.diff
Type: application/octet-stream
Size: 1212 bytes
Desc: omoracle.c.diff
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111005/528d4ebf/attachment.obj>

From pchacin at inteleye.com  Wed Oct  5 12:53:38 2011
From: pchacin at inteleye.com (Pablo Chacin)
Date: Wed, 05 Oct 2011 12:53:38 +0200
Subject: [rsyslog] How to generate RFC 5424 log data from my application
In-Reply-To: <alpine.DEB.2.02.1110041328570.7797@asgard.lang.hm>
References: <4E8B2EDB.3010208@inteleye.com>
	<alpine.DEB.2.02.1110041328570.7797@asgard.lang.hm>
Message-ID: <4E8C3732.3010000@inteleye.com>

Hi David
> you need the software that creates them to do the formatting 
Exactly, question is, is there any? should I do it "by hand" and if so, how?
> yes, rsyslog handles RFC5424, you may note that the author of that RFC 
> is the primary author of rsyslog. 

I understand that rsyslog handles RFC5424. My question was more about 
this scenario: if my application uses the standard syslog calls to 
generate log messages, but the message text is formatted following the 
RFC5424 specifications (in particular, adding the tags), will this be 
enough for rsyslog to recognise the format and process it properly?

More over, will this format be a valid message format for the syslog 
API? will it accept the tags?


Anyone in this list has done this so far and could comment on his/her 
experience?

Thanks in advance

Pablo

On 10/04/2011 10:30 PM, david at lang.hm wrote:
> On Tue, 4 Oct 2011, Pablo Chacin wrote:
>
>> Hi all
>>
>> Is there a way to generate log data using the RFC 5424 format?
>
> remember that rsyslog transports and outputs log messages, with very 
> few exceptions it doesn't create them. you need the software that 
> creates them to do the formatting
>
>> If not, would an standard syslog message on which I add structured 
>> data according the RFC 5424
>> be recognised and processed properly by rsyslog?
>
> yes, rsyslog handles RFC5424, you may note that the author of that RFC 
> is the primary author of rsyslog.
>
> David Lang
>
>> Thanks in advance
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Wed Oct  5 13:08:28 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 5 Oct 2011 04:08:28 -0700 (PDT)
Subject: [rsyslog] How to generate RFC 5424 log data from my application
In-Reply-To: <4E8C3732.3010000@inteleye.com>
References: <4E8B2EDB.3010208@inteleye.com>
	<alpine.DEB.2.02.1110041328570.7797@asgard.lang.hm>
	<4E8C3732.3010000@inteleye.com>
Message-ID: <alpine.DEB.2.02.1110050401500.5597@asgard.lang.hm>

On Wed, 5 Oct 2011, Pablo Chacin wrote:

> Hi David
>> you need the software that creates them to do the formatting 
> Exactly, question is, is there any? should I do it "by hand" and if so, how?
>> yes, rsyslog handles RFC5424, you may note that the author of that RFC is 
>> the primary author of rsyslog. 
>
> I understand that rsyslog handles RFC5424. My question was more about this 
> scenario: if my application uses the standard syslog calls to generate log 
> messages, but the message text is formatted following the RFC5424 
> specifications (in particular, adding the tags), will this be enough for 
> rsyslog to recognise the format and process it properly?
>
> More over, will this format be a valid message format for the syslog API? 
> will it accept the tags?

rsyslog doesn't process the tags, the message is just one long string, so 
it doesn't matter what the body of the message is, rsyslog will send it 
on (or write it out). the only differences at the transport layer are in 
the header and timestamp, the message itself is not interpreted by rsyslog 
(unless you do advanced filtering, but even that's string manipulation)

David Lang

>
> Anyone in this list has done this so far and could comment on his/her 
> experience?
>
> Thanks in advance
>
> Pablo
>
> On 10/04/2011 10:30 PM, david at lang.hm wrote:
>> On Tue, 4 Oct 2011, Pablo Chacin wrote:
>> 
>>> Hi all
>>> 
>>> Is there a way to generate log data using the RFC 5424 format?
>> 
>> remember that rsyslog transports and outputs log messages, with very few 
>> exceptions it doesn't create them. you need the software that creates them 
>> to do the formatting
>> 
>>> If not, would an standard syslog message on which I add structured data 
>>> according the RFC 5424
>>> be recognised and processed properly by rsyslog?
>> 
>> yes, rsyslog handles RFC5424, you may note that the author of that RFC is 
>> the primary author of rsyslog.
>> 
>> David Lang
>> 
>>> Thanks in advance
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>> 
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Wed Oct  5 13:26:11 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 5 Oct 2011 13:26:11 +0200
Subject: [rsyslog] How to generate RFC 5424 log data from my application
In-Reply-To: <alpine.DEB.2.02.1110050401500.5597@asgard.lang.hm>
References: <4E8B2EDB.3010208@inteleye.com><alpine.DEB.2.02.1110041328570.7797@asgard.lang.hm><4E8C3732.3010000@inteleye.com>
	<alpine.DEB.2.02.1110050401500.5597@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72812E5@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, October 05, 2011 1:08 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to generate RFC 5424 log data from my
application
> 
> On Wed, 5 Oct 2011, Pablo Chacin wrote:
> 
> > Hi David
> >> you need the software that creates them to do the formatting
> > Exactly, question is, is there any? should I do it "by hand" and if so,
how?
> >> yes, rsyslog handles RFC5424, you may note that the author of that
> >> RFC is the primary author of rsyslog.
> >
> > I understand that rsyslog handles RFC5424. My question was more about
> > this
> > scenario: if my application uses the standard syslog calls to generate
> > log messages, but the message text is formatted following the RFC5424
> > specifications (in particular, adding the tags), will this be enough
> > for rsyslog to recognise the format and process it properly?
> >
> > More over, will this format be a valid message format for the syslog API?
> > will it accept the tags?
> 
> rsyslog doesn't process the tags, the message is just one long string, so
it
> doesn't matter what the body of the message is, rsyslog will send it on (or
> write it out). the only differences at the transport layer are in the
header and
> timestamp, the message itself is not interpreted by rsyslog (unless you do
> advanced filtering, but even that's string manipulation)

Yeah, that's really a sad story :-(

The POSIX API does not at all  support RFC5424 and there currently is no
valid way to do that. I was tempted to write a small library the provides
this ability, but stepped back from that because it sounds like an awful idea
to try to bring this as a standard into all Linux distros. Someone (sorry,
forgot the name, maybe it was Martin who wrote the FreeBSD RFC5424
enhancement) suggested that we use some specific format inside the syslog()
api for that path. Out of the bad choices, this seems to be the least worse
one. But I have not yet really begun to look at this. Maybe some thing I
finally should put on the agenda (interest in RFC5424 seems to increase, so
it may be the right time now).

Rainer

From victor.lu at citi.com  Wed Oct  5 23:44:05 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Wed, 5 Oct 2011 16:44:05 -0500
Subject: [rsyslog] ifdef (loghost) in rsyslog.conf
In-Reply-To: <FFA1DBC60A814545906CB1DCDA645ED01DCE7D27A5@extxmb32.nam.nsroot.net>
References: <FFA1DBC60A814545906CB1DCDA645ED01DCE7D27A5@extxmb32.nam.nsroot.net>
Message-ID: <35B12B7283BF44478AFA717323EE52951CB2190B43@extxmb32.nam.nsroot.net>

Hi there,

One of the requirement we got is to migrate the log facilities and files in syslog.conf  to rsyslog,conf without any change.  But I found the following ifdef statements are not recognized by rsyslogd. It always comes back with syntax error.

# if a non-loghost machine chooses to have authentication messages
# sent to the loghost machine, un-comment out the following line:
#auth.notice                    ifdef(`LOGHOST', /var/log/authlog, @loghost)

mail.debug                      ifdef(`LOGHOST', /var/log/syslog, @loghost)

#
# non-loghost machines will use the following lines to cause "user"
# log messages to be logged locally.
#
ifdef(`LOGHOST', ,
user.err                                        /dev/sysmsg
user.err                                        /var/adm/messages
user.alert                                      `root, operator'
user.emerg                                      *
)

Is there a way to convert them in rsyslog understandable language?

Thanks

Victor


From david at lang.hm  Thu Oct  6 00:28:49 2011
From: david at lang.hm (david at lang.hm)
Date: Wed, 5 Oct 2011 15:28:49 -0700 (PDT)
Subject: [rsyslog] ifdef (loghost) in rsyslog.conf
In-Reply-To: <35B12B7283BF44478AFA717323EE52951CB2190B43@extxmb32.nam.nsroot.net>
References: <FFA1DBC60A814545906CB1DCDA645ED01DCE7D27A5@extxmb32.nam.nsroot.net>
	<35B12B7283BF44478AFA717323EE52951CB2190B43@extxmb32.nam.nsroot.net>
Message-ID: <alpine.DEB.2.02.1110051527010.7797@asgard.lang.hm>

On Wed, 5 Oct 2011, Lu, Victor  wrote:

> Hi there,
>
> One of the requirement we got is to migrate the log facilities and files in syslog.conf  to rsyslog,conf without any change.  But I found the following ifdef statements are not recognized by rsyslogd. It always comes back with syntax error.
>
> # if a non-loghost machine chooses to have authentication messages
> # sent to the loghost machine, un-comment out the following line:
> #auth.notice                    ifdef(`LOGHOST', /var/log/authlog, @loghost)
>
> mail.debug                      ifdef(`LOGHOST', /var/log/syslog, @loghost)
>
> #
> # non-loghost machines will use the following lines to cause "user"
> # log messages to be logged locally.
> #
> ifdef(`LOGHOST', ,
> user.err                                        /dev/sysmsg
> user.err                                        /var/adm/messages
> user.alert                                      `root, operator'
> user.emerg                                      *
> )
>
> Is there a way to convert them in rsyslog understandable language?

what you are asking for is for rsyslog to look at an environment variable 
from within the config file. As far as I know there is no way to do this.

What I would do is to create two versions of your config file (one if you 
are a loghost and one if you are not) and then have some external config 
management tool use the appropriate config.

out of curiosity, what syslog daemon were you using before? I did not know 
of any that would allow you to do this sort of thing.

David Lang

From rgerhards at hq.adiscon.com  Thu Oct  6 07:48:33 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 6 Oct 2011 07:48:33 +0200
Subject: [rsyslog] ifdef (loghost) in rsyslog.conf
In-Reply-To: <alpine.DEB.2.02.1110051527010.7797@asgard.lang.hm>
References: <FFA1DBC60A814545906CB1DCDA645ED01DCE7D27A5@extxmb32.nam.nsroot.net><35B12B7283BF44478AFA717323EE52951CB2190B43@extxmb32.nam.nsroot.net>
	<alpine.DEB.2.02.1110051527010.7797@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72812EB@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, October 06, 2011 12:29 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] ifdef (loghost) in rsyslog.conf
> 
> On Wed, 5 Oct 2011, Lu, Victor  wrote:
> 
> > Hi there,
> >
> > One of the requirement we got is to migrate the log facilities and
> files in syslog.conf  to rsyslog,conf without any change.  But I found
> the following ifdef statements are not recognized by rsyslogd. It
> always comes back with syntax error.
> >
> > # if a non-loghost machine chooses to have authentication messages
> > # sent to the loghost machine, un-comment out the following line:
> > #auth.notice                    ifdef(`LOGHOST', /var/log/authlog,
> @loghost)
> >
> > mail.debug                      ifdef(`LOGHOST', /var/log/syslog,
> @loghost)
> >
> > #
> > # non-loghost machines will use the following lines to cause "user"
> > # log messages to be logged locally.
> > #
> > ifdef(`LOGHOST', ,
> > user.err                                        /dev/sysmsg
> > user.err                                        /var/adm/messages
> > user.alert                                      `root, operator'
> > user.emerg                                      *
> > )
> >
> > Is there a way to convert them in rsyslog understandable language?
> 
> what you are asking for is for rsyslog to look at an environment
> variable
> from within the config file. As far as I know there is no way to do
> this.
> 
> What I would do is to create two versions of your config file (one if
> you
> are a loghost and one if you are not) and then have some external
> config
> management tool use the appropriate config.
> 
> out of curiosity, what syslog daemon were you using before? I did not
> know
> of any that would allow you to do this sort of thing.

Same question here, I never saw this construct.

Anyhow: I think it should not be too hard to add capabilities to support what
you need. Unfortunately, I am currently quite busy with paid work. If this is
for your company, you may consider purchasing a support contract or
sponsoring the development of such a feature. If you use rsyslog for profit,
that route is probably far less expensive than using a lot of time to tweak
the config.

Rainer

From victor.lu at citi.com  Thu Oct  6 23:34:03 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Thu, 6 Oct 2011 16:34:03 -0500
Subject: [rsyslog] ifdef (loghost) in rsyslog.conf
In-Reply-To: <alpine.DEB.2.02.1110051527010.7797@asgard.lang.hm>
References: <FFA1DBC60A814545906CB1DCDA645ED01DCE7D27A5@extxmb32.nam.nsroot.net>
	<35B12B7283BF44478AFA717323EE52951CB2190B43@extxmb32.nam.nsroot.net>
	<alpine.DEB.2.02.1110051527010.7797@asgard.lang.hm>
Message-ID: <35B12B7283BF44478AFA717323EE52951CB2271065@extxmb32.nam.nsroot.net>

That is Solaris syslog.

http://woss.name/2007/06/17/solaris-logging-to-a-separate-loghost-the-easy-way/


-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
Sent: Wednesday, October 05, 2011 6:29 PM
To: rsyslog-users
Subject: Re: [rsyslog] ifdef (loghost) in rsyslog.conf

On Wed, 5 Oct 2011, Lu, Victor  wrote:

> Hi there,
>
> One of the requirement we got is to migrate the log facilities and files in syslog.conf  to rsyslog,conf without any change.  But I found the following ifdef statements are not recognized by rsyslogd. It always comes back with syntax error.
>
> # if a non-loghost machine chooses to have authentication messages
> # sent to the loghost machine, un-comment out the following line:
> #auth.notice                    ifdef(`LOGHOST', /var/log/authlog, @loghost)
>
> mail.debug                      ifdef(`LOGHOST', /var/log/syslog, @loghost)
>
> #
> # non-loghost machines will use the following lines to cause "user"
> # log messages to be logged locally.
> #
> ifdef(`LOGHOST', ,
> user.err                                        /dev/sysmsg
> user.err                                        /var/adm/messages
> user.alert                                      `root, operator'
> user.emerg                                      *
> )
>
> Is there a way to convert them in rsyslog understandable language?

what you are asking for is for rsyslog to look at an environment variable 
from within the config file. As far as I know there is no way to do this.

What I would do is to create two versions of your config file (one if you 
are a loghost and one if you are not) and then have some external config 
management tool use the appropriate config.

out of curiosity, what syslog daemon were you using before? I did not know 
of any that would allow you to do this sort of thing.

David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From bison at garbagebrain.org  Fri Oct  7 23:28:34 2011
From: bison at garbagebrain.org (Brad Ison)
Date: Fri, 7 Oct 2011 16:28:34 -0500
Subject: [rsyslog] Incorrect dynamic file names
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72812CE@GRFEXC.intern.adiscon.com>
References: <CALEqfkg0ZmaOhLA_aCAmrmP8oUsz0tNh7Zn1iYWq-dxCYa3mjQ@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA72812CE@GRFEXC.intern.adiscon.com>
Message-ID: <CALEqfkibK4RzoG42exMgRD8fXgzCiGFPPnH-t8=u086msjcc9Q@mail.gmail.com>

On Fri, Sep 30, 2011 at 12:33 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> You should upgrade to the current version. 5.8.1 is missing many patches. I
> guess the problem goes away once you have done that...

I've upgraded to 5.8.5 and I'm still seeing this. I can also reliably
reproduce it in a test environment with two VM's.

Most often the filename produced is simply ".log" as if the
programname field were empty. The messages, including the syslog tag,
in that file are totally normal.

Unfortunately, it doesn't seem to happen when debug output has been
enabled whether using the '-d' flag or setting RSYSLOG_DEBUG options,
or at least it hasn't happened yet. It happens within a matter of
minutes without debugging enabled.

I tried putting the debug log on a tmpfs in case writing to that file
was slowing things down enough to affect the behavior, but that
doesn't seem to have had any effect.


Any tips on how to further troubleshoot this?


Thanks again!

--
Brad

From bison at garbagebrain.org  Sat Oct  8 07:29:41 2011
From: bison at garbagebrain.org (Brad Ison)
Date: Sat, 8 Oct 2011 00:29:41 -0500
Subject: [rsyslog] Incorrect dynamic file names
In-Reply-To: <CALEqfkibK4RzoG42exMgRD8fXgzCiGFPPnH-t8=u086msjcc9Q@mail.gmail.com>
References: <CALEqfkg0ZmaOhLA_aCAmrmP8oUsz0tNh7Zn1iYWq-dxCYa3mjQ@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA72812CE@GRFEXC.intern.adiscon.com>
	<CALEqfkibK4RzoG42exMgRD8fXgzCiGFPPnH-t8=u086msjcc9Q@mail.gmail.com>
Message-ID: <CALEqfkjRmCXqeGEjUuAEz1EGuBgUZT_mV9ceLUFmoNJtV1V4QA@mail.gmail.com>

I just wanted to follow up to say that I've reproduced this with 5.8.5
and 6.1.12 now. I was able to capture debug output from 6.1.12.

I've created a forum post as that seems a more appropriate place to
post debug logs:

http://kb.monitorware.com/incorrect-dynamic-file-names-t11001.html

--
Brad

From victor.lu at citi.com  Tue Oct 11 17:13:58 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Tue, 11 Oct 2011 10:13:58 -0500
Subject: [rsyslog] kill -HUP command issue
Message-ID: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>

Hi there,

The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.

  http://www.rsyslog.com/doc/v4compatibility.html

  http://www.rsyslog.com/doc/v5compatibility.html

Here is the note from the website:  That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.

However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?

Thanks in advance.

Regards,

Victor



From mbiebl at gmail.com  Tue Oct 11 20:59:49 2011
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 11 Oct 2011 20:59:49 +0200
Subject: [rsyslog] kill -HUP command issue
In-Reply-To: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
References: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
Message-ID: <CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>

2011/10/11 Lu, Victor <victor.lu at citi.com>:
> Hi there,
>
> The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.
>
> ?http://www.rsyslog.com/doc/v4compatibility.html
>
> ?http://www.rsyslog.com/doc/v5compatibility.html
>
> Here is the note from the website: ?That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.
>
> However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?

You could use systemd for that:
http://www.freedesktop.org/wiki/Software/systemd/syslog

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From david at lang.hm  Tue Oct 11 22:40:50 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 11 Oct 2011 13:40:50 -0700 (PDT)
Subject: [rsyslog] kill -HUP command issue
In-Reply-To: <CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>
References: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
	<CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1110111334320.28233@asgard.lang.hm>

On Tue, 11 Oct 2011, Michael Biebl wrote:

> 2011/10/11 Lu, Victor <victor.lu at citi.com>:
>> Hi there,
>>
>> The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.
>>
>> ?http://www.rsyslog.com/doc/v4compatibility.html
>>
>> ?http://www.rsyslog.com/doc/v5compatibility.html
>>
>> Here is the note from the website: ?That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.
>>
>> However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?
>
> You could use systemd for that:
> http://www.freedesktop.org/wiki/Software/systemd/syslog

systemd doesn't solve the problem

the reason that -HUP was changed from being a restart to just reopeing 
files (which is enough for log rotation to work) is because doing a full 
shutdown can take a long time, and can eventually timeout and throw away 
logs anyway.

consider the case where you have messages in your queue that you cannot 
write to a destination (say a remote server is down for example), unless 
you have a disk assisted queue setup you cannot do anything except throw 
these messages away.

you also cannot shutdown without message loss if you have a continuous 
stream of new messages arriving.

having a -HUP do a full restart caused message loss at every -HUP because 
of the 'new messages continually arriving' for the common UDP syslog case, 
but without doing a HUP (or equivalent), you can't roll the log files as 
rsyslog would continue to write to the old (open) files.

also, changing configurations (which is where you need to do a restart) is 
a _very_ rare condition compared to log rotation.



you can avoid message loss on restart by using RELP for your transport 
protocol and disk assisted queues.

does this solve your problems? or is there some other reason you are 
looking to do a full restart instead of just re-opening files and network 
connections on a HUP?

David Lang

From victor.lu at citi.com  Tue Oct 11 23:01:50 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Tue, 11 Oct 2011 16:01:50 -0500
Subject: [rsyslog] kill -HUP command issue
In-Reply-To: <alpine.DEB.2.02.1110111334320.28233@asgard.lang.hm>
References: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
	<CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>
	<alpine.DEB.2.02.1110111334320.28233@asgard.lang.hm>
Message-ID: <35B12B7283BF44478AFA717323EE52951CD7DC3D68@extxmb32.nam.nsroot.net>

David,

I am not clear about what you are saying. Which version of the rsyslog are based on for the following statements.

" the reason that -HUP was changed from being a restart to just reopeing files (which is enough for log rotation to work) is because doing a full shutdown can take a long time, and can eventually timeout and throw away logs anyway."

That is what I exactly need. We can't do a full restart of rsyslogd at all.  

However, if you look at the following website. The -HUP command was not supported, 
http://www.rsyslog.com/doc/v4compatibility.html

The command normally used by the log rotation
$ kill -HUP `cat /var/run/rsyslogd.pid`

Will be replaced with rsyslog restart.

$ /etc/init.d/rsyslog restart

And I have tested using kill -HUP command to rsyslogd on version 5.8.5, it does not have any effect to the rsyslogd process. 

Thanks

Victor
-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
Sent: Tuesday, October 11, 2011 4:41 PM
To: rsyslog-users
Subject: Re: [rsyslog] kill -HUP command issue

On Tue, 11 Oct 2011, Michael Biebl wrote:

> 2011/10/11 Lu, Victor <victor.lu at citi.com>:
>> Hi there,
>>
>> The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.
>>
>> ?http://www.rsyslog.com/doc/v4compatibility.html
>>
>> ?http://www.rsyslog.com/doc/v5compatibility.html
>>
>> Here is the note from the website: ?That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.
>>
>> However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?
>
> You could use systemd for that:
> http://www.freedesktop.org/wiki/Software/systemd/syslog

systemd doesn't solve the problem

the reason that -HUP was changed from being a restart to just reopeing files (which is enough for log rotation to work) is because doing a full shutdown can take a long time, and can eventually timeout and throw away logs anyway.

consider the case where you have messages in your queue that you cannot write to a destination (say a remote server is down for example), unless you have a disk assisted queue setup you cannot do anything except throw these messages away.

you also cannot shutdown without message loss if you have a continuous stream of new messages arriving.

having a -HUP do a full restart caused message loss at every -HUP because of the 'new messages continually arriving' for the common UDP syslog case, but without doing a HUP (or equivalent), you can't roll the log files as rsyslog would continue to write to the old (open) files.

also, changing configurations (which is where you need to do a restart) is a _very_ rare condition compared to log rotation.



you can avoid message loss on restart by using RELP for your transport 
protocol and disk assisted queues.

does this solve your problems? or is there some other reason you are 
looking to do a full restart instead of just re-opening files and network 
connections on a HUP?

David Lang

From david at lang.hm  Tue Oct 11 23:12:41 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 11 Oct 2011 14:12:41 -0700 (PDT)
Subject: [rsyslog] kill -HUP command issue
In-Reply-To: <35B12B7283BF44478AFA717323EE52951CD7DC3D68@extxmb32.nam.nsroot.net>
References: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
	<CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>
	<alpine.DEB.2.02.1110111334320.28233@asgard.lang.hm>
	<35B12B7283BF44478AFA717323EE52951CD7DC3D68@extxmb32.nam.nsroot.net>
Message-ID: <alpine.DEB.2.02.1110111408290.28233@asgard.lang.hm>

On Tue, 11 Oct 2011, Lu, Victor  wrote:

> David,
>
> I am not clear about what you are saying. Which version of the rsyslog are based on for the following statements.
>
> " the reason that -HUP was changed from being a restart to just reopeing 
> files (which is enough for log rotation to work) is because doing a full 
> shutdown can take a long time, and can eventually timeout and throw away 
> logs anyway."
>
> That is what I exactly need. We can't do a full restart of rsyslogd at all.

a full restart of rsyslog is a kill and restart (/etc/init.d/rsyslog stop; 
/etc/init.d/rsyslog restart)

> However, if you look at the following website. The -HUP command was not supported,
> http://www.rsyslog.com/doc/v4compatibility.html
>
> The command normally used by the log rotation
> $ kill -HUP `cat /var/run/rsyslogd.pid`
>
> Will be replaced with rsyslog restart.
>
> $ /etc/init.d/rsyslog restart

and what does this script do if it's not a kill -HUP? this script isn't 
provided by rsyslog, it's provided by the distribution

> And I have tested using kill -HUP command to rsyslogd on version 5.8.5, 
> it does not have any effect to the rsyslogd process.

I use  kill -HUP all the time for versions 5.x and 6.x

kill -HUP doesn't do anything visibile on the system, but if you do a mv 
on the logfiles that rsyslog is writing to, rsyslog will continue to write 
to the file in it's new location. Then you do a kill -HUP and rsyslog 
closes those files and opens them under their configured names.

David Lang


  > Thanks
>
> Victor
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, October 11, 2011 4:41 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] kill -HUP command issue
>
> On Tue, 11 Oct 2011, Michael Biebl wrote:
>
>> 2011/10/11 Lu, Victor <victor.lu at citi.com>:
>>> Hi there,
>>>
>>> The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.
>>>
>>> ?http://www.rsyslog.com/doc/v4compatibility.html
>>>
>>> ?http://www.rsyslog.com/doc/v5compatibility.html
>>>
>>> Here is the note from the website: ?That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.
>>>
>>> However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?
>>
>> You could use systemd for that:
>> http://www.freedesktop.org/wiki/Software/systemd/syslog
>
> systemd doesn't solve the problem
>
> the reason that -HUP was changed from being a restart to just reopeing files (which is enough for log rotation to work) is because doing a full shutdown can take a long time, and can eventually timeout and throw away logs anyway.
>
> consider the case where you have messages in your queue that you cannot write to a destination (say a remote server is down for example), unless you have a disk assisted queue setup you cannot do anything except throw these messages away.
>
> you also cannot shutdown without message loss if you have a continuous stream of new messages arriving.
>
> having a -HUP do a full restart caused message loss at every -HUP because of the 'new messages continually arriving' for the common UDP syslog case, but without doing a HUP (or equivalent), you can't roll the log files as rsyslog would continue to write to the old (open) files.
>
> also, changing configurations (which is where you need to do a restart) is a _very_ rare condition compared to log rotation.
>
>
>
> you can avoid message loss on restart by using RELP for your transport
> protocol and disk assisted queues.
>
> does this solve your problems? or is there some other reason you are
> looking to do a full restart instead of just re-opening files and network
> connections on a HUP?
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From david at lang.hm  Tue Oct 11 23:13:46 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 11 Oct 2011 14:13:46 -0700 (PDT)
Subject: [rsyslog] kill -HUP command issue
In-Reply-To: <35B12B7283BF44478AFA717323EE52951CD7DC3D68@extxmb32.nam.nsroot.net>
References: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
	<CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>
	<alpine.DEB.2.02.1110111334320.28233@asgard.lang.hm>
	<35B12B7283BF44478AFA717323EE52951CD7DC3D68@extxmb32.nam.nsroot.net>
Message-ID: <alpine.DEB.2.02.1110111412540.28233@asgard.lang.hm>

On Tue, 11 Oct 2011, Lu, Victor  wrote:

> However, if you look at the following website. The -HUP command was not supported,
> http://www.rsyslog.com/doc/v4compatibility.html

note that what this is saying is that the full restart where the config 
file gets re-read is not supported, -HUP still works for log rotation 
purposes.

David Lang


> The command normally used by the log rotation
> $ kill -HUP `cat /var/run/rsyslogd.pid`
>
> Will be replaced with rsyslog restart.
>
> $ /etc/init.d/rsyslog restart
>
> And I have tested using kill -HUP command to rsyslogd on version 5.8.5, it does not have any effect to the rsyslogd process.
>
> Thanks
>
> Victor
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, October 11, 2011 4:41 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] kill -HUP command issue
>
> On Tue, 11 Oct 2011, Michael Biebl wrote:
>
>> 2011/10/11 Lu, Victor <victor.lu at citi.com>:
>>> Hi there,
>>>
>>> The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.
>>>
>>> ?http://www.rsyslog.com/doc/v4compatibility.html
>>>
>>> ?http://www.rsyslog.com/doc/v5compatibility.html
>>>
>>> Here is the note from the website: ?That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.
>>>
>>> However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?
>>
>> You could use systemd for that:
>> http://www.freedesktop.org/wiki/Software/systemd/syslog
>
> systemd doesn't solve the problem
>
> the reason that -HUP was changed from being a restart to just reopeing files (which is enough for log rotation to work) is because doing a full shutdown can take a long time, and can eventually timeout and throw away logs anyway.
>
> consider the case where you have messages in your queue that you cannot write to a destination (say a remote server is down for example), unless you have a disk assisted queue setup you cannot do anything except throw these messages away.
>
> you also cannot shutdown without message loss if you have a continuous stream of new messages arriving.
>
> having a -HUP do a full restart caused message loss at every -HUP because of the 'new messages continually arriving' for the common UDP syslog case, but without doing a HUP (or equivalent), you can't roll the log files as rsyslog would continue to write to the old (open) files.
>
> also, changing configurations (which is where you need to do a restart) is a _very_ rare condition compared to log rotation.
>
>
>
> you can avoid message loss on restart by using RELP for your transport
> protocol and disk assisted queues.
>
> does this solve your problems? or is there some other reason you are
> looking to do a full restart instead of just re-opening files and network
> connections on a HUP?
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From victor.lu at citi.com  Tue Oct 11 23:29:45 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Tue, 11 Oct 2011 16:29:45 -0500
Subject: [rsyslog] kill -HUP command issue
In-Reply-To: <alpine.DEB.2.02.1110111412540.28233@asgard.lang.hm>
References: <35B12B7283BF44478AFA717323EE52951CD73D9FDE@extxmb32.nam.nsroot.net>
	<CAGWsdOhzBw6OyHaiduNAE2-nhpNhv1JgCA2HjAUHF008JPzc_g@mail.gmail.com>
	<alpine.DEB.2.02.1110111334320.28233@asgard.lang.hm>
	<35B12B7283BF44478AFA717323EE52951CD7DC3D68@extxmb32.nam.nsroot.net>
	<alpine.DEB.2.02.1110111412540.28233@asgard.lang.hm>
Message-ID: <35B12B7283BF44478AFA717323EE52951CD7DC3E07@extxmb32.nam.nsroot.net>

David,

Thanks for clarification. 

-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
Sent: Tuesday, October 11, 2011 5:14 PM
To: rsyslog-users
Subject: Re: [rsyslog] kill -HUP command issue

On Tue, 11 Oct 2011, Lu, Victor  wrote:

> However, if you look at the following website. The -HUP command was 
> not supported, http://www.rsyslog.com/doc/v4compatibility.html

note that what this is saying is that the full restart where the config file gets re-read is not supported, -HUP still works for log rotation purposes.

David Lang


> The command normally used by the log rotation $ kill -HUP `cat 
> /var/run/rsyslogd.pid`
>
> Will be replaced with rsyslog restart.
>
> $ /etc/init.d/rsyslog restart
>
> And I have tested using kill -HUP command to rsyslogd on version 5.8.5, it does not have any effect to the rsyslogd process.
>
> Thanks
>
> Victor
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, October 11, 2011 4:41 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] kill -HUP command issue
>
> On Tue, 11 Oct 2011, Michael Biebl wrote:
>
>> 2011/10/11 Lu, Victor <victor.lu at citi.com>:
>>> Hi there,
>>>
>>> The following links discuss why kill -HUP is not supported by rsyslog, version 5 and above.
>>>
>>> ?http://www.rsyslog.com/doc/v4compatibility.html
>>>
>>> ?http://www.rsyslog.com/doc/v5compatibility.html
>>>
>>> Here is the note from the website: ?That code complexity reduction (and thus performance improvement) needs the restart-type HUP code to be removed, so these changes can (and will) only happen in version 5.
>>>
>>> However, restart of the rsyslog daemon will cause the syslog messages loss. Has anybody thought about it? Is there a way to guarantee no system log message loss like kill -HUP command provided?
>>
>> You could use systemd for that:
>> http://www.freedesktop.org/wiki/Software/systemd/syslog
>
> systemd doesn't solve the problem
>
> the reason that -HUP was changed from being a restart to just reopeing files (which is enough for log rotation to work) is because doing a full shutdown can take a long time, and can eventually timeout and throw away logs anyway.
>
> consider the case where you have messages in your queue that you cannot write to a destination (say a remote server is down for example), unless you have a disk assisted queue setup you cannot do anything except throw these messages away.
>
> you also cannot shutdown without message loss if you have a continuous stream of new messages arriving.
>
> having a -HUP do a full restart caused message loss at every -HUP because of the 'new messages continually arriving' for the common UDP syslog case, but without doing a HUP (or equivalent), you can't roll the log files as rsyslog would continue to write to the old (open) files.
>
> also, changing configurations (which is where you need to do a restart) is a _very_ rare condition compared to log rotation.
>
>
>
> you can avoid message loss on restart by using RELP for your transport 
> protocol and disk assisted queues.
>
> does this solve your problems? or is there some other reason you are 
> looking to do a full restart instead of just re-opening files and 
> network connections on a HUP?
>
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From victor.lu at citi.com  Tue Oct 11 23:36:45 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Tue, 11 Oct 2011 16:36:45 -0500
Subject: [rsyslog] Duplicated kernel messages
Message-ID: <35B12B7283BF44478AFA717323EE52951CD7DC3E22@extxmb32.nam.nsroot.net>

Hi there,

The following message is on Solaris 10 platform.   When I do a su, the messages from kernel always come. I did not see that message when I use syslog daemon. Is this a normal behavior in rsyslog? Is there something to do in compilation because I did not see it on RHEL?


2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
2011-10-11T16:35:21.179019-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2
2011-10-11T16:35:21.182744-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2

2011-10-11T16:36:39.450123-04:00 h8-420r-01 kernel: Oct 11 16:36:39 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
2011-10-11T16:36:39-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
2011-10-11T16:36:39-04:00 h8-420r-01 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2
2011-10-11T16:36:39.454056-04:00 h8-420r-01 kernel: Oct 11 16:36:39 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2

Thanks

Victor

From david at lang.hm  Tue Oct 11 23:44:58 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 11 Oct 2011 14:44:58 -0700 (PDT)
Subject: [rsyslog] Duplicated kernel messages
In-Reply-To: <35B12B7283BF44478AFA717323EE52951CD7DC3E22@extxmb32.nam.nsroot.net>
References: <35B12B7283BF44478AFA717323EE52951CD7DC3E22@extxmb32.nam.nsroot.net>
Message-ID: <alpine.DEB.2.02.1110111442000.28233@asgard.lang.hm>

On Tue, 11 Oct 2011, Lu, Victor  wrote:

> Hi there,
>
> The following message is on Solaris 10 platform.   When I do a su, the messages from kernel always come. I did not see that message when I use syslog daemon. Is this a normal behavior in rsyslog? Is there something to do in compilation because I did not see it on RHEL?
>
>
> 2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
> 2011-10-11T16:35:21.179019-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
> 2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2
> 2011-10-11T16:35:21.182744-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2
>
> 2011-10-11T16:36:39.450123-04:00 h8-420r-01 kernel: Oct 11 16:36:39 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
> 2011-10-11T16:36:39-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED
> 2011-10-11T16:36:39-04:00 h8-420r-01 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2
> 2011-10-11T16:36:39.454056-04:00 h8-420r-01 kernel: Oct 11 16:36:39 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2
>

it looks to me like the log message is probably being delivered to rsyslog 
twice. There is a property that you can put into a template that indicates 
how the log message got to rsyslog ( I don't remember it's name right 
now), I would suggest creating a custom template that includes this and 
then see how the logs are arriving.

the other possibility is that you may have two rules in your rsyslog.conf 
file that are both matching this, but if that was the case I would expect 
the duplicate lines next to each other (but it's possible that the batch 
processing of log messages would produce the result you are seeing)

David Lang

From rpkelly22 at gmail.com  Wed Oct 12 02:17:09 2011
From: rpkelly22 at gmail.com (Ryan Kelly)
Date: Tue, 11 Oct 2011 20:17:09 -0400
Subject: [rsyslog] File-based templates using message content?
Message-ID: <20111012001709.GA16460@llserver.lakeliving.com>

List:

I'm wondering if it is possible to match on and extract some value from
the syslog message, and use that in a template for a file name? I'm
pretty sure this isn't available, or probably even a good idea.

What I had in mind was something like:

if $msg matches '[someflag:(\d+)]' then "/var/log/flagged-$1.log"

So that the matched value's first capture can be used in the template
(obviously incomplete and obviously the 'matches' operator isn't real,
but hopefully you all get the idea).

-Ryan Kelly


From gkra at unnerving.org  Wed Oct 12 02:38:27 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Tue, 11 Oct 2011 17:38:27 -0700 (PDT)
Subject: [rsyslog] templates not working on second-hop relay?
In-Reply-To: <2062946997.5691.1318379378266.JavaMail.root@zmail.binarytribe.com>
Message-ID: <50553165.5695.1318379907806.JavaMail.root@zmail.binarytribe.com>

I'm currently using rsyslog 4.6.5, as packaged by IUS for CentOS 5.6.

I've replicated this problem on rsyslog 5.8.5, compiled from sources on CentOS 5.6 and Ubuntu 11.04.

I've been testing a configuration where there's two hops in my logging.

Basically, client -> local loghost -> archive loghost.

On the client, I'm using a template to add a tag to the start of the $msg property. On the servers, I then look for this tag for dynafile purposes, and use a second template to strip out the tag from $msg before it's written to file or passed to a database. On a simple client -> loghost setup, this works fine.

However, when I then added an archive loghost, which the local loghost relays everything to, none of the templates seem to be working.

The client config can be simplified to:

#---------------------------------------
$template SiteIDForwardFormat, "<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%[SITE:datacenter/dev]%msg:R,ERE,3,FIELD::sp-if-no-1st-sp%%msg%"

*.*				@@loghost:1514;SiteIDForwardFormat
#---------------------------------------

The loghost config is, essentially:

#---------------------------------------
$ModLoad imtcp.so  
$InputTCPServerRun 1514

$CreateDirs on

$template SiteIDTaggedMsg, "%timestamp% %hostname% %syslogtag%%msg:R,ERE,3,FIELD:(\[SITE:([-/a-zA-Z0-9]+)\] ){0,1}(.*)$--end%\n"

$template SiteIDTaggedPath, "/data/syslog/logs/%msg:R,ERE,2,BLANK:(\[SITE:([-/a-zA-Z0-9]+)\] ){0,1}(.*)$--end%/%hostname:::secpath-replace%/%$year%/%$month%/syslog.log"

*.*					?SiteIDTaggedPath;SiteIDTaggedMsg

*.*					@@logarchive
#---------------------------------------

For all intents and purposes, the archive loghost has the exact same config as the local loghost, sans the @@logarchive action.

On the local loghost, messages are being properly filed into the expected dynafiles paths (/data/syslog/logs/datacenter/dev/hostname/year/month/syslog.log) and the "[SITE:something]" text is correctly removed from the messages written to the files.

On the archive loghost, however, the dynafiles are missing the site component (ending up as /data/syslog/logs/hostname/year/month/syslog.log), and all the messages in the files still have the "[SITE:something]" text prepended to the $msg property.

Why is the second (archive) log host not able to properly parse the messages with the templates? am I missing something, or is the relay step munging the event in such a way as the EREs are no longer working as I think they should?

Thanks for any help,

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu


From gkra at unnerving.org  Wed Oct 12 02:49:19 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Tue, 11 Oct 2011 17:49:19 -0700 (PDT)
Subject: [rsyslog] File-based templates using message content?
In-Reply-To: <20111012001709.GA16460@llserver.lakeliving.com>
Message-ID: <1892374811.5701.1318380559143.JavaMail.root@zmail.binarytribe.com>

----- Original Message -----
> What I had in mind was something like:
> 
> if $msg matches '[someflag:(\d+)]' then "/var/log/flagged-$1.log"

I think you'd do that right in a template:

$template FlaggedMsgLog, "/var/log/flagged-%msg:R,ERE,1,ZERO:\[someflag:([0-9]+)\]--end%.log"

And then an action would be something like:

if msg contains '[someflag:' then ?FlaggedMsgLog

Double-check your expressions with the regex checker page (a serious help in my own configs) at: http://www.rsyslog.com/regex/

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu


From david at lang.hm  Wed Oct 12 07:03:52 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 11 Oct 2011 22:03:52 -0700 (PDT)
Subject: [rsyslog] File-based templates using message content?
In-Reply-To: <20111012001709.GA16460@llserver.lakeliving.com>
References: <20111012001709.GA16460@llserver.lakeliving.com>
Message-ID: <alpine.DEB.2.02.1110112202350.12310@asgard.lang.hm>

On Tue, 11 Oct 2011, Ryan Kelly wrote:

> List:
>
> I'm wondering if it is possible to match on and extract some value from
> the syslog message, and use that in a template for a file name? I'm
> pretty sure this isn't available, or probably even a good idea.
>
> What I had in mind was something like:
>
> if $msg matches '[someflag:(\d+)]' then "/var/log/flagged-$1.log"
>
> So that the matched value's first capture can be used in the template
> (obviously incomplete and obviously the 'matches' operator isn't real,
> but hopefully you all get the idea).

the string created by the dynafile template is passed to the OS to create 
the filename (and path), so anything that you can do in a template can be 
part of the filename.

David Lang

From david at lang.hm  Wed Oct 12 07:07:41 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 11 Oct 2011 22:07:41 -0700 (PDT)
Subject: [rsyslog] templates not working on second-hop relay?
In-Reply-To: <50553165.5695.1318379907806.JavaMail.root@zmail.binarytribe.com>
References: <50553165.5695.1318379907806.JavaMail.root@zmail.binarytribe.com>
Message-ID: <alpine.DEB.2.02.1110112205360.12310@asgard.lang.hm>

On Tue, 11 Oct 2011, Gregory K. Ruiz-Ade wrote:

> I'm currently using rsyslog 4.6.5, as packaged by IUS for CentOS 5.6.
>
> I've replicated this problem on rsyslog 5.8.5, compiled from sources on CentOS 5.6 and Ubuntu 11.04.
>
> I've been testing a configuration where there's two hops in my logging.
>
> Basically, client -> local loghost -> archive loghost.
>
> On the client, I'm using a template to add a tag to the start of the $msg property. On the servers, I then look for this tag for dynafile purposes, and use a second template to strip out the tag from $msg before it's written to file or passed to a database. On a simple client -> loghost setup, this works fine.
>
> However, when I then added an archive loghost, which the local loghost relays everything to, none of the templates seem to be working.
>
> The client config can be simplified to:
>
> #---------------------------------------
> $template SiteIDForwardFormat, "<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%[SITE:datacenter/dev]%msg:R,ERE,3,FIELD::sp-if-no-1st-sp%%msg%"
>
> *.*				@@loghost:1514;SiteIDForwardFormat
> #---------------------------------------
>
> The loghost config is, essentially:
>
> #---------------------------------------
> $ModLoad imtcp.so
> $InputTCPServerRun 1514
>
> $CreateDirs on
>
> $template SiteIDTaggedMsg, "%timestamp% %hostname% %syslogtag%%msg:R,ERE,3,FIELD:(\[SITE:([-/a-zA-Z0-9]+)\] ){0,1}(.*)$--end%\n"
>
> $template SiteIDTaggedPath, "/data/syslog/logs/%msg:R,ERE,2,BLANK:(\[SITE:([-/a-zA-Z0-9]+)\] ){0,1}(.*)$--end%/%hostname:::secpath-replace%/%$year%/%$month%/syslog.log"
>
> *.*					?SiteIDTaggedPath;SiteIDTaggedMsg
>
> *.*					@@logarchive
> #---------------------------------------
>
> For all intents and purposes, the archive loghost has the exact same config as the local loghost, sans the @@logarchive action.
>
> On the local loghost, messages are being properly filed into the expected dynafiles paths (/data/syslog/logs/datacenter/dev/hostname/year/month/syslog.log) and the "[SITE:something]" text is correctly removed from the messages written to the files.
>
> On the archive loghost, however, the dynafiles are missing the site component (ending up as /data/syslog/logs/hostname/year/month/syslog.log), and all the messages in the files still have the "[SITE:something]" text prepended to the $msg property.
>
> Why is the second (archive) log host not able to properly parse the messages with the templates? am I missing something, or is the relay step munging the event in such a way as the EREs are no longer working as I think they should?
>
> Thanks for any help,

I think that what you end up doing is manipulating the logfile too much on 
the relay system.

make a temporary output file on the relay system to log with the debug 
format (RSYSLOG_DEBUG IIRC) and you will see everything that is arriving, 
then decide if you need to manipulate it more before sending it on to the 
archive server.

remember that each step in the process is working with what it was sent, 
not what the software originally wrote to the log.

David Lang

From rgerhards at hq.adiscon.com  Wed Oct 12 07:18:47 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Oct 2011 07:18:47 +0200
Subject: [rsyslog] File-based templates using message content?
In-Reply-To: <20111012001709.GA16460@llserver.lakeliving.com>
References: <20111012001709.GA16460@llserver.lakeliving.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281312@GRFEXC.intern.adiscon.com>

It's not directly supported, but you could use the same regexp in both
places. Should be fairly equivalent at the price of a higher performance
toll. It's hard to do what you suggest because filters and actions are two
very different objects in very different code at very different processing
stages.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ryan Kelly
> Sent: Wednesday, October 12, 2011 2:17 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] File-based templates using message content?
> 
> List:
> 
> I'm wondering if it is possible to match on and extract some value from
> the syslog message, and use that in a template for a file name? I'm
> pretty sure this isn't available, or probably even a good idea.
> 
> What I had in mind was something like:
> 
> if $msg matches '[someflag:(\d+)]' then "/var/log/flagged-$1.log"
> 
> So that the matched value's first capture can be used in the template
> (obviously incomplete and obviously the 'matches' operator isn't real,
> but hopefully you all get the idea).
> 
> -Ryan Kelly
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rpkelly22 at gmail.com  Wed Oct 12 13:14:12 2011
From: rpkelly22 at gmail.com (Ryan Kelly)
Date: Wed, 12 Oct 2011 07:14:12 -0400
Subject: [rsyslog] File-based templates using message content?
In-Reply-To: <1892374811.5701.1318380559143.JavaMail.root@zmail.binarytribe.com>
References: <20111012001709.GA16460@llserver.lakeliving.com>
	<1892374811.5701.1318380559143.JavaMail.root@zmail.binarytribe.com>
Message-ID: <20111012111412.GA16772@llserver.lakeliving.com>

>> What I had in mind was something like:
>> 
>> if $msg matches '[someflag:(\d+)]' then "/var/log/flagged-$1.log"
> 
> I think you'd do that right in a template:
> 
> $template FlaggedMsgLog, "/var/log/flagged-%msg:R,ERE,1,ZERO:\[someflag:([0-9]+)\]--end%.log"
> 
> And then an action would be something like:
> 
> if msg contains '[someflag:' then ?FlaggedMsgLog
This does *exactly* what I want. Thank you!

> Double-check your expressions with the regex checker page (a serious help in my own configs) at: http://www.rsyslog.com/regex/
And this is really cool too :)

-Ryan Kelly

From victor.lu at citi.com  Wed Oct 12 15:25:58 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Wed, 12 Oct 2011 08:25:58 -0500
Subject: [rsyslog] Duplicated kernel messages
In-Reply-To: <alpine.DEB.2.02.1110111442000.28233@asgard.lang.hm>
References: <35B12B7283BF44478AFA717323EE52951CD7DC3E22@extxmb32.nam.nsroot.net>
	<alpine.DEB.2.02.1110111442000.28233@asgard.lang.hm>
Message-ID: <35B12B7283BF44478AFA717323EE52951CD7DC4212@extxmb32.nam.nsroot.net>

David,



The message is from imklog module and the only difference for these two messages is that the second message added kernel and timestamp, e.g. kernel: Oct 11 16:35:21 in below message.



2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED

2011-10-11T16:35:21.179019-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED



I am wondering if there is any option when we compile that imklog module to disable the second message.


Thanks

Victor





-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
Sent: Tuesday, October 11, 2011 5:45 PM
To: rsyslog-users
Subject: Re: [rsyslog] Duplicated kernel messages



On Tue, 11 Oct 2011, Lu, Victor  wrote:



> Hi there,

>

> The following message is on Solaris 10 platform.   When I do a su, the messages from kernel always come. I did not see that message when I use syslog daemon. Is this a normal behavior in rsyslog? Is there something to do in compilation because I did not see it on RHEL?

>

>

> 2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED

> 2011-10-11T16:35:21.179019-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED

> 2011-10-11T16:35:21-04:00 h8-420r-01 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2

> 2011-10-11T16:35:21.182744-04:00 h8-420r-01 kernel: Oct 11 16:35:21 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2

>

> 2011-10-11T16:36:39.450123-04:00 h8-420r-01 kernel: Oct 11 16:36:39 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED

> 2011-10-11T16:36:39-04:00 h8-420r-01 su: [ID 805675 auth.notice] User : root  Service : su  TTY : /dev/pts/2  Remote Host : N/A  Remote User : N/A  Status : AUTHENTICATED

> 2011-10-11T16:36:39-04:00 h8-420r-01 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2

> 2011-10-11T16:36:39.454056-04:00 h8-420r-01 kernel: Oct 11 16:36:39 su: [ID 366847 auth.notice] 'su root' succeeded for vl10243 on /dev/pts/2

>



it looks to me like the log message is probably being delivered to rsyslog

twice. There is a property that you can put into a template that indicates

how the log message got to rsyslog ( I don't remember it's name right

now), I would suggest creating a custom template that includes this and

then see how the logs are arriving.



the other possibility is that you may have two rules in your rsyslog.conf

file that are both matching this, but if that was the case I would expect

the duplicate lines next to each other (but it's possible that the batch

processing of log messages would produce the result you are seeing)



David Lang

_______________________________________________

rsyslog mailing list

http://lists.adiscon.net/mailman/listinfo/rsyslog

http://www.rsyslog.com

From gkra at unnerving.org  Wed Oct 12 19:51:40 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Wed, 12 Oct 2011 10:51:40 -0700
Subject: [rsyslog] templates not working on second-hop relay?
In-Reply-To: <alpine.DEB.2.02.1110112205360.12310@asgard.lang.hm>
References: <50553165.5695.1318379907806.JavaMail.root@zmail.binarytribe.com>
	<alpine.DEB.2.02.1110112205360.12310@asgard.lang.hm>
Message-ID: <20111012175139.GA7434@izetta.home.unnerving.org>

On Tue, Oct 11, 2011 at 10:07:41PM -0700, david at lang.hm wrote:

> I think that what you end up doing is manipulating the logfile too
> much on the relay system.

The only manipulations which should be happening on the relay system
should be limited to the file writing action, using the dynafile
template and the format template.  The following action relaying to the
archive server should have no manipulations applied.

> make a temporary output file on the relay system to log with the
> debug format (RSYSLOG_DEBUG IIRC) and you will see everything that
> is arriving, then decide if you need to manipulate it more before
> sending it on to the archive server.

I'm setting this up on my relay server right now. Hopefully you're right
and the answer will jump out at me. :)

> remember that each step in the process is working with what it was
> sent, not what the software originally wrote to the log.

Right, so the relay server should contain the modified $msg format sent
from the client, and since the relay action itself (*.* @@logarchive)
contains no further manipulations, it *should* just pass through as-is,
without further manipulations...

At any rate, to the debugging logfile I go!

Thanks,

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu

From gkra at unnerving.org  Thu Oct 13 01:40:18 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Wed, 12 Oct 2011 16:40:18 -0700
Subject: [rsyslog] templates not working on second-hop relay?
In-Reply-To: <50553165.5695.1318379907806.JavaMail.root@zmail.binarytribe.com>
References: <2062946997.5691.1318379378266.JavaMail.root@zmail.binarytribe.com>
	<50553165.5695.1318379907806.JavaMail.root@zmail.binarytribe.com>
Message-ID: <20111012234018.GA18159@izetta.home.unnerving.org>

Okay, the solution was actually simpler than I thought.

Basically, on the loghost that was relaying to an archive logger, I
needed to define a custom template for the relaying, which would ensure
the syslog events would be sent out exactly as they were coming in. This
ended up being rather simple.

The original template from the rsyslog clients is:

$template SiteIDForwardFormat, "<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%[SITE:sitename]%msg:::sp-if-no-1st-sp%%msg%"

To forward these properly and unmolested to the archive loghost, I
needed to run them through this template on the relay host:

$template SiteIDRelayFormat, "<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg%"

Then, the relay action was split up into a couple actions to make sure
things were forwarded correctly to the archive loghost:

:msg, startswith, "[SITE:"	@@logarchive:1514;SiteIDRelayFormat
:msg, !startswith, "[SITE:"	@@logarchive:1514;SiteIDForwardFormat

I.e., anything that's already tagged gets relayed with the relay format,
and anything that's not has the tag added and is forwarded on.

Thanks for the suggestion of the debug log; running both the relay and
the archiver with debug logging is what led me to the solution.

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu

From rgerhards at hq.adiscon.com  Thu Oct 13 07:41:23 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Oct 2011 07:41:23 +0200
Subject: [rsyslog] Incorrect dynamic file names
In-Reply-To: <CALEqfkjRmCXqeGEjUuAEz1EGuBgUZT_mV9ceLUFmoNJtV1V4QA@mail.gmail.com>
References: <CALEqfkg0ZmaOhLA_aCAmrmP8oUsz0tNh7Zn1iYWq-dxCYa3mjQ@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA72812CE@GRFEXC.intern.adiscon.com><CALEqfkibK4RzoG42exMgRD8fXgzCiGFPPnH-t8=u086msjcc9Q@mail.gmail.com>
	<CALEqfkjRmCXqeGEjUuAEz1EGuBgUZT_mV9ceLUFmoNJtV1V4QA@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728131F@GRFEXC.intern.adiscon.com>

Just FYI: I am handling this via the forum now.

rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Brad Ison
> Sent: Saturday, October 08, 2011 7:30 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Incorrect dynamic file names
> 
> I just wanted to follow up to say that I've reproduced this with 5.8.5
> and 6.1.12 now. I was able to capture debug output from 6.1.12.
> 
> I've created a forum post as that seems a more appropriate place to
> post debug logs:
> 
> http://kb.monitorware.com/incorrect-dynamic-file-names-t11001.html
> 
> --
> Brad
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From horvath.peter77 at freemail.hu  Thu Oct 13 11:03:29 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Thu, 13 Oct 2011 10:03:29 +0100
Subject: [rsyslog] Dynamic file names
Message-ID: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>

I would like to get opinions about this:

I have the following line in my rsyslog conf:
$template DynFile,"/var/log/syslog-%HOSTNAME%"
*.*;auth,authpriv.none  ?DynFile

And it is not working.
After hours of different tries realized if i remove ;auth,authpriv.none
It starts to work magically.
$template DynFile,"/var/log/syslog-%HOSTNAME%"
*.*  ?DynFile

However i had to touch the files manually because these error messages
appeared in the log:
rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' - discarding message

It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0

Do you have any idea what the problem with my original try and why
rsyslog cannot open logfiles?


Thank you
Peter

From rpkelly22 at gmail.com  Thu Oct 13 12:47:34 2011
From: rpkelly22 at gmail.com (Ryan Kelly)
Date: Thu, 13 Oct 2011 06:47:34 -0400
Subject: [rsyslog] Dynamic file names
In-Reply-To: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>
Message-ID: <20111013104734.GA17060@llserver.lakeliving.com>

> I would like to get opinions about this:
> 
> I have the following line in my rsyslog conf:
> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> *.*;auth,authpriv.none  ?DynFile
> 
> And it is not working.
At a glance it looks ok. Try invoking rsyslog with -N1 to see if it
complains about your configuration.

> After hours of different tries realized if i remove ;auth,authpriv.none
> It starts to work magically.
> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> *.*  ?DynFile
> 
> However i had to touch the files manually because these error messages
> appeared in the log:
> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' - discarding message
The dynamic files aren't created when rsyslog starts, so it needs
permission to write them after it drops permissions (the default
configuration in Ubuntu). If you try to write the file to /var/log
(which you are) you will get this error because /var/log is owned by
root and syslog cannot write new files there. At our site we work around
this by creating a new folder owned by syslog.

> It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0
The important lines to note are these:
$PrivDropToUser syslog
$PrivDropToGroup syslog

Which are why the file can't be created dynamically in /var/log.

> Do you have any idea what the problem with my original try and why
> rsyslog cannot open logfiles?

-Ryan Kelly

From a.piesk at gmx.net  Thu Oct 13 19:44:10 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Thu, 13 Oct 2011 19:44:10 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs if
 remote destination is unresponsive
Message-ID: <4E97236A.4010409@gmx.net>

Hello list,

recently i had big trouble with rsyslog 5.8.5 64bit on RHEL5 and would like to know why it happened
and how to fix it.

the setup:

machine A with rsyslog logs locally and forwards everything to machine B.

rsyslog configuration of A:

# cat /etc/rsyslog.conf
$ModLoad imuxsock
$ModLoad imklog
$ModLoad immark
$MarkMessagePeriod 1200
$SystemLogRateLimitInterval 0
$ActionFileDefaultTemplate RSYSLOG_FileFormat
*.info;mail.none;authpriv.none;cron.none;local6.none;local0.none    /var/log/messages
authpriv.*                                              /var/log/secure
mail.*                                                  -/var/log/maillog
cron.*                                                  /var/log/cron
*.emerg                                                 *
uucp,news.crit                                          /var/log/spooler
local7.*                                                /var/log/boot
$IncludeConfig /etc/rsyslog.d/*.conf

# cat /etc/rsyslog.d/forwarder.conf
$ActionForwardDefaultTemplate RSYSLOG_ForwardFormat
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList
$ActionQueueMaxDiskSpace 1024m
$ActionQueueHighWatermark 100
$ActionQueueLowWatermark 10
$ActionQueueCheckpointInterval 10
$ActionQueueFileName forward
$ActionQueueMaxFileSize 10m
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$ActionWriteAllMarkMessages on
*.*     @@machine_B:514;RSYSLOG_ForwardFormat

what happened:

machine B kindly stopped with a kernel panic.
for unknown reasons all TCP packets were dropped, TCP sessions hung until TCP timed out.

machine A stopped local logging (no disk spool file was created). after some time all applicaions
which log via syslog became unresponsive (a simple login took ages). after stopping rsyslog
everything went back to normal.


i was able to reproduce it by simulation the TCP black-hole with iptables:

machine A:
# while true; do date | logger; sleep 1; done

machine B:

2011-10-11T17:38:38.000000+02:00 machine_A logger: Tue Oct 11 17:38:38 CEST 2011
2011-10-11T17:38:39.000000+02:00 machine_A logger: Tue Oct 11 17:38:39 CEST 2011
2011-10-11T17:38:40.000000+02:00 machine_Alogger: Tue Oct 11 17:38:40 CEST 2011
2011-10-11T17:38:41.000000+02:00 machine_Alogger: Tue Oct 11 17:38:41 CEST 2011
2011-10-11T17:38:42.000000+02:00 machine_Alogger: Tue Oct 11 17:38:42 CEST 2011
2011-10-11T17:38:43.000000+02:00 machine_Alogger: Tue Oct 11 17:38:43 CEST 2011
2011-10-11T17:38:44.000000+02:00 machine_Alogger: Tue Oct 11 17:38:44 CEST 2011
2011-10-11T17:38:45.000000+02:00 machine_Alogger: Tue Oct 11 17:38:45 CEST 2011
2011-10-11T17:38:46.000000+02:00 machine_Alogger: Tue Oct 11 17:38:46 CEST 2011
2011-10-11T17:38:47.000000+02:00 machine_Alogger: Tue Oct 11 17:38:47 CEST 2011
<iptables -A OUTPUT -p tcp --dport 514 -j DROP>

machine A:

2011-10-11T17:39:20+02:00 machine_Alogger: Tue Oct 11 17:39:20 CEST 2011
2011-10-11T17:39:21+02:00 machine_Alogger: Tue Oct 11 17:39:21 CEST 2011
2011-10-11T17:39:22+02:00 machine_Alogger: Tue Oct 11 17:39:22 CEST 2011
2011-10-11T17:39:23+02:00 machine_Alogger: Tue Oct 11 17:39:23 CEST 2011
2011-10-11T17:39:24+02:00 machine_Alogger: Tue Oct 11 17:39:24 CEST 2011
2011-10-11T17:39:25+02:00 machine_Alogger: Tue Oct 11 17:39:25 CEST 2011
2011-10-11T17:39:26+02:00 machine_Alogger: Tue Oct 11 17:39:26 CEST 2011
2011-10-11T17:39:27+02:00 machine_Alogger: Tue Oct 11 17:39:27 CEST 2011
<logging stops>

i expected that rsyslog starts spooling to disk if the remote server is unreachable but this is not
the case if the TCP sessions hangs:
 
# ls -l /var/spool/rsyslog/forward*
ls: /var/spool/rsyslog/forward*: No such file or directory

i ran the same test with iptables .. -j REJECT and everything worked like expected.


is the observed behaviour correct? should local logging stop if the remote server is unresponsive?
why didn't switch rsyslog to spooling? because the TCP session to the remote server hung?
what can i do on the rsyslog part to prevent such a scenario from happen again?

i would like to accomplish the following:

if remote destination is unreachable/unresponsive switch to disk spooling but continue to log locally.
after the connection has been re-established transfer the backlog to the remote destination.

is this possible? and if yes, how?

regards,
-ap


From mike.forbes at koordinates.com  Thu Oct 13 21:05:25 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Fri, 14 Oct 2011 08:05:25 +1300
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
 if remote destination is unresponsive
In-Reply-To: <4E97236A.4010409@gmx.net>
References: <4E97236A.4010409@gmx.net>
Message-ID: <CADWNQmJMG-jVdEcb6EQRzdD7RaCrimVWVtVBdmH_STdooVo_Bw@mail.gmail.com>

I can confirm this too on ubuntu lucid with rsyslogd 5.8.5, 64bit.


On Fri, Oct 14, 2011 at 6:44 AM, Andreas Piesk <a.piesk at gmx.net> wrote:
> Hello list,
>
> recently i had big trouble with rsyslog 5.8.5 64bit on RHEL5 and would like to know why it happened
> and how to fix it.
>
> the setup:
>
> machine A with rsyslog logs locally and forwards everything to machine B.
>
> rsyslog configuration of A:
>
> # cat /etc/rsyslog.conf
> $ModLoad imuxsock
> $ModLoad imklog
> $ModLoad immark
> $MarkMessagePeriod 1200
> $SystemLogRateLimitInterval 0
> $ActionFileDefaultTemplate RSYSLOG_FileFormat
> *.info;mail.none;authpriv.none;cron.none;local6.none;local0.none ? ?/var/log/messages
> authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure
> mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog
> cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron
> *.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? *
> uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler
> local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot
> $IncludeConfig /etc/rsyslog.d/*.conf
>
> # cat /etc/rsyslog.d/forwarder.conf
> $ActionForwardDefaultTemplate RSYSLOG_ForwardFormat
> $WorkDirectory /var/spool/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueMaxDiskSpace 1024m
> $ActionQueueHighWatermark 100
> $ActionQueueLowWatermark 10
> $ActionQueueCheckpointInterval 10
> $ActionQueueFileName forward
> $ActionQueueMaxFileSize 10m
> $ActionResumeRetryCount -1
> $ActionQueueSaveOnShutdown on
> $ActionWriteAllMarkMessages on
> *.* ? ? @@machine_B:514;RSYSLOG_ForwardFormat
>
> what happened:
>
> machine B kindly stopped with a kernel panic.
> for unknown reasons all TCP packets were dropped, TCP sessions hung until TCP timed out.
>
> machine A stopped local logging (no disk spool file was created). after some time all applicaions
> which log via syslog became unresponsive (a simple login took ages). after stopping rsyslog
> everything went back to normal.
>
>
> i was able to reproduce it by simulation the TCP black-hole with iptables:
>
> machine A:
> # while true; do date | logger; sleep 1; done
>
> machine B:
>
> 2011-10-11T17:38:38.000000+02:00 machine_A logger: Tue Oct 11 17:38:38 CEST 2011
> 2011-10-11T17:38:39.000000+02:00 machine_A logger: Tue Oct 11 17:38:39 CEST 2011
> 2011-10-11T17:38:40.000000+02:00 machine_Alogger: Tue Oct 11 17:38:40 CEST 2011
> 2011-10-11T17:38:41.000000+02:00 machine_Alogger: Tue Oct 11 17:38:41 CEST 2011
> 2011-10-11T17:38:42.000000+02:00 machine_Alogger: Tue Oct 11 17:38:42 CEST 2011
> 2011-10-11T17:38:43.000000+02:00 machine_Alogger: Tue Oct 11 17:38:43 CEST 2011
> 2011-10-11T17:38:44.000000+02:00 machine_Alogger: Tue Oct 11 17:38:44 CEST 2011
> 2011-10-11T17:38:45.000000+02:00 machine_Alogger: Tue Oct 11 17:38:45 CEST 2011
> 2011-10-11T17:38:46.000000+02:00 machine_Alogger: Tue Oct 11 17:38:46 CEST 2011
> 2011-10-11T17:38:47.000000+02:00 machine_Alogger: Tue Oct 11 17:38:47 CEST 2011
> <iptables -A OUTPUT -p tcp --dport 514 -j DROP>
>
> machine A:
>
> 2011-10-11T17:39:20+02:00 machine_Alogger: Tue Oct 11 17:39:20 CEST 2011
> 2011-10-11T17:39:21+02:00 machine_Alogger: Tue Oct 11 17:39:21 CEST 2011
> 2011-10-11T17:39:22+02:00 machine_Alogger: Tue Oct 11 17:39:22 CEST 2011
> 2011-10-11T17:39:23+02:00 machine_Alogger: Tue Oct 11 17:39:23 CEST 2011
> 2011-10-11T17:39:24+02:00 machine_Alogger: Tue Oct 11 17:39:24 CEST 2011
> 2011-10-11T17:39:25+02:00 machine_Alogger: Tue Oct 11 17:39:25 CEST 2011
> 2011-10-11T17:39:26+02:00 machine_Alogger: Tue Oct 11 17:39:26 CEST 2011
> 2011-10-11T17:39:27+02:00 machine_Alogger: Tue Oct 11 17:39:27 CEST 2011
> <logging stops>
>
> i expected that rsyslog starts spooling to disk if the remote server is unreachable but this is not
> the case if the TCP sessions hangs:
>
> # ls -l /var/spool/rsyslog/forward*
> ls: /var/spool/rsyslog/forward*: No such file or directory
>
> i ran the same test with iptables .. -j REJECT and everything worked like expected.
>
>
> is the observed behaviour correct? should local logging stop if the remote server is unresponsive?
> why didn't switch rsyslog to spooling? because the TCP session to the remote server hung?
> what can i do on the rsyslog part to prevent such a scenario from happen again?
>
> i would like to accomplish the following:
>
> if remote destination is unreachable/unresponsive switch to disk spooling but continue to log locally.
> after the connection has been re-established transfer the backlog to the remote destination.
>
> is this possible? and if yes, how?
>
> regards,
> -ap
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>



-- 
// Mike
GPG: BFC7 3F32 2CCF D91F 53E1 ?DF88 1578 B2E4 1399 6844

From rgerhards at hq.adiscon.com  Thu Oct 13 21:25:43 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Oct 2011 21:25:43 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
	if remote destination is unresponsive
Message-ID: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com>

Watermarks are extremely low, this may cause problems. Can you provide debug log?

Rainer
Mike Forbes <mike.forbes at koordinates.com> hat geschrieben:I can confirm this too on ubuntu lucid with rsyslogd 5.8.5, 64bit.


On Fri, Oct 14, 2011 at 6:44 AM, Andreas Piesk <a.piesk at gmx.net> wrote:
> Hello list,
>
> recently i had big trouble with rsyslog 5.8.5 64bit on RHEL5 and would like to know why it happened
> and how to fix it.
>
> the setup:
>
> machine A with rsyslog logs locally and forwards everything to machine B.
>
> rsyslog configuration of A:
>
> # cat /etc/rsyslog.conf
> $ModLoad imuxsock
> $ModLoad imklog
> $ModLoad immark
> $MarkMessagePeriod 1200
> $SystemLogRateLimitInterval 0
> $ActionFileDefaultTemplate RSYSLOG_FileFormat
> *.info;mail.none;authpriv.none;cron.none;local6.none;local0.none    /var/log/messages
> authpriv.*                                              /var/log/secure
> mail.*                                                  -/var/log/maillog
> cron.*                                                  /var/log/cron
> *.emerg                                                 *
> uucp,news.crit                                          /var/log/spooler
> local7.*                                                /var/log/boot
> $IncludeConfig /etc/rsyslog.d/*.conf
>
> # cat /etc/rsyslog.d/forwarder.conf
> $ActionForwardDefaultTemplate RSYSLOG_ForwardFormat
> $WorkDirectory /var/spool/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueMaxDiskSpace 1024m
> $ActionQueueHighWatermark 100
> $ActionQueueLowWatermark 10
> $ActionQueueCheckpointInterval 10
> $ActionQueueFileName forward
> $ActionQueueMaxFileSize 10m
> $ActionResumeRetryCount -1
> $ActionQueueSaveOnShutdown on
> $ActionWriteAllMarkMessages on
> *.*     @@machine_B:514;RSYSLOG_ForwardFormat
>
> what happened:
>
> machine B kindly stopped with a kernel panic.
> for unknown reasons all TCP packets were dropped, TCP sessions hung until TCP timed out.
>
> machine A stopped local logging (no disk spool file was created). after some time all applicaions
> which log via syslog became unresponsive (a simple login took ages). after stopping rsyslog
> everything went back to normal.
>
>
> i was able to reproduce it by simulation the TCP black-hole with iptables:
>
> machine A:
> # while true; do date | logger; sleep 1; done
>
> machine B:
>
> 2011-10-11T17:38:38.000000+02:00 machine_A logger: Tue Oct 11 17:38:38 CEST 2011
> 2011-10-11T17:38:39.000000+02:00 machine_A logger: Tue Oct 11 17:38:39 CEST 2011
> 2011-10-11T17:38:40.000000+02:00 machine_Alogger: Tue Oct 11 17:38:40 CEST 2011
> 2011-10-11T17:38:41.000000+02:00 machine_Alogger: Tue Oct 11 17:38:41 CEST 2011
> 2011-10-11T17:38:42.000000+02:00 machine_Alogger: Tue Oct 11 17:38:42 CEST 2011
> 2011-10-11T17:38:43.000000+02:00 machine_Alogger: Tue Oct 11 17:38:43 CEST 2011
> 2011-10-11T17:38:44.000000+02:00 machine_Alogger: Tue Oct 11 17:38:44 CEST 2011
> 2011-10-11T17:38:45.000000+02:00 machine_Alogger: Tue Oct 11 17:38:45 CEST 2011
> 2011-10-11T17:38:46.000000+02:00 machine_Alogger: Tue Oct 11 17:38:46 CEST 2011
> 2011-10-11T17:38:47.000000+02:00 machine_Alogger: Tue Oct 11 17:38:47 CEST 2011
> <iptables -A OUTPUT -p tcp --dport 514 -j DROP>
>
> machine A:
>
> 2011-10-11T17:39:20+02:00 machine_Alogger: Tue Oct 11 17:39:20 CEST 2011
> 2011-10-11T17:39:21+02:00 machine_Alogger: Tue Oct 11 17:39:21 CEST 2011
> 2011-10-11T17:39:22+02:00 machine_Alogger: Tue Oct 11 17:39:22 CEST 2011
> 2011-10-11T17:39:23+02:00 machine_Alogger: Tue Oct 11 17:39:23 CEST 2011
> 2011-10-11T17:39:24+02:00 machine_Alogger: Tue Oct 11 17:39:24 CEST 2011
> 2011-10-11T17:39:25+02:00 machine_Alogger: Tue Oct 11 17:39:25 CEST 2011
> 2011-10-11T17:39:26+02:00 machine_Alogger: Tue Oct 11 17:39:26 CEST 2011
> 2011-10-11T17:39:27+02:00 machine_Alogger: Tue Oct 11 17:39:27 CEST 2011
> <logging stops>
>
> i expected that rsyslog starts spooling to disk if the remote server is unreachable but this is not
> the case if the TCP sessions hangs:
>
> # ls -l /var/spool/rsyslog/forward*
> ls: /var/spool/rsyslog/forward*: No such file or directory
>
> i ran the same test with iptables .. -j REJECT and everything worked like expected.
>
>
> is the observed behaviour correct? should local logging stop if the remote server is unresponsive?
> why didn't switch rsyslog to spooling? because the TCP session to the remote server hung?
> what can i do on the rsyslog part to prevent such a scenario from happen again?
>
> i would like to accomplish the following:
>
> if remote destination is unreachable/unresponsive switch to disk spooling but continue to log locally.
> after the connection has been re-established transfer the backlog to the remote destination.
>
> is this possible? and if yes, how?
>
> regards,
> -ap
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>



-- 
// Mike
GPG: BFC7 3F32 2CCF D91F 53E1  DF88 1578 B2E4 1399 6844
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From gkra at unnerving.org  Thu Oct 13 21:25:58 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Thu, 13 Oct 2011 12:25:58 -0700
Subject: [rsyslog] RPM or SPEC for rsyslog 5?
Message-ID: <E3D595D4-0832-436A-8721-4F6848A7C57E@unnerving.org>

Does anyone have RPMs, SRPMs or a SPEC file they'd be willing to share for rsyslog 5? I'm not really finding anything via google (my google fu is weak today).

Thanks,

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu




From chb at muc.de  Thu Oct 13 21:36:52 2011
From: chb at muc.de (Christian Brunner)
Date: Thu, 13 Oct 2011 21:36:52 +0200
Subject: [rsyslog] RPM or SPEC for rsyslog 5?
In-Reply-To: <E3D595D4-0832-436A-8721-4F6848A7C57E@unnerving.org>
References: <E3D595D4-0832-436A-8721-4F6848A7C57E@unnerving.org>
Message-ID: <CAO47_-9Na5F2R_mk636ASAC6mZ7euZZQ5sYMSCbZthBN4uWZ7Q@mail.gmail.com>

You can find a SPEC file for 5.8.5 in the fedora-pkg git:

git://pkgs.fedoraproject.org/rsyslog.git

or

http://pkgs.fedoraproject.org/gitweb/?p=rsyslog.git;a=blob;f=rsyslog.spec;h=e6c5fadd47fb83e08842fade1f049230bdb658d6;hb=master

Regards,
Christian

2011/10/13 Gregory K. Ruiz-Ade <gkra at unnerving.org>:
> Does anyone have RPMs, SRPMs or a SPEC file they'd be willing to share for rsyslog 5? I'm not really finding anything via google (my google fu is weak today).
>
> Thanks,
>
> Gregory
>
> --
> Gregory K. Ruiz-Ade <gkra at unnerving.org>
> OpenPGP Key ID: EAF4844B ?keyserver: pgpkeys.mit.edu
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From mike.forbes at koordinates.com  Thu Oct 13 21:50:47 2011
From: mike.forbes at koordinates.com (Mike Forbes)
Date: Fri, 14 Oct 2011 08:50:47 +1300
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
 if remote destination is unresponsive
In-Reply-To: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com>
References: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com>
Message-ID: <CADWNQmLoy-oayxNK5e3jtTFN=y7jj+rjrTN0kC92c7gcFm-SwA@mail.gmail.com>

On Fri, Oct 14, 2011 at 8:25 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Watermarks are extremely low, this may cause problems. Can you provide debug log?
>

What would you like, specifically? I understand the debug logs can be
rather large.

From victor.lu at citi.com  Thu Oct 13 22:53:30 2011
From: victor.lu at citi.com (Lu, Victor )
Date: Thu, 13 Oct 2011 15:53:30 -0500
Subject: [rsyslog] Duplicated messages on Solaris
In-Reply-To: <FFA1DBC60A814545906CB1DCDA645ED01DCE7D279B@extxmb32.nam.nsroot.net>
References: <35B12B7283BF44478AFA717323EE52951CA8AB3585@extxmb32.nam.nsroot.net>
	<9B6E2A8877C38245BFB15CC491A11DA72812A9@GRFEXC.intern.adiscon.com>
	<FFA1DBC60A814545906CB1DCDA645ED01DCE7D279B@extxmb32.nam.nsroot.net>
Message-ID: <35B12B7283BF44478AFA717323EE52951CD811A16F@extxmb32.nam.nsroot.net>

FYI,

I have verified and tested, on solaris, imsolaris captures both kernel logging and local system logging, e.g via logger command. Looks like the imklog module is not needed. With imklog module loaded, it will generate duplicated messages. 

Thanks

Victor

-----Original Message-----
From: Lu, Victor [CCC-OT_IT] 
Sent: Monday, September 26, 2011 1:25 PM
To: rsyslog-users
Subject: RE: [rsyslog] Duplicated messages on Solaris

Hi Rainer,

Thanks for quick response. For product version, I am using the latest stable version 5.8.5. Could you let me know which version fixed timestamp issue and how the message look like after the fix.

The following  is what you posted on the web site. Is this because of special kernel input device that produced duplicated message? Any suggestions to have the same behavior like what we have on Linux?  

Website

http://www.rsyslog.com/doc/imsolaris.html

Solaris Input Module

Module Name:    imsolaris

Author: Rainer Gerhards <rgerhards at adiscon.com>

Description:

Reads local Solaris log messages including the kernel log.

This module is specifically tailored for Solaris. Under Solaris, there is no special kernel input device. Instead, both kernel messages as well as messages emitted via syslog() are received from a single source.

This module obeys the Solaris door() mechanism to detect a running syslogd instance. As such, only one can be active at one time. If it detects another active intance at startup, the module disables itself, but rsyslog will continue to run.

Configuration Directives:

    $IMSolarisLogSocketName <name>
    This is the name of the log socket (stream) to read. If not given, /dev/log is read.



-----Original Message-----
From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
Sent: Monday, September 26, 2011 12:55 PM
To: rsyslog-users
Subject: Re: [rsyslog] Duplicated messages on Solaris

Mhhh... I have no idea why Solaris' logger writes to both locations. But I
also don't see how I should tell which one to drop...

As of the timestamps: are you sure you use the newest version of the branch
in question? I remember that I recently fixed something in that regard.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Lu, Victor
> Sent: Monday, September 26, 2011 6:19 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Duplicated messages on Solaris
> 
> On Solaris,
> 
> 
> 1)      If I use both $Modload  ImkLog and $Modload imsolaris,
> 
>        A logger command will always generate message twice.
>        2011-09-26T11:08:46-04:00 i8-420-02 test: [ID 702911
> user.notice] This is a test
>         2011-09-26T11:08:46.962612-04:00 i8-420-02 kernel: Sep 26
> 11:08:46 test: [ID 702911 user.notice] This is a test
> 
>         su command will return only one message.
>         2011-09-26T12:08:21.643321-04:00 i8-420-02 kernel: Sep 26
> 12:08:21 su: [ID 366847 auth.info] 'su root' succeeded for vl10243 on
> /dev/pts/4
> 
> 
> 2)      If I use $Modload imklog only, the logger command will return
> only one message.
> 
>        2011-09-26T12:02:20.667780-04:00 i8-420-02 kernel: Sep 26
> 12:02:20 test: [ID 702911 user.notice] this is a test
> 
>         su command will return only one message.
> 
>          2011-09-26T12:02:47.700657-04:00 i8-420-02 kernel: Sep 26
> 12:02:47 su: [ID 366847 auth.info] 'su root' succeeded for vl10243 on
> /dev/pts/4
> 
> 
> 3)      If I use $Modload imsolaris only
> 
>         The logger command will return the following message.
> 
>        2011-09-26T12:06:01-04:00 i8-420-02 test: [ID 702911
> user.notice] this is a test
> 
>       su command will not return any message.
> 
> I only need one message to be generated in the system log (same on
> Linux), not duplicated.
> 
> It looks like I can use imklog module alone to capture both kernel and
> logger command message. But I am not sure if I still could miss other
> type of system events without using imsolaris module.
> 
> For the kernel message generated, I don't like duplicated time stamp
> 
> For example, the following event,
> 2011-09-26T12:02:20.667780-04:00 i8-420-02 kernel: Sep 26 12:02:20
> test: [ID 702911 user.notice] this is a test
> 
> The timestamp  after kernel: Sep 26 12:02:20  because I already have
> the event time 2011-09-26T12:02:20.667780-04:00.
> 
> Any suggestions? Anybody have a sample rsyslog.conf on Solaris to
> share?
> 
> Thanks
> 
> Victor Lu
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Fri Oct 14 07:16:31 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Oct 2011 07:16:31 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
	if remote destination is unresponsive
In-Reply-To: <CADWNQmLoy-oayxNK5e3jtTFN=y7jj+rjrTN0kC92c7gcFm-SwA@mail.gmail.com>
References: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com>
	<CADWNQmLoy-oayxNK5e3jtTFN=y7jj+rjrTN0kC92c7gcFm-SwA@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281330@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mike Forbes
> Sent: Thursday, October 13, 2011 9:51 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> hangs if remote destination is unresponsive
> 
> On Fri, Oct 14, 2011 at 8:25 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> > Watermarks are extremely low, this may cause problems. Can you
> provide debug log?
> >
> 
> What would you like, specifically? I understand the debug logs can be
> rather large.

Can't say before I see it. Up to 2..4 gig is OK (zipped ;)). But I'd first
increase the watermark settings.

Rainer

From david at lang.hm  Fri Oct 14 07:36:40 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 13 Oct 2011 22:36:40 -0700 (PDT)
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
 if remote destination is unresponsive
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281330@GRFEXC.intern.adiscon.com>
References: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com>
	<CADWNQmLoy-oayxNK5e3jtTFN=y7jj+rjrTN0kC92c7gcFm-SwA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281330@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1110132234540.12310@asgard.lang.hm>

this is probably a stupid question, but where in the configuration is the 
disk assisted queue enabled?

I see the queue type as being "linked list", which I thought was a 
memory-only queue type.

without the disk assist type won't it just fill the allocated memory and 
stop?

David Lang

On Fri, 14 Oct 2011, Rainer Gerhards wrote:

>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mike Forbes
>> Sent: Thursday, October 13, 2011 9:51 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
>> hangs if remote destination is unresponsive
>>
>> On Fri, Oct 14, 2011 at 8:25 AM, Rainer Gerhards
>> <rgerhards at hq.adiscon.com> wrote:
>>> Watermarks are extremely low, this may cause problems. Can you
>> provide debug log?
>>>
>>
>> What would you like, specifically? I understand the debug logs can be
>> rather large.
>
> Can't say before I see it. Up to 2..4 gig is OK (zipped ;)). But I'd first
> increase the watermark settings.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Fri Oct 14 08:52:24 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Oct 2011 08:52:24 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
	if remote destination is unresponsive
In-Reply-To: <alpine.DEB.2.02.1110132234540.12310@asgard.lang.hm>
References: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com><CADWNQmLoy-oayxNK5e3jtTFN=y7jj+rjrTN0kC92c7gcFm-SwA@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA7281330@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1110132234540.12310@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281331@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, October 14, 2011 7:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> hangs if remote destination is unresponsive
> 
> this is probably a stupid question, but where in the configuration is
> the
> disk assisted queue enabled?
> 
> I see the queue type as being "linked list", which I thought was a
> memory-only queue type.
> 
> without the disk assist type won't it just fill the allocated memory
> and
> stop?


Lol, you are right ;)

Rainer
> David Lang
> 
> On Fri, 14 Oct 2011, Rainer Gerhards wrote:
> 
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Mike Forbes
> >> Sent: Thursday, October 13, 2011 9:51 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> >> hangs if remote destination is unresponsive
> >>
> >> On Fri, Oct 14, 2011 at 8:25 AM, Rainer Gerhards
> >> <rgerhards at hq.adiscon.com> wrote:
> >>> Watermarks are extremely low, this may cause problems. Can you
> >> provide debug log?
> >>>
> >>
> >> What would you like, specifically? I understand the debug logs can
> be
> >> rather large.
> >
> > Can't say before I see it. Up to 2..4 gig is OK (zipped ;)). But I'd
> first
> > increase the watermark settings.
> >
> > Rainer
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From a.piesk at gmx.net  Fri Oct 14 19:46:11 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Fri, 14 Oct 2011 19:46:11 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
 if remote destination is unresponsive
In-Reply-To: <alpine.DEB.2.02.1110132234540.12310@asgard.lang.hm>
References: <005a01cc89dd$b1e9f59a$100013ac@intern.adiscon.com>	<CADWNQmLoy-oayxNK5e3jtTFN=y7jj+rjrTN0kC92c7gcFm-SwA@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA7281330@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1110132234540.12310@asgard.lang.hm>
Message-ID: <4E987563.5090809@gmx.net>

On 14.10.2011 07:36, david at lang.hm wrote:
> this is probably a stupid question, but where in the configuration is the disk assisted queue enabled?
> 
> I see the queue type as being "linked list", which I thought was a memory-only queue type.
> 
> without the disk assist type won't it just fill the allocated memory and stop?
> 

i followed the example given here http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html.

i thought a DA-queue is an in-memory queue with a filename (in forwarder.conf):

$ActionQueueType LinkedList
$ActionQueueFileName forward

as explained in http://www.rsyslog.com/doc/queues.html:

Disk-Assisted Memory Queues
If a disk queue name is defined for in-memory queues (via $<object>QueueFileName), they
automatically become "disk-assisted" (DA). In that mode, data is written to disk (and read back) on
an as-needed basis.


what must be changed in the configs to achieve the desired behaviour? i don't know what's wrong with
the configuration so i ask you guys.


BTW: i have a debug log ready but if it's configuration error, well, then it's not needed.

regards,
-ap

From rgerhards at hq.adiscon.com  Fri Oct 14 20:07:25 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Oct 2011 20:07:25 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
	if remote destination is unresponsive
Message-ID: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com>

Sorry, i was too quick, you are right. 

Before going to the debug log: have you increased the watermarks?

Rainet

Andreas Piesk <a.piesk at gmx.net> hat geschrieben:On 14.10.2011 07:36, david at lang.hm wrote:
> this is probably a stupid question, but where in the configuration is the disk assisted queue enabled?
> 
> I see the queue type as being "linked list", which I thought was a memory-only queue type.
> 
> without the disk assist type won't it just fill the allocated memory and stop?
> 

i followed the example given here http://www.rsyslog.com/doc/rsyslog_reliable_forwarding.html.

i thought a DA-queue is an in-memory queue with a filename (in forwarder.conf):

$ActionQueueType LinkedList
$ActionQueueFileName forward

as explained in http://www.rsyslog.com/doc/queues.html:

Disk-Assisted Memory Queues
If a disk queue name is defined for in-memory queues (via $<object>QueueFileName), they
automatically become "disk-assisted" (DA). In that mode, data is written to disk (and read back) on
an as-needed basis.


what must be changed in the configs to achieve the desired behaviour? i don't know what's wrong with
the configuration so i ask you guys.


BTW: i have a debug log ready but if it's configuration error, well, then it's not needed.

regards,
-ap
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From a.piesk at gmx.net  Fri Oct 14 20:22:40 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Fri, 14 Oct 2011 20:22:40 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
 if remote destination is unresponsive
In-Reply-To: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com>
Message-ID: <4E987DF0.2070707@gmx.net>

On 14.10.2011 20:07, Rainer Gerhards wrote:
> Sorry, i was too quick, you are right. 

no problem.

> Before going to the debug log: have you increased the watermarks?

yes, i tried the same test with default watermarks. got the same result, the logging just stops
later (about 5 mins instead of 45s).

the debug log is from the test run with default watermarks. it's about 180k bzip2-compressed.

should i send it to rgerhards at hq.adiscon.com?

regards,
-ap

From rgerhards at hq.adiscon.com  Sat Oct 15 10:28:04 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 15 Oct 2011 10:28:04 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangs
	if remote destination is unresponsive
In-Reply-To: <4E987DF0.2070707@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com>
	<4E987DF0.2070707@gmx.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andreas Piesk
> Sent: Friday, October 14, 2011 8:23 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> hangs if remote destination is unresponsive
> 
> On 14.10.2011 20:07, Rainer Gerhards wrote:
> > Sorry, i was too quick, you are right.
> 
> no problem.
> 
> > Before going to the debug log: have you increased the watermarks?
> 
> yes, i tried the same test with default watermarks. got the same
> result, the logging just stops
> later (about 5 mins instead of 45s).
> 
> the debug log is from the test run with default watermarks. it's about
> 180k bzip2-compressed.
> 
> should i send it to rgerhards at hq.adiscon.com?

Yes, pls do.

Rainer
> 
> regards,
> -ap
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Sat Oct 15 12:49:54 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 15 Oct 2011 12:49:54 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
	remote destination is unresponsive
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>

Thanks for the debug log. I've so far just have a quick look, but there seem
to come no more messages in. Can you point me to exactly what happens and
what is problematic, if possible in sequence so that I can try to find that
inside the debug log.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Saturday, October 15, 2011 10:28 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> hangsif remote destination is unresponsive
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andreas Piesk
> > Sent: Friday, October 14, 2011 8:23 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> > hangs if remote destination is unresponsive
> >
> > On 14.10.2011 20:07, Rainer Gerhards wrote:
> > > Sorry, i was too quick, you are right.
> >
> > no problem.
> >
> > > Before going to the debug log: have you increased the watermarks?
> >
> > yes, i tried the same test with default watermarks. got the same
> > result, the logging just stops
> > later (about 5 mins instead of 45s).
> >
> > the debug log is from the test run with default watermarks. it's
> about
> > 180k bzip2-compressed.
> >
> > should i send it to rgerhards at hq.adiscon.com?
> 
> Yes, pls do.
> 
> Rainer
> >
> > regards,
> > -ap
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From a.piesk at gmx.net  Sun Oct 16 20:51:53 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Sun, 16 Oct 2011 20:51:53 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>
Message-ID: <4E9B27C9.1090006@gmx.net>

On 15.10.2011 12:49, Rainer Gerhards wrote:
> Thanks for the debug log. I've so far just have a quick look, but there seem
> to come no more messages in. Can you point me to exactly what happens and
> what is problematic, if possible in sequence so that I can try to find that
> inside the debug log.

OK, i try to explain how to reproduce it. i must say i had a hard time to reproduce it in a virtual
environment but fortunately i succeeded :)

one thing i didn't mention (shame on me) was the fact, that rsyslog on the client also reads some
logfiles with imfile. my first attempt to reproduce it didn't work because rsyslog was setup to read
logfiles with imfile but nothing was written to these files. after adapting the test case by
simultanously generating syslog messages with logger and writing to a monitored logfile i always got
a hanging rsyslog.

here's my test case. first the used client rsyslog configs:

# cat /etc/rsyslog.conf
$ModLoad imuxsock
$ModLoad imklog
$ModLoad immark
$MarkMessagePeriod 1200
$SystemLogRateLimitInterval 0
$ActionFileDefaultTemplate RSYSLOG_FileFormat
*.info;mail.none;authpriv.none;cron.none;local6.none;local0.none	/var/log/messages
authpriv.*								/var/log/secure
mail.*									-/var/log/mail
cron.*									/var/log/cron
*.emerg									*
uucp,news.crit								/var/log/spooler
local7.*								/var/log/boot

# cat /etc/rsyslog.d/forwarder.conf
$ActionForwardDefaultTemplate RSYSLOG_ForwardFormat
$WorkDirectory /var/spool/rsyslog
$ActionQueueType LinkedList
$ActionQueueMaxDiskSpace 1024m
$ActionQueueCheckpointInterval 10
$ActionQueueFileName forward
$ActionQueueMaxFileSize 1m
$ActionResumeRetryCount -1
$ActionQueueSaveOnShutdown on
$ActionWriteAllMarkMessages on
*.*     @@10.10.0.254:514;RSYSLOG_ForwardFormat

# cat /etc/rsyslog.d/pickup.conf
$ModLoad imfile
$InputFilePollInterval 10
$InputFileName /var/log/test.log
$InputFileTag test:
$InputFileStateFile test
$InputFilePersistStateInterval 10
$InputFileSeverity info
$InputFileFacility local6
$InputRunFileMonitor

generate messages on the client:

client # while true; do date; date >> /var/log/test.log; date | logger; sleep 1; done

the messages are logged on the client by rsyslog:

client # tail -f messages test.log
==> messages <==
2011-10-16T19:34:34+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:34:34 CEST 2011

==> test.log <==
Sun Oct 16 19:34:34 CEST 2011

and on the server too:

server # tail -f logger.log test.log
==> logger.log <==
2011-10-16T19:35:12.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:12 CEST 2011
2011-10-16T19:35:13.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:13 CEST 2011
2011-10-16T19:35:14.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:14 CEST 2011
2011-10-16T19:35:15.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:15 CEST 2011
2011-10-16T19:35:16.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:16 CEST 2011
2011-10-16T19:35:17.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:17 CEST 2011
2011-10-16T19:35:18.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:18 CEST 2011
2011-10-16T19:35:19.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:19 CEST 2011
2011-10-16T19:35:20.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:20 CEST 2011
2011-10-16T19:35:21.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:35:21 CEST 2011

==> test.log <==
2011-10-16T19:35:21.100215+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:11 CEST 2011
2011-10-16T19:35:21.100307+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:12 CEST 2011
2011-10-16T19:35:21.100464+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:13 CEST 2011
2011-10-16T19:35:21.100608+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:14 CEST 2011
2011-10-16T19:35:21.100867+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:15 CEST 2011
2011-10-16T19:35:21.101008+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:16 CEST 2011
2011-10-16T19:35:21.101228+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:17 CEST 2011
2011-10-16T19:35:21.101409+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:18 CEST 2011
2011-10-16T19:35:21.101549+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:19 CEST 2011
2011-10-16T19:35:21.101680+02:00 gold-centos5-x86_64 test: Sun Oct 16 19:35:20 CEST 2011

now simulate an unresponsive server:

server # iptables -A INPUT -s gold-centos5-x86_64 -p tcp --dport 514 -j DROP

the logging on the server stops immediately, of course.

the logging on the client continues but not for long:

==> messages <==
2011-10-16T19:41:09+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:41:09 CEST 2011

==> test.log <==
Sun Oct 16 19:41:09 CEST 2011

==> messages <==
2011-10-16T19:41:10+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:41:10 CEST 2011

==> test.log <==
Sun Oct 16 19:41:10 CEST 2011
Sun Oct 16 19:41:11 CEST 2011
Sun Oct 16 19:41:12 CEST 2011
Sun Oct 16 19:41:13 CEST 2011
Sun Oct 16 19:41:14 CEST 2011
Sun Oct 16 19:41:15 CEST 2011
Sun Oct 16 19:41:16 CEST 2011
Sun Oct 16 19:41:17 CEST 2011
Sun Oct 16 19:41:18 CEST 2011
Sun Oct 16 19:41:19 CEST 2011
Sun Oct 16 19:41:20 CEST 2011
Sun Oct 16 19:41:21 CEST 2011
Sun Oct 16 19:41:22 CEST 2011
Sun Oct 16 19:41:23 CEST 2011
Sun Oct 16 19:41:24 CEST 2011
Sun Oct 16 19:41:25 CEST 2011
Sun Oct 16 19:41:26 CEST 2011
Sun Oct 16 19:41:27 CEST 2011
Sun Oct 16 19:41:29 CEST 2011
Sun Oct 16 19:41:30 CEST 2011
Sun Oct 16 19:41:31 CEST 2011

logging to messages stopped. there's no spool file:

client # ls -l /var/spool/rsyslog/
total 8
-rw------- 1 root root 195 Oct 16 19:42 test


to make it easier for you i'll send the logs of such a test run along with descriptions to
rgerhards at hq.adiscon.com.

the logging stops at or around these messages in rsyslog_debug.log:

7644.220848000:44b3c940: strm 0x2acdd9c6a0e0: file 7 read 0 bytes
7644.220858000:44b3c940: stream checking for file change on '/var/log/test.log', inode 2359447/23594
47
7644.220897000:40cc0940: wti 0x2acdd9c532d0: worker awoke from idle processing
7644.220911000:40cc0940: we deleted 0 objects and enqueued 0 objects
7644.220916000:40cc0940: delete batch from store, new sizes: log 10, phys 10
7644.220924000:40cc0940: processBatch: batch of 10 elements must be processed
..
7644.221601000:40cc0940: Processing next action
7644.221606000:40cc0940: Called action(Batch), logging to builtin-fwd
7644.221611000:40cc0940: action 0x2acdd9c4d030: filterOK:1 state:0 execWhenPrev:0 prevWasSusp:0
7644.221626000:40cc0940: action 8 queue: enqueueMsg: FullDelay mark reached for full delayable
message - blocking.


if i run the same test with exactly the same config but _without_ writing to /var/log/test.log
("while true; do date; date | logger; sleep 1; done"), rsyslog creates a spool file and doesn't block.

regards,
-ap

From rgerhards at hq.adiscon.com  Mon Oct 17 11:11:35 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Oct 2011 11:11:35 +0200
Subject: [rsyslog] packaging rsyslog v6
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>

Hi all,

rsyslog v6 will be very important for me because it finally solves the config
language issue. However, I anticipate that most distros will take quite some
while befor packaging it. In order to accelerate adoption, I was thinking of
creating packages for some important -and not so up-to-date- distros, like
RHEL. However, I have never created packages before. I'd appreciate if you
could point me to some resources that help me getting started with that.

Thanks,
Rainer

From rgerhards at hq.adiscon.com  Mon Oct 17 11:44:31 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Oct 2011 11:44:31 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
	remote destination is unresponsive
In-Reply-To: <4E9B27C9.1090006@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>
	<4E9B27C9.1090006@gmx.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>

I am not sure if I understood your scenario correctly. Is this your lab
setup:

- make rsyslog monitor a file f
- continuously append lines to f
- send messages both to remote server and log them locally
- break connection to remote server
- some more lines from f are logged locally, and then no more lines form f
are processed

If so, I think I can explain what happens. The file monitor is flagged as a
full delayable message source. That means if it begins to fill queues, the
input can be blocked without causing harm. This is most useful in (almost)
all setups, because otherwise a large file monitor would copy over
potentially huge amounts of data to a queue file if a remote destination goes
down (in fact, we have seen this in earlier releases). However, this means
that the file monitor is actually blocked, and thus it will also not submit
messages for local submission. I think this is what happens here (it took me
a while to actually understand the scenario...).

I can see that the assumption that a file monitor message can be delayed may
not be valid in some cases, e.g. with extended offline time of a receiver.
However, trust me that it is useful in many more cases. So what could be
done, if I am right with my analysis, is to create a config setting so that
imfile's priority can be changed to non-delayable. This probably solves your
situation, but you should note that this can place considerable burden on
your server in case of failover. Also, you probably need around two to three
times the disk space of those files that are to be enqueued.

Am I on the right path?
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andreas Piesk
> Sent: Sunday, October 16, 2011 8:52 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
hangsif
> remote destination is unresponsive
> 
> On 15.10.2011 12:49, Rainer Gerhards wrote:
> > Thanks for the debug log. I've so far just have a quick look, but there
seem
> > to come no more messages in. Can you point me to exactly what happens
> and
> > what is problematic, if possible in sequence so that I can try to find
that
> > inside the debug log.
> 
> OK, i try to explain how to reproduce it. i must say i had a hard time to
> reproduce it in a virtual
> environment but fortunately i succeeded :)
> 
> one thing i didn't mention (shame on me) was the fact, that rsyslog on the
> client also reads some
> logfiles with imfile. my first attempt to reproduce it didn't work because
> rsyslog was setup to read
> logfiles with imfile but nothing was written to these files. after adapting
the
> test case by
> simultanously generating syslog messages with logger and writing to a
> monitored logfile i always got
> a hanging rsyslog.
> 
> here's my test case. first the used client rsyslog configs:
> 
> # cat /etc/rsyslog.conf
> $ModLoad imuxsock
> $ModLoad imklog
> $ModLoad immark
> $MarkMessagePeriod 1200
> $SystemLogRateLimitInterval 0
> $ActionFileDefaultTemplate RSYSLOG_FileFormat
> *.info;mail.none;authpriv.none;cron.none;local6.none;local0.none
> 	/var/log/messages
> authpriv.*
> 	/var/log/secure
> mail.*
-
> /var/log/mail
> cron.*
> 	/var/log/cron
> *.emerg
> 	*
> uucp,news.crit
> 	/var/log/spooler
> local7.*
> 	/var/log/boot
> 
> # cat /etc/rsyslog.d/forwarder.conf
> $ActionForwardDefaultTemplate RSYSLOG_ForwardFormat
> $WorkDirectory /var/spool/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueMaxDiskSpace 1024m
> $ActionQueueCheckpointInterval 10
> $ActionQueueFileName forward
> $ActionQueueMaxFileSize 1m
> $ActionResumeRetryCount -1
> $ActionQueueSaveOnShutdown on
> $ActionWriteAllMarkMessages on
> *.*     @@10.10.0.254:514;RSYSLOG_ForwardFormat
> 
> # cat /etc/rsyslog.d/pickup.conf
> $ModLoad imfile
> $InputFilePollInterval 10
> $InputFileName /var/log/test.log
> $InputFileTag test:
> $InputFileStateFile test
> $InputFilePersistStateInterval 10
> $InputFileSeverity info
> $InputFileFacility local6
> $InputRunFileMonitor
> 
> generate messages on the client:
> 
> client # while true; do date; date >> /var/log/test.log; date | logger;
sleep 1;
> done
> 
> the messages are logged on the client by rsyslog:
> 
> client # tail -f messages test.log
> ==> messages <==
> 2011-10-16T19:34:34+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:34:34
> CEST 2011
> 
> ==> test.log <==
> Sun Oct 16 19:34:34 CEST 2011
> 
> and on the server too:
> 
> server # tail -f logger.log test.log
> ==> logger.log <==
> 2011-10-16T19:35:12.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:12 CEST 2011
> 2011-10-16T19:35:13.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:13 CEST 2011
> 2011-10-16T19:35:14.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:14 CEST 2011
> 2011-10-16T19:35:15.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:15 CEST 2011
> 2011-10-16T19:35:16.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:16 CEST 2011
> 2011-10-16T19:35:17.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:17 CEST 2011
> 2011-10-16T19:35:18.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:18 CEST 2011
> 2011-10-16T19:35:19.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:19 CEST 2011
> 2011-10-16T19:35:20.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:20 CEST 2011
> 2011-10-16T19:35:21.000000+02:00 gold-centos5-x86_64 logger: Sun Oct 16
> 19:35:21 CEST 2011
> 
> ==> test.log <==
> 2011-10-16T19:35:21.100215+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:11 CEST 2011
> 2011-10-16T19:35:21.100307+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:12 CEST 2011
> 2011-10-16T19:35:21.100464+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:13 CEST 2011
> 2011-10-16T19:35:21.100608+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:14 CEST 2011
> 2011-10-16T19:35:21.100867+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:15 CEST 2011
> 2011-10-16T19:35:21.101008+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:16 CEST 2011
> 2011-10-16T19:35:21.101228+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:17 CEST 2011
> 2011-10-16T19:35:21.101409+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:18 CEST 2011
> 2011-10-16T19:35:21.101549+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:19 CEST 2011
> 2011-10-16T19:35:21.101680+02:00 gold-centos5-x86_64 test: Sun Oct 16
> 19:35:20 CEST 2011
> 
> now simulate an unresponsive server:
> 
> server # iptables -A INPUT -s gold-centos5-x86_64 -p tcp --dport 514 -j
DROP
> 
> the logging on the server stops immediately, of course.
> 
> the logging on the client continues but not for long:
> 
> ==> messages <==
> 2011-10-16T19:41:09+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:41:09
> CEST 2011
> 
> ==> test.log <==
> Sun Oct 16 19:41:09 CEST 2011
> 
> ==> messages <==
> 2011-10-16T19:41:10+02:00 gold-centos5-x86_64 logger: Sun Oct 16 19:41:10
> CEST 2011
> 
> ==> test.log <==
> Sun Oct 16 19:41:10 CEST 2011
> Sun Oct 16 19:41:11 CEST 2011
> Sun Oct 16 19:41:12 CEST 2011
> Sun Oct 16 19:41:13 CEST 2011
> Sun Oct 16 19:41:14 CEST 2011
> Sun Oct 16 19:41:15 CEST 2011
> Sun Oct 16 19:41:16 CEST 2011
> Sun Oct 16 19:41:17 CEST 2011
> Sun Oct 16 19:41:18 CEST 2011
> Sun Oct 16 19:41:19 CEST 2011
> Sun Oct 16 19:41:20 CEST 2011
> Sun Oct 16 19:41:21 CEST 2011
> Sun Oct 16 19:41:22 CEST 2011
> Sun Oct 16 19:41:23 CEST 2011
> Sun Oct 16 19:41:24 CEST 2011
> Sun Oct 16 19:41:25 CEST 2011
> Sun Oct 16 19:41:26 CEST 2011
> Sun Oct 16 19:41:27 CEST 2011
> Sun Oct 16 19:41:29 CEST 2011
> Sun Oct 16 19:41:30 CEST 2011
> Sun Oct 16 19:41:31 CEST 2011
> 
> logging to messages stopped. there's no spool file:
> 
> client # ls -l /var/spool/rsyslog/
> total 8
> -rw------- 1 root root 195 Oct 16 19:42 test
> 
> 
> to make it easier for you i'll send the logs of such a test run along with
> descriptions to
> rgerhards at hq.adiscon.com.
> 
> the logging stops at or around these messages in rsyslog_debug.log:
> 
> 7644.220848000:44b3c940: strm 0x2acdd9c6a0e0: file 7 read 0 bytes
> 7644.220858000:44b3c940: stream checking for file change on
> '/var/log/test.log', inode 2359447/23594
> 47
> 7644.220897000:40cc0940: wti 0x2acdd9c532d0: worker awoke from idle
> processing
> 7644.220911000:40cc0940: we deleted 0 objects and enqueued 0 objects
> 7644.220916000:40cc0940: delete batch from store, new sizes: log 10, phys
10
> 7644.220924000:40cc0940: processBatch: batch of 10 elements must be
> processed
> ..
> 7644.221601000:40cc0940: Processing next action
> 7644.221606000:40cc0940: Called action(Batch), logging to builtin-fwd
> 7644.221611000:40cc0940: action 0x2acdd9c4d030: filterOK:1 state:0
> execWhenPrev:0 prevWasSusp:0
> 7644.221626000:40cc0940: action 8 queue: enqueueMsg: FullDelay mark
> reached for full delayable
> message - blocking.
> 
> 
> if i run the same test with exactly the same config but _without_ writing
to
> /var/log/test.log
> ("while true; do date; date | logger; sleep 1; done"), rsyslog creates a
spool
> file and doesn't block.
> 
> regards,
> -ap
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mbiebl at gmail.com  Mon Oct 17 16:34:01 2011
From: mbiebl at gmail.com (Michael Biebl)
Date: Mon, 17 Oct 2011 16:34:01 +0200
Subject: [rsyslog] packaging rsyslog v6
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
Message-ID: <CAGWsdOg3h7mNYBH8dcEQLxFc4_nV9vWLCyn2otHwiqkNw9CoMg@mail.gmail.com>

2011/10/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> rsyslog v6 will be very important for me because it finally solves the config
> language issue. However, I anticipate that most distros will take quite some
> while befor packaging it. In order to accelerate adoption, I was thinking of
> creating packages for some important -and not so up-to-date- distros, like
> RHEL. However, I have never created packages before. I'd appreciate if you
> could point me to some resources that help me getting started with that.

I haven't decided yet if the next Debian stable release will be using v5 or v6.

I should probaby start with uploading v6 packages to the experimental
archive and ask for wider testing...

Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Mon Oct 17 16:35:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Oct 2011 16:35:37 +0200
Subject: [rsyslog] packaging rsyslog v6
In-Reply-To: <CAGWsdOg3h7mNYBH8dcEQLxFc4_nV9vWLCyn2otHwiqkNw9CoMg@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
	<CAGWsdOg3h7mNYBH8dcEQLxFc4_nV9vWLCyn2otHwiqkNw9CoMg@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281343@GRFEXC.intern.adiscon.com>



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Monday, October 17, 2011 4:34 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] packaging rsyslog v6
> 
> 2011/10/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Hi all,
> >
> > rsyslog v6 will be very important for me because it finally solves the
> > config language issue. However, I anticipate that most distros will
> > take quite some while befor packaging it. In order to accelerate
> > adoption, I was thinking of creating packages for some important -and
> > not so up-to-date- distros, like RHEL. However, I have never created
> > packages before. I'd appreciate if you could point me to some resources
> that help me getting started with that.
> 
> I haven't decided yet if the next Debian stable release will be using v5 or
v6.
> 
> I should probaby start with uploading v6 packages to the experimental
> archive and ask for wider testing... 

Sounds great, as usual excellent support. Just wait until I tell you it is
worth the effort. We are between 2..4 weeks away from a version that
justifies it :)

Rainer
> 
> Michael
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
universe are
> pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mbiebl at gmail.com  Mon Oct 17 16:43:16 2011
From: mbiebl at gmail.com (Michael Biebl)
Date: Mon, 17 Oct 2011 16:43:16 +0200
Subject: [rsyslog] packaging rsyslog v6
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281343@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
	<CAGWsdOg3h7mNYBH8dcEQLxFc4_nV9vWLCyn2otHwiqkNw9CoMg@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281343@GRFEXC.intern.adiscon.com>
Message-ID: <CAGWsdOhu9YJwMMsfEdihVAT=JPYi+ezC1MUgMW+pSHYdmZv1dw@mail.gmail.com>

2011/10/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Sounds great, as usual excellent support. Just wait until I tell you it is
> worth the effort. We are between 2..4 weeks away from a version that
> justifies it :)

I've filed http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=645640
so I don't forget about this issue.

Feel free to CC 645640 at bugs.debian.org when v6 is ready (you don't
need to subscribe or anything).

Cheers,
Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Mon Oct 17 16:44:15 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Oct 2011 16:44:15 +0200
Subject: [rsyslog] packaging rsyslog v6
In-Reply-To: <CAGWsdOhu9YJwMMsfEdihVAT=JPYi+ezC1MUgMW+pSHYdmZv1dw@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com><CAGWsdOg3h7mNYBH8dcEQLxFc4_nV9vWLCyn2otHwiqkNw9CoMg@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA7281343@GRFEXC.intern.adiscon.com>
	<CAGWsdOhu9YJwMMsfEdihVAT=JPYi+ezC1MUgMW+pSHYdmZv1dw@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281344@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Monday, October 17, 2011 4:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] packaging rsyslog v6
> 
> 2011/10/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Sounds great, as usual excellent support. Just wait until I tell you
> > it is worth the effort. We are between 2..4 weeks away from a version
> > that justifies it :)
> 
> I've filed http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=645640
> so I don't forget about this issue.
> 
> Feel free to CC 645640 at bugs.debian.org when v6 is ready (you don't need to
> subscribe or anything).

Excellent, will (hopefully ;)) do!

Rainer
> 
> Cheers,
> Michael
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
universe are
> pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From a.piesk at gmx.net  Mon Oct 17 20:00:43 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Mon, 17 Oct 2011 20:00:43 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>
Message-ID: <4E9C6D4B.4000602@gmx.net>

On 17.10.2011 11:44, Rainer Gerhards wrote:
> I am not sure if I understood your scenario correctly. Is this your lab
> setup:
> 
> - make rsyslog monitor a file f
> - continuously append lines to f
> - send messages both to remote server and log them locally
> - break connection to remote server
> - some more lines from f are logged locally, and then no more lines form f
> are processed

almost, you forgot the messages sent by logger via syslog() :)

- make rsyslog receive messages via syslog()
- make rsyslog monitor a file f
- continuously send messages via syslog() and append lines to f
- log messages locally and send all messages to remote server
- break connection to remote server
- after some time rsyslog stops logging any messages, from syslog() and f
- after some more time all applications logging via syslog() almost hang ( a simple login takes ages)

> If so, I think I can explain what happens. The file monitor is flagged as a
> full delayable message source. That means if it begins to fill queues, the
> input can be blocked without causing harm. This is most useful in (almost)
> all setups, because otherwise a large file monitor would copy over
> potentially huge amounts of data to a queue file if a remote destination goes
> down (in fact, we have seen this in earlier releases). However, this means
> that the file monitor is actually blocked, and thus it will also not submit
> messages for local submission. I think this is what happens here (it took me
> a while to actually understand the scenario...).

i wouldn't care if only the file monitor blocks but the real problem is the hanging syslog() call.
why does a blocked file monitor also blocks the processing of messages sent by syslog()?
i expected that at least messages sent via syslog() are processed but rsyslog stops to log anything
and thus these messages are lost because they are not spooled to disk.

> I can see that the assumption that a file monitor message can be delayed may
> not be valid in some cases, e.g. with extended offline time of a receiver.
> However, trust me that it is useful in many more cases. So what could be
> done, if I am right with my analysis, is to create a config setting so that
> imfile's priority can be changed to non-delayable. This probably solves your
> situation, but you should note that this can place considerable burden on
> your server in case of failover. Also, you probably need around two to three
> times the disk space of those files that are to be enqueued.

i think it's reasonable to let the file monitor block but this shouldn't affect other input sources,
right?
if your proposed solution fixes the hanging syslog() calls too, i'm willing to try and test it :)

regards,
-ap

From david at lang.hm  Mon Oct 17 21:01:23 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 17 Oct 2011 12:01:23 -0700 (PDT)
Subject: [rsyslog] packaging rsyslog v6
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1110171159330.28233@asgard.lang.hm>

checkinstall does a great job of creating a generic package (for .deb, 
.rpm. and slackware .tgz)

now you will either need to enter the dependancy, etc information, or 
tweak the package a bit after it's created, but I find this _much_ easier 
than doing everything from scratch

David Lang


On Mon, 17 Oct 2011, Rainer Gerhards wrote:

> Date: Mon, 17 Oct 2011 11:11:35 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] packaging rsyslog v6
> 
> Hi all,
>
> rsyslog v6 will be very important for me because it finally solves the config
> language issue. However, I anticipate that most distros will take quite some
> while befor packaging it. In order to accelerate adoption, I was thinking of
> creating packages for some important -and not so up-to-date- distros, like
> RHEL. However, I have never created packages before. I'd appreciate if you
> could point me to some resources that help me getting started with that.
>
> Thanks,
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From a.piesk at gmx.net  Mon Oct 17 21:36:37 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Mon, 17 Oct 2011 21:36:37 +0200
Subject: [rsyslog] packaging rsyslog v6
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA728133C@GRFEXC.intern.adiscon.com>
Message-ID: <4E9C83C5.3030306@gmx.net>

On 17.10.2011 11:11, Rainer Gerhards wrote:
> 
> rsyslog v6 will be very important for me because it finally solves the config
> language issue. However, I anticipate that most distros will take quite some
> while befor packaging it. In order to accelerate adoption, I was thinking of
> creating packages for some important -and not so up-to-date- distros, like
> RHEL. However, I have never created packages before. I'd appreciate if you
> could point me to some resources that help me getting started with that.

as i create rsyslog v5 RHEL packages for myself i could create one for v6 too so you don't have to
start from scratch. my rpms usually build rsyslog with tls only but that can be changed :)

regards,
-ap

From rgerhards at hq.adiscon.com  Tue Oct 18 08:39:46 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Oct 2011 08:39:46 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
	remote destination is unresponsive
In-Reply-To: <4E9C6D4B.4000602@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>
	<4E9C6D4B.4000602@gmx.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andreas Piesk
> Sent: Monday, October 17, 2011 8:01 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
> hangsif remote destination is unresponsive
> 
> On 17.10.2011 11:44, Rainer Gerhards wrote:
> > I am not sure if I understood your scenario correctly. Is this your
> lab
> > setup:
> >
> > - make rsyslog monitor a file f
> > - continuously append lines to f
> > - send messages both to remote server and log them locally
> > - break connection to remote server
> > - some more lines from f are logged locally, and then no more lines
> form f
> > are processed
> 
> almost, you forgot the messages sent by logger via syslog() :)

Indeed and that is another story. Of course, that should not be blocked.
Trying to dig deeper into the issue: what happens if you do not monitor the
file but just use syslog() input. Does it then block?

Rainer
> 
> - make rsyslog receive messages via syslog()
> - make rsyslog monitor a file f
> - continuously send messages via syslog() and append lines to f
> - log messages locally and send all messages to remote server
> - break connection to remote server
> - after some time rsyslog stops logging any messages, from syslog() and
> f
> - after some more time all applications logging via syslog() almost
> hang ( a simple login takes ages)
> 
> > If so, I think I can explain what happens. The file monitor is
> flagged as a
> > full delayable message source. That means if it begins to fill
> queues, the
> > input can be blocked without causing harm. This is most useful in
> (almost)
> > all setups, because otherwise a large file monitor would copy over
> > potentially huge amounts of data to a queue file if a remote
> destination goes
> > down (in fact, we have seen this in earlier releases). However, this
> means
> > that the file monitor is actually blocked, and thus it will also not
> submit
> > messages for local submission. I think this is what happens here (it
> took me
> > a while to actually understand the scenario...).
> 
> i wouldn't care if only the file monitor blocks but the real problem is
> the hanging syslog() call.
> why does a blocked file monitor also blocks the processing of messages
> sent by syslog()?
> i expected that at least messages sent via syslog() are processed but
> rsyslog stops to log anything
> and thus these messages are lost because they are not spooled to disk.
> 
> > I can see that the assumption that a file monitor message can be
> delayed may
> > not be valid in some cases, e.g. with extended offline time of a
> receiver.
> > However, trust me that it is useful in many more cases. So what could
> be
> > done, if I am right with my analysis, is to create a config setting
> so that
> > imfile's priority can be changed to non-delayable. This probably
> solves your
> > situation, but you should note that this can place considerable
> burden on
> > your server in case of failover. Also, you probably need around two
> to three
> > times the disk space of those files that are to be enqueued.
> 
> i think it's reasonable to let the file monitor block but this
> shouldn't affect other input sources,
> right?
> if your proposed solution fixes the hanging syslog() calls too, i'm
> willing to try and test it :)
> 
> regards,
> -ap
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From a.piesk at gmx.net  Tue Oct 18 20:06:15 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Tue, 18 Oct 2011 20:06:15 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>	<4E9C6D4B.4000602@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>
Message-ID: <4E9DC017.7070100@gmx.net>

On 18.10.2011 08:39, Rainer Gerhards wrote:
> 
> Indeed and that is another story. Of course, that should not be blocked.
> Trying to dig deeper into the issue: what happens if you do not monitor the
> file but just use syslog() input. Does it then block?
> 

no, without any file monitors it does not block.
even if i have a file monitor configured but do not write into the monitored file is does not block.

regards,
-ap

From gkra at unnerving.org  Wed Oct 19 03:21:26 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Tue, 18 Oct 2011 18:21:26 -0700
Subject: [rsyslog] pri-text property incorrectly appending pri?
Message-ID: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org>

I've been doing some testing and debugging on my rules in rsyslog (5.9.5, built from sources on CentOS 5.6), and I've discovered that the property "pri-text" doesn't just give you the textual form.

using:

-----
$template testfmt, "%pri-text%\n"
*.* /var/log/testing;testfmt
-----

I end up with, for example:

-----
local0.notice<133>
-----

in my logfile.


On the bright side, I now know why my rules aren't working, like:

-----
:pri-text, !isequal, "local0.err" ~
*.* /var/log/local0.err.log
-----

Since there is always the contents of "%pri%" tacked on the end, nothing will ever be equal to "local0.err" and my log file stays empty. I can work around this for now by using startswith instead of isequal, but the behavior still bugs me.

Is this a bug, or intended behavior?

Thanks,

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu




From wma443 at herr-der-mails.de  Wed Oct 19 15:43:32 2011
From: wma443 at herr-der-mails.de (wma443 at herr-der-mails.de)
Date: Wed, 19 Oct 2011 15:43:32 +0200
Subject: [rsyslog] minimum gcc version required for compilation of rsyslog
	4.8.0/5.8.5
Message-ID: <20111019134332.201190@gmx.net>

Dear list,

Does anyone have experience in compiling one of the current rsyslog versions (4.8.0 or 5.8.5) with a version of gcc that is 4.1.0 or older?

Kind regards,

Carl

From rgerhards at hq.adiscon.com  Thu Oct 20 11:58:32 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 11:58:32 +0200
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Gregory K. Ruiz-Ade
> Sent: Wednesday, October 19, 2011 3:21 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] pri-text property incorrectly appending pri?
> 
> I've been doing some testing and debugging on my rules in rsyslog
> (5.9.5, built from sources on CentOS 5.6), and I've discovered that the
> property "pri-text" doesn't just give you the textual form.
> 
> using:
> 
> -----
> $template testfmt, "%pri-text%\n"
> *.* /var/log/testing;testfmt
> -----
> 
> I end up with, for example:
> 
> -----
> local0.notice<133>
> -----
> 
> in my logfile.
> 
> 
> On the bright side, I now know why my rules aren't working, like:
> 
> -----
> :pri-text, !isequal, "local0.err" ~
> *.* /var/log/local0.err.log
> -----
> 
> Since there is always the contents of "%pri%" tacked on the end,
> nothing will ever be equal to "local0.err" and my log file stays empty.
> I can work around this for now by using startswith instead of isequal,
> but the behavior still bugs me.
> 
> Is this a bug, or intended behavior?

That's a really ugly issue. I have checked to code, and it was this way all
the time. It's intentional. While I don't remember introducing this property
at all, I can see it is intentional by the way the property is formatted
("%s<%d>" basically). The doc does not have the number inside the spec, but
clearly shows it in examples.

Usually, I'd say I remove the numerical PRI, but I don't know who may be
relying on it. Tough call. Even more ugly is introducing something like
pri-text-alternate, but maybe that's the way to go.

Anybody with suggestions?


As a side-note, using

local0.err /var/log/local0.err.log

as a rule is much more efficient than what you wrote above.

Rainer 

From rgerhards at hq.adiscon.com  Thu Oct 20 12:10:06 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 12:10:06 +0200
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org>
	<9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728136F@GRFEXC.intern.adiscon.com>

> Usually, I'd say I remove the numerical PRI, but I don't know who may
> be
> relying on it. Tough call. Even more ugly is introducing something like
> pri-text-alternate, but maybe that's the way to go.
> 
> Anybody with suggestions?

For now I have corrected the doc so that it consistently tells the intended
behaviour (with numerical PRI appended).

Rainer

From horvath.peter77 at freemail.hu  Thu Oct 20 13:48:25 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Thu, 20 Oct 2011 12:48:25 +0100
Subject: [rsyslog] Dynamic file names
In-Reply-To: <20111013104734.GA17060@llserver.lakeliving.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>
	<20111013104734.GA17060@llserver.lakeliving.com>
Message-ID: <CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com>

I continued to extend my config after i managed to solve this issue.

Logs are comming in from localhost and remote host on TCP 514.

Apaches send their logs to the syslog with the following config:

ErrorLog "|/usr/bin/logger -p local6.warn -t httpd_error_vhostname"
CustomLog "|/usr/bin/logger -p local6.info -t httpd_access_vhostname" combined

Added the following lines to rsyslog conf
$template ApacheLogFormat,"%msg:2:10000%\n"
$template local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONTH%%$DAY%"
$template local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MONTH%%$DAY%"

if $syslogfacility-text == 'local6' and $programname startswith
'httpd_error' then -?local6error;ApacheLogFormat
#& ~
if $syslogfacility-text == 'local6' and $programname startswith
'httpd_access' then -?local6access;ApacheLogFormat
#& ~

I getting this error message in syslog:
rsyslogd: Could not open dynamic file
'/var/log/vhostname_access_log.20111020' - discarding message
rsyslogd: Could not open dynamic file
'/var/log/vhostname_error_log.20111020' - discarding message

I've already given /var/log to syslog user and the files are created
perfectly however it cannot write them for some reason.
I tried to open files in a different location and also same effect,
files are created but rsyslog tells me could not open.
Files are created with this mask.
-rw-r-----  1 syslog    syslog      0 2011-10-20 11:34
vhostname_access_log.20111020
-rw-r-----  1 syslog    syslog      0 2011-10-20 11:34
vhostname_error_log.20111020

Am i missing something?

Sorry if i am missing something obvious.


On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
>> I would like to get opinions about this:
>>
>> I have the following line in my rsyslog conf:
>> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> *.*;auth,authpriv.none ??DynFile
>>
>> And it is not working.
> At a glance it looks ok. Try invoking rsyslog with -N1 to see if it
> complains about your configuration.
>
>> After hours of different tries realized if i remove ;auth,authpriv.none
>> It starts to work magically.
>> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> *.* ??DynFile
>>
>> However i had to touch the files manually because these error messages
>> appeared in the log:
>> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' - discarding message
> The dynamic files aren't created when rsyslog starts, so it needs
> permission to write them after it drops permissions (the default
> configuration in Ubuntu). If you try to write the file to /var/log
> (which you are) you will get this error because /var/log is owned by
> root and syslog cannot write new files there. At our site we work around
> this by creating a new folder owned by syslog.
>
>> It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0
> The important lines to note are these:
> $PrivDropToUser syslog
> $PrivDropToGroup syslog
>
> Which are why the file can't be created dynamically in /var/log.
>
>> Do you have any idea what the problem with my original try and why
>> rsyslog cannot open logfiles?
>
> -Ryan Kelly
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Thu Oct 20 13:53:02 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 13:53:02 +0200
Subject: [rsyslog] Dynamic file names
In-Reply-To: <CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com><20111013104734.GA17060@llserver.lakeliving.com>
	<CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> Sent: Thursday, October 20, 2011 1:48 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Dynamic file names
> 
> I continued to extend my config after i managed to solve this issue.
> 
> Logs are comming in from localhost and remote host on TCP 514.
> 
> Apaches send their logs to the syslog with the following config:
> 
> ErrorLog "|/usr/bin/logger -p local6.warn -t httpd_error_vhostname"
> CustomLog "|/usr/bin/logger -p local6.info -t httpd_access_vhostname"
> combined
> 
> Added the following lines to rsyslog conf $template
> ApacheLogFormat,"%msg:2:10000%\n"
> $template
> local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONT
> H%%$DAY%"
> $template
> local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MO
> NTH%%$DAY%"
> 
> if $syslogfacility-text == 'local6' and $programname startswith
'httpd_error'
> then -?local6error;ApacheLogFormat #& ~ if $syslogfacility-text == 'local6'
> and $programname startswith 'httpd_access' then -
> ?local6access;ApacheLogFormat #& ~
> 
> I getting this error message in syslog:
> rsyslogd: Could not open dynamic file
> '/var/log/vhostname_access_log.20111020' - discarding message
> rsyslogd: Could not open dynamic file
> '/var/log/vhostname_error_log.20111020' - discarding message
> 
> I've already given /var/log to syslog user and the files are created
perfectly
> however it cannot write them for some reason.
> I tried to open files in a different location and also same effect, files
are
> created but rsyslog tells me could not open.
> Files are created with this mask.
> -rw-r-----  1 syslog    syslog      0 2011-10-20 11:34
> vhostname_access_log.20111020
> -rw-r-----  1 syslog    syslog      0 2011-10-20 11:34
> vhostname_error_log.20111020
> 
> Am i missing something?
> 
> Sorry if i am missing something obvious.

I suggest to remove

$PrivDropToUser syslog
$PrivDropToGroup syslog

>From your config and retry. When it then works, we know for sure it is
related to the permissions.

Rainer
> 
> 
> On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
> >> I would like to get opinions about this:
> >>
> >> I have the following line in my rsyslog conf:
> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> >> *.*;auth,authpriv.none ??DynFile
> >>
> >> And it is not working.
> > At a glance it looks ok. Try invoking rsyslog with -N1 to see if it
> > complains about your configuration.
> >
> >> After hours of different tries realized if i remove
> >> ;auth,authpriv.none It starts to work magically.
> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> >> *.* ??DynFile
> >>
> >> However i had to touch the files manually because these error
> >> messages appeared in the log:
> >> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' -
> >> discarding message
> > The dynamic files aren't created when rsyslog starts, so it needs
> > permission to write them after it drops permissions (the default
> > configuration in Ubuntu). If you try to write the file to /var/log
> > (which you are) you will get this error because /var/log is owned by
> > root and syslog cannot write new files there. At our site we work
> > around this by creating a new folder owned by syslog.
> >
> >> It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0
> > The important lines to note are these:
> > $PrivDropToUser syslog
> > $PrivDropToGroup syslog
> >
> > Which are why the file can't be created dynamically in /var/log.
> >
> >> Do you have any idea what the problem with my original try and why
> >> rsyslog cannot open logfiles?
> >
> > -Ryan Kelly
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From horvath.peter77 at freemail.hu  Thu Oct 20 14:17:48 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Thu, 20 Oct 2011 13:17:48 +0100
Subject: [rsyslog] Dynamic file names
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>
	<20111013104734.GA17060@llserver.lakeliving.com>
	<CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com>
Message-ID: <CAO9xhuav7Y875cu05gjd5hT5r6Ty_7MH_r5_b5nt187oEXTzpg@mail.gmail.com>

Thank you, that makes it working however:

This is the default settings
$FileOwner syslog
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
$Umask 0022
$PrivDropToUser syslog
$PrivDropToGroup syslog

If I comment out the privileges drop it is working
But if I just change the already created files from
-rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
vhostname_access_log.20111020
-rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
vhostname_error_log.20111020

to

-rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
vhostname_access_log.20111020
-rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34 vhostname_error_log.20111020

insted commenting out, it is also starts working.


On 20 October 2011 12:53, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> Sent: Thursday, October 20, 2011 1:48 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Dynamic file names
>>
>> I continued to extend my config after i managed to solve this issue.
>>
>> Logs are comming in from localhost and remote host on TCP 514.
>>
>> Apaches send their logs to the syslog with the following config:
>>
>> ErrorLog "|/usr/bin/logger -p local6.warn -t httpd_error_vhostname"
>> CustomLog "|/usr/bin/logger -p local6.info -t httpd_access_vhostname"
>> combined
>>
>> Added the following lines to rsyslog conf $template
>> ApacheLogFormat,"%msg:2:10000%\n"
>> $template
>> local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONT
>> H%%$DAY%"
>> $template
>> local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MO
>> NTH%%$DAY%"
>>
>> if $syslogfacility-text == 'local6' and $programname startswith
> 'httpd_error'
>> then -?local6error;ApacheLogFormat #& ~ if $syslogfacility-text == 'local6'
>> and $programname startswith 'httpd_access' then -
>> ?local6access;ApacheLogFormat #& ~
>>
>> I getting this error message in syslog:
>> rsyslogd: Could not open dynamic file
>> '/var/log/vhostname_access_log.20111020' - discarding message
>> rsyslogd: Could not open dynamic file
>> '/var/log/vhostname_error_log.20111020' - discarding message
>>
>> I've already given /var/log to syslog user and the files are created
> perfectly
>> however it cannot write them for some reason.
>> I tried to open files in a different location and also same effect, files
> are
>> created but rsyslog tells me could not open.
>> Files are created with this mask.
>> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> vhostname_access_log.20111020
>> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> vhostname_error_log.20111020
>>
>> Am i missing something?
>>
>> Sorry if i am missing something obvious.
>
> I suggest to remove
>
> $PrivDropToUser syslog
> $PrivDropToGroup syslog
>
> From your config and retry. When it then works, we know for sure it is
> related to the permissions.
>
> Rainer
>>
>>
>> On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
>> >> I would like to get opinions about this:
>> >>
>> >> I have the following line in my rsyslog conf:
>> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> >> *.*;auth,authpriv.none ??DynFile
>> >>
>> >> And it is not working.
>> > At a glance it looks ok. Try invoking rsyslog with -N1 to see if it
>> > complains about your configuration.
>> >
>> >> After hours of different tries realized if i remove
>> >> ;auth,authpriv.none It starts to work magically.
>> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> >> *.* ??DynFile
>> >>
>> >> However i had to touch the files manually because these error
>> >> messages appeared in the log:
>> >> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' -
>> >> discarding message
>> > The dynamic files aren't created when rsyslog starts, so it needs
>> > permission to write them after it drops permissions (the default
>> > configuration in Ubuntu). If you try to write the file to /var/log
>> > (which you are) you will get this error because /var/log is owned by
>> > root and syslog cannot write new files there. At our site we work
>> > around this by creating a new folder owned by syslog.
>> >
>> >> It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0
>> > The important lines to note are these:
>> > $PrivDropToUser syslog
>> > $PrivDropToGroup syslog
>> >
>> > Which are why the file can't be created dynamically in /var/log.
>> >
>> >> Do you have any idea what the problem with my original try and why
>> >> rsyslog cannot open logfiles?
>> >
>> > -Ryan Kelly
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Thu Oct 20 14:37:29 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 14:37:29 +0200
Subject: [rsyslog] Dynamic file names
In-Reply-To: <CAO9xhuav7Y875cu05gjd5hT5r6Ty_7MH_r5_b5nt187oEXTzpg@mail.gmail.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com><20111013104734.GA17060@llserver.lakeliving.com><CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com>
	<CAO9xhuav7Y875cu05gjd5hT5r6Ty_7MH_r5_b5nt187oEXTzpg@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281374@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> Sent: Thursday, October 20, 2011 2:18 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Dynamic file names
> 
> Thank you, that makes it working however:
> 
> This is the default settings
> $FileOwner syslog
> $FileGroup adm
> $FileCreateMode 0640
> $DirCreateMode 0755
> $Umask 0022
> $PrivDropToUser syslog
> $PrivDropToGroup syslog
> 
> If I comment out the privileges drop it is working But if I just change the
> already created files from
> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> vhostname_access_log.20111020
> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> vhostname_error_log.20111020
> 
> to
> 
> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
> vhostname_access_log.20111020
> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
> vhostname_error_log.20111020
> 
> insted commenting out, it is also starts working.

Mhhh, this doesn't make much sense to me. The user should be able to open
files for writing if it has permissions... and it looks like it has. Can you
check if rsyslog actually runs under the syslog user? 

Rainer


> 
> 
> On 20 October 2011 12:53, Rainer Gerhards <rgerhards at hq.adiscon.com>
> wrote:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> >> Sent: Thursday, October 20, 2011 1:48 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] Dynamic file names
> >>
> >> I continued to extend my config after i managed to solve this issue.
> >>
> >> Logs are comming in from localhost and remote host on TCP 514.
> >>
> >> Apaches send their logs to the syslog with the following config:
> >>
> >> ErrorLog "|/usr/bin/logger -p local6.warn -t httpd_error_vhostname"
> >> CustomLog "|/usr/bin/logger -p local6.info -t httpd_access_vhostname"
> >> combined
> >>
> >> Added the following lines to rsyslog conf $template
> >> ApacheLogFormat,"%msg:2:10000%\n"
> >> $template
> >>
> local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONT
> >> H%%$DAY%"
> >> $template
> >>
> local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MO
> >> NTH%%$DAY%"
> >>
> >> if $syslogfacility-text == 'local6' and $programname startswith
> > 'httpd_error'
> >> then -?local6error;ApacheLogFormat #& ~ if $syslogfacility-text ==
'local6'
> >> and $programname startswith 'httpd_access' then -
> >> ?local6access;ApacheLogFormat #& ~
> >>
> >> I getting this error message in syslog:
> >> rsyslogd: Could not open dynamic file
> >> '/var/log/vhostname_access_log.20111020' - discarding message
> >> rsyslogd: Could not open dynamic file
> >> '/var/log/vhostname_error_log.20111020' - discarding message
> >>
> >> I've already given /var/log to syslog user and the files are created
> > perfectly
> >> however it cannot write them for some reason.
> >> I tried to open files in a different location and also same effect,
> >> files
> > are
> >> created but rsyslog tells me could not open.
> >> Files are created with this mask.
> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> >> vhostname_access_log.20111020
> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> >> vhostname_error_log.20111020
> >>
> >> Am i missing something?
> >>
> >> Sorry if i am missing something obvious.
> >
> > I suggest to remove
> >
> > $PrivDropToUser syslog
> > $PrivDropToGroup syslog
> >
> > From your config and retry. When it then works, we know for sure it is
> > related to the permissions.
> >
> > Rainer
> >>
> >>
> >> On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
> >> >> I would like to get opinions about this:
> >> >>
> >> >> I have the following line in my rsyslog conf:
> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> >> >> *.*;auth,authpriv.none ??DynFile
> >> >>
> >> >> And it is not working.
> >> > At a glance it looks ok. Try invoking rsyslog with -N1 to see if it
> >> > complains about your configuration.
> >> >
> >> >> After hours of different tries realized if i remove
> >> >> ;auth,authpriv.none It starts to work magically.
> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> >> >> *.* ??DynFile
> >> >>
> >> >> However i had to touch the files manually because these error
> >> >> messages appeared in the log:
> >> >> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' -
> >> >> discarding message
> >> > The dynamic files aren't created when rsyslog starts, so it needs
> >> > permission to write them after it drops permissions (the default
> >> > configuration in Ubuntu). If you try to write the file to /var/log
> >> > (which you are) you will get this error because /var/log is owned
> >> > by root and syslog cannot write new files there. At our site we
> >> > work around this by creating a new folder owned by syslog.
> >> >
> >> >> It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0
> >> > The important lines to note are these:
> >> > $PrivDropToUser syslog
> >> > $PrivDropToGroup syslog
> >> >
> >> > Which are why the file can't be created dynamically in /var/log.
> >> >
> >> >> Do you have any idea what the problem with my original try and why
> >> >> rsyslog cannot open logfiles?
> >> >
> >> > -Ryan Kelly
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> >
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From horvath.peter77 at freemail.hu  Thu Oct 20 15:13:49 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Thu, 20 Oct 2011 14:13:49 +0100
Subject: [rsyslog] Dynamic file names
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281374@GRFEXC.intern.adiscon.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>
	<20111013104734.GA17060@llserver.lakeliving.com>
	<CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com>
	<CAO9xhuav7Y875cu05gjd5hT5r6Ty_7MH_r5_b5nt187oEXTzpg@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281374@GRFEXC.intern.adiscon.com>
Message-ID: <CAO9xhua80Yg1OEPBA_Znhh1P4aHKAPL9S=+zHRDzj5JsR3Dr9A@mail.gmail.com>

root      2722     1  0 12:19 ?        00:00:00 rsyslogd -c4
ubuntu 10.04 LTS default settings everywhere

On 20 October 2011 13:37, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> Sent: Thursday, October 20, 2011 2:18 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Dynamic file names
>>
>> Thank you, that makes it working however:
>>
>> This is the default settings
>> $FileOwner syslog
>> $FileGroup adm
>> $FileCreateMode 0640
>> $DirCreateMode 0755
>> $Umask 0022
>> $PrivDropToUser syslog
>> $PrivDropToGroup syslog
>>
>> If I comment out the privileges drop it is working But if I just change the
>> already created files from
>> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> vhostname_access_log.20111020
>> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> vhostname_error_log.20111020
>>
>> to
>>
>> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
>> vhostname_access_log.20111020
>> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
>> vhostname_error_log.20111020
>>
>> insted commenting out, it is also starts working.
>
> Mhhh, this doesn't make much sense to me. The user should be able to open
> files for writing if it has permissions... and it looks like it has. Can you
> check if rsyslog actually runs under the syslog user?
>
> Rainer
>
>
>>
>>
>> On 20 October 2011 12:53, Rainer Gerhards <rgerhards at hq.adiscon.com>
>> wrote:
>> >> -----Original Message-----
>> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> >> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> >> Sent: Thursday, October 20, 2011 1:48 PM
>> >> To: rsyslog-users
>> >> Subject: Re: [rsyslog] Dynamic file names
>> >>
>> >> I continued to extend my config after i managed to solve this issue.
>> >>
>> >> Logs are comming in from localhost and remote host on TCP 514.
>> >>
>> >> Apaches send their logs to the syslog with the following config:
>> >>
>> >> ErrorLog "|/usr/bin/logger -p local6.warn -t httpd_error_vhostname"
>> >> CustomLog "|/usr/bin/logger -p local6.info -t httpd_access_vhostname"
>> >> combined
>> >>
>> >> Added the following lines to rsyslog conf $template
>> >> ApacheLogFormat,"%msg:2:10000%\n"
>> >> $template
>> >>
>> local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONT
>> >> H%%$DAY%"
>> >> $template
>> >>
>> local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MO
>> >> NTH%%$DAY%"
>> >>
>> >> if $syslogfacility-text == 'local6' and $programname startswith
>> > 'httpd_error'
>> >> then -?local6error;ApacheLogFormat #& ~ if $syslogfacility-text ==
> 'local6'
>> >> and $programname startswith 'httpd_access' then -
>> >> ?local6access;ApacheLogFormat #& ~
>> >>
>> >> I getting this error message in syslog:
>> >> rsyslogd: Could not open dynamic file
>> >> '/var/log/vhostname_access_log.20111020' - discarding message
>> >> rsyslogd: Could not open dynamic file
>> >> '/var/log/vhostname_error_log.20111020' - discarding message
>> >>
>> >> I've already given /var/log to syslog user and the files are created
>> > perfectly
>> >> however it cannot write them for some reason.
>> >> I tried to open files in a different location and also same effect,
>> >> files
>> > are
>> >> created but rsyslog tells me could not open.
>> >> Files are created with this mask.
>> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> >> vhostname_access_log.20111020
>> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> >> vhostname_error_log.20111020
>> >>
>> >> Am i missing something?
>> >>
>> >> Sorry if i am missing something obvious.
>> >
>> > I suggest to remove
>> >
>> > $PrivDropToUser syslog
>> > $PrivDropToGroup syslog
>> >
>> > From your config and retry. When it then works, we know for sure it is
>> > related to the permissions.
>> >
>> > Rainer
>> >>
>> >>
>> >> On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
>> >> >> I would like to get opinions about this:
>> >> >>
>> >> >> I have the following line in my rsyslog conf:
>> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> >> >> *.*;auth,authpriv.none ??DynFile
>> >> >>
>> >> >> And it is not working.
>> >> > At a glance it looks ok. Try invoking rsyslog with -N1 to see if it
>> >> > complains about your configuration.
>> >> >
>> >> >> After hours of different tries realized if i remove
>> >> >> ;auth,authpriv.none It starts to work magically.
>> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> >> >> *.* ??DynFile
>> >> >>
>> >> >> However i had to touch the files manually because these error
>> >> >> messages appeared in the log:
>> >> >> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' -
>> >> >> discarding message
>> >> > The dynamic files aren't created when rsyslog starts, so it needs
>> >> > permission to write them after it drops permissions (the default
>> >> > configuration in Ubuntu). If you try to write the file to /var/log
>> >> > (which you are) you will get this error because /var/log is owned
>> >> > by root and syslog cannot write new files there. At our site we
>> >> > work around this by creating a new folder owned by syslog.
>> >> >
>> >> >> It is an Ubuntu 10.04 LTS with the repo install of rsyslog 4.2.0
>> >> > The important lines to note are these:
>> >> > $PrivDropToUser syslog
>> >> > $PrivDropToGroup syslog
>> >> >
>> >> > Which are why the file can't be created dynamically in /var/log.
>> >> >
>> >> >> Do you have any idea what the problem with my original try and why
>> >> >> rsyslog cannot open logfiles?
>> >> >
>> >> > -Ryan Kelly
>> >> > _______________________________________________
>> >> > rsyslog mailing list
>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> > http://www.rsyslog.com
>> >> >
>> >> _______________________________________________
>> >> rsyslog mailing list
>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> http://www.rsyslog.com
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From rgerhards at hq.adiscon.com  Thu Oct 20 15:16:38 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 15:16:38 +0200
Subject: [rsyslog] Dynamic file names
In-Reply-To: <CAO9xhua80Yg1OEPBA_Znhh1P4aHKAPL9S=+zHRDzj5JsR3Dr9A@mail.gmail.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com><20111013104734.GA17060@llserver.lakeliving.com><CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com><CAO9xhuav7Y875cu05gjd5hT5r6Ty_7MH_r5_b5nt187oEXTzpg@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA7281374@GRFEXC.intern.adiscon.com>
	<CAO9xhua80Yg1OEPBA_Znhh1P4aHKAPL9S=+zHRDzj5JsR3Dr9A@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281377@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> Sent: Thursday, October 20, 2011 3:14 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Dynamic file names
> 
> root      2722     1  0 12:19 ?        00:00:00 rsyslogd -c4
> ubuntu 10.04 LTS default settings everywhere

Even more puzzling. I suggest you ask on an Ubuntu list, and would appreciate
if you could post the result here. The Ubuntu package was broken in various
ways due to the way they dropped privileges but did not sync that with the
rest of their packages. The one you use may have such defects (to the best of
my knowledge they still have not solved all issues).

Rainer
> 
> On 20 October 2011 13:37, Rainer Gerhards <rgerhards at hq.adiscon.com>
> wrote:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> >> Sent: Thursday, October 20, 2011 2:18 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] Dynamic file names
> >>
> >> Thank you, that makes it working however:
> >>
> >> This is the default settings
> >> $FileOwner syslog
> >> $FileGroup adm
> >> $FileCreateMode 0640
> >> $DirCreateMode 0755
> >> $Umask 0022
> >> $PrivDropToUser syslog
> >> $PrivDropToGroup syslog
> >>
> >> If I comment out the privileges drop it is working But if I just
> change the
> >> already created files from
> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> >> vhostname_access_log.20111020
> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> >> vhostname_error_log.20111020
> >>
> >> to
> >>
> >> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
> >> vhostname_access_log.20111020
> >> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
> >> vhostname_error_log.20111020
> >>
> >> insted commenting out, it is also starts working.
> >
> > Mhhh, this doesn't make much sense to me. The user should be able to
> open
> > files for writing if it has permissions... and it looks like it has.
> Can you
> > check if rsyslog actually runs under the syslog user?
> >
> > Rainer
> >
> >
> >>
> >>
> >> On 20 October 2011 12:53, Rainer Gerhards <rgerhards at hq.adiscon.com>
> >> wrote:
> >> >> -----Original Message-----
> >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> >> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> >> >> Sent: Thursday, October 20, 2011 1:48 PM
> >> >> To: rsyslog-users
> >> >> Subject: Re: [rsyslog] Dynamic file names
> >> >>
> >> >> I continued to extend my config after i managed to solve this
> issue.
> >> >>
> >> >> Logs are comming in from localhost and remote host on TCP 514.
> >> >>
> >> >> Apaches send their logs to the syslog with the following config:
> >> >>
> >> >> ErrorLog "|/usr/bin/logger -p local6.warn -t
> httpd_error_vhostname"
> >> >> CustomLog "|/usr/bin/logger -p local6.info -t
> httpd_access_vhostname"
> >> >> combined
> >> >>
> >> >> Added the following lines to rsyslog conf $template
> >> >> ApacheLogFormat,"%msg:2:10000%\n"
> >> >> $template
> >> >>
> >> local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONT
> >> >> H%%$DAY%"
> >> >> $template
> >> >>
> >> local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MO
> >> >> NTH%%$DAY%"
> >> >>
> >> >> if $syslogfacility-text == 'local6' and $programname startswith
> >> > 'httpd_error'
> >> >> then -?local6error;ApacheLogFormat #& ~ if $syslogfacility-text
> ==
> > 'local6'
> >> >> and $programname startswith 'httpd_access' then -
> >> >> ?local6access;ApacheLogFormat #& ~
> >> >>
> >> >> I getting this error message in syslog:
> >> >> rsyslogd: Could not open dynamic file
> >> >> '/var/log/vhostname_access_log.20111020' - discarding message
> >> >> rsyslogd: Could not open dynamic file
> >> >> '/var/log/vhostname_error_log.20111020' - discarding message
> >> >>
> >> >> I've already given /var/log to syslog user and the files are
> created
> >> > perfectly
> >> >> however it cannot write them for some reason.
> >> >> I tried to open files in a different location and also same
> effect,
> >> >> files
> >> > are
> >> >> created but rsyslog tells me could not open.
> >> >> Files are created with this mask.
> >> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> >> >> vhostname_access_log.20111020
> >> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
> >> >> vhostname_error_log.20111020
> >> >>
> >> >> Am i missing something?
> >> >>
> >> >> Sorry if i am missing something obvious.
> >> >
> >> > I suggest to remove
> >> >
> >> > $PrivDropToUser syslog
> >> > $PrivDropToGroup syslog
> >> >
> >> > From your config and retry. When it then works, we know for sure
> it is
> >> > related to the permissions.
> >> >
> >> > Rainer
> >> >>
> >> >>
> >> >> On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
> >> >> >> I would like to get opinions about this:
> >> >> >>
> >> >> >> I have the following line in my rsyslog conf:
> >> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> >> >> >> *.*;auth,authpriv.none ??DynFile
> >> >> >>
> >> >> >> And it is not working.
> >> >> > At a glance it looks ok. Try invoking rsyslog with -N1 to see
> if it
> >> >> > complains about your configuration.
> >> >> >
> >> >> >> After hours of different tries realized if i remove
> >> >> >> ;auth,authpriv.none It starts to work magically.
> >> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
> >> >> >> *.* ??DynFile
> >> >> >>
> >> >> >> However i had to touch the files manually because these error
> >> >> >> messages appeared in the log:
> >> >> >> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' -
> >> >> >> discarding message
> >> >> > The dynamic files aren't created when rsyslog starts, so it
> needs
> >> >> > permission to write them after it drops permissions (the
> default
> >> >> > configuration in Ubuntu). If you try to write the file to
> /var/log
> >> >> > (which you are) you will get this error because /var/log is
> owned
> >> >> > by root and syslog cannot write new files there. At our site we
> >> >> > work around this by creating a new folder owned by syslog.
> >> >> >
> >> >> >> It is an Ubuntu 10.04 LTS with the repo install of rsyslog
> 4.2.0
> >> >> > The important lines to note are these:
> >> >> > $PrivDropToUser syslog
> >> >> > $PrivDropToGroup syslog
> >> >> >
> >> >> > Which are why the file can't be created dynamically in
> /var/log.
> >> >> >
> >> >> >> Do you have any idea what the problem with my original try and
> why
> >> >> >> rsyslog cannot open logfiles?
> >> >> >
> >> >> > -Ryan Kelly
> >> >> > _______________________________________________
> >> >> > rsyslog mailing list
> >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >> > http://www.rsyslog.com
> >> >> >
> >> >> _______________________________________________
> >> >> rsyslog mailing list
> >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >> http://www.rsyslog.com
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> >
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From horvath.peter77 at freemail.hu  Thu Oct 20 15:58:10 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Thu, 20 Oct 2011 14:58:10 +0100
Subject: [rsyslog] Dynamic file names
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281377@GRFEXC.intern.adiscon.com>
References: <CAO9xhubYj5kFM5EptQupy1=UBPPh+sT+gDKy3taxfVrU4815EA@mail.gmail.com>
	<20111013104734.GA17060@llserver.lakeliving.com>
	<CAO9xhubTN3YOUMbYzEyotK+QDFiQKgYVcoM6i2Wnv=X4PU4Y1Q@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281372@GRFEXC.intern.adiscon.com>
	<CAO9xhuav7Y875cu05gjd5hT5r6Ty_7MH_r5_b5nt187oEXTzpg@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281374@GRFEXC.intern.adiscon.com>
	<CAO9xhua80Yg1OEPBA_Znhh1P4aHKAPL9S=+zHRDzj5JsR3Dr9A@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281377@GRFEXC.intern.adiscon.com>
Message-ID: <CAO9xhuYPdTqmk4=vHvJXiksOUDAkBsAOESjwUG+j7yc=YfQnzA@mail.gmail.com>

All right i created a bug report with the details.

https://bugs.launchpad.net/ubuntu/+source/rsyslog/+bug/878857

Thanks for the help
Peter

On 20 October 2011 14:16, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> Sent: Thursday, October 20, 2011 3:14 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] Dynamic file names
>>
>> root ? ? ?2722 ? ? 1 ?0 12:19 ? ? ? ? ?00:00:00 rsyslogd -c4
>> ubuntu 10.04 LTS default settings everywhere
>
> Even more puzzling. I suggest you ask on an Ubuntu list, and would appreciate
> if you could post the result here. The Ubuntu package was broken in various
> ways due to the way they dropped privileges but did not sync that with the
> rest of their packages. The one you use may have such defects (to the best of
> my knowledge they still have not solved all issues).
>
> Rainer
>>
>> On 20 October 2011 13:37, Rainer Gerhards <rgerhards at hq.adiscon.com>
>> wrote:
>> >> -----Original Message-----
>> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> >> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> >> Sent: Thursday, October 20, 2011 2:18 PM
>> >> To: rsyslog-users
>> >> Subject: Re: [rsyslog] Dynamic file names
>> >>
>> >> Thank you, that makes it working however:
>> >>
>> >> This is the default settings
>> >> $FileOwner syslog
>> >> $FileGroup adm
>> >> $FileCreateMode 0640
>> >> $DirCreateMode 0755
>> >> $Umask 0022
>> >> $PrivDropToUser syslog
>> >> $PrivDropToGroup syslog
>> >>
>> >> If I comment out the privileges drop it is working But if I just
>> change the
>> >> already created files from
>> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> >> vhostname_access_log.20111020
>> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> >> vhostname_error_log.20111020
>> >>
>> >> to
>> >>
>> >> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
>> >> vhostname_access_log.20111020
>> >> -rw-r----- ?1 syslog ? ?adm ? ? ?0 2011-10-20 11:34
>> >> vhostname_error_log.20111020
>> >>
>> >> insted commenting out, it is also starts working.
>> >
>> > Mhhh, this doesn't make much sense to me. The user should be able to
>> open
>> > files for writing if it has permissions... and it looks like it has.
>> Can you
>> > check if rsyslog actually runs under the syslog user?
>> >
>> > Rainer
>> >
>> >
>> >>
>> >>
>> >> On 20 October 2011 12:53, Rainer Gerhards <rgerhards at hq.adiscon.com>
>> >> wrote:
>> >> >> -----Original Message-----
>> >> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> >> >> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> >> >> Sent: Thursday, October 20, 2011 1:48 PM
>> >> >> To: rsyslog-users
>> >> >> Subject: Re: [rsyslog] Dynamic file names
>> >> >>
>> >> >> I continued to extend my config after i managed to solve this
>> issue.
>> >> >>
>> >> >> Logs are comming in from localhost and remote host on TCP 514.
>> >> >>
>> >> >> Apaches send their logs to the syslog with the following config:
>> >> >>
>> >> >> ErrorLog "|/usr/bin/logger -p local6.warn -t
>> httpd_error_vhostname"
>> >> >> CustomLog "|/usr/bin/logger -p local6.info -t
>> httpd_access_vhostname"
>> >> >> combined
>> >> >>
>> >> >> Added the following lines to rsyslog conf $template
>> >> >> ApacheLogFormat,"%msg:2:10000%\n"
>> >> >> $template
>> >> >>
>> >> local6error,"/var/log/%programname:13:50%_error_log.%$YEAR%%$MONT
>> >> >> H%%$DAY%"
>> >> >> $template
>> >> >>
>> >> local6access,"/var/log/%programname:14:50%_access_log.%$YEAR%%$MO
>> >> >> NTH%%$DAY%"
>> >> >>
>> >> >> if $syslogfacility-text == 'local6' and $programname startswith
>> >> > 'httpd_error'
>> >> >> then -?local6error;ApacheLogFormat #& ~ if $syslogfacility-text
>> ==
>> > 'local6'
>> >> >> and $programname startswith 'httpd_access' then -
>> >> >> ?local6access;ApacheLogFormat #& ~
>> >> >>
>> >> >> I getting this error message in syslog:
>> >> >> rsyslogd: Could not open dynamic file
>> >> >> '/var/log/vhostname_access_log.20111020' - discarding message
>> >> >> rsyslogd: Could not open dynamic file
>> >> >> '/var/log/vhostname_error_log.20111020' - discarding message
>> >> >>
>> >> >> I've already given /var/log to syslog user and the files are
>> created
>> >> > perfectly
>> >> >> however it cannot write them for some reason.
>> >> >> I tried to open files in a different location and also same
>> effect,
>> >> >> files
>> >> > are
>> >> >> created but rsyslog tells me could not open.
>> >> >> Files are created with this mask.
>> >> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> >> >> vhostname_access_log.20111020
>> >> >> -rw-r----- ?1 syslog ? ?syslog ? ? ?0 2011-10-20 11:34
>> >> >> vhostname_error_log.20111020
>> >> >>
>> >> >> Am i missing something?
>> >> >>
>> >> >> Sorry if i am missing something obvious.
>> >> >
>> >> > I suggest to remove
>> >> >
>> >> > $PrivDropToUser syslog
>> >> > $PrivDropToGroup syslog
>> >> >
>> >> > From your config and retry. When it then works, we know for sure
>> it is
>> >> > related to the permissions.
>> >> >
>> >> > Rainer
>> >> >>
>> >> >>
>> >> >> On 13 October 2011 11:47, Ryan Kelly <rpkelly22 at gmail.com> wrote:
>> >> >> >> I would like to get opinions about this:
>> >> >> >>
>> >> >> >> I have the following line in my rsyslog conf:
>> >> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> >> >> >> *.*;auth,authpriv.none ??DynFile
>> >> >> >>
>> >> >> >> And it is not working.
>> >> >> > At a glance it looks ok. Try invoking rsyslog with -N1 to see
>> if it
>> >> >> > complains about your configuration.
>> >> >> >
>> >> >> >> After hours of different tries realized if i remove
>> >> >> >> ;auth,authpriv.none It starts to work magically.
>> >> >> >> $template DynFile,"/var/log/syslog-%HOSTNAME%"
>> >> >> >> *.* ??DynFile
>> >> >> >>
>> >> >> >> However i had to touch the files manually because these error
>> >> >> >> messages appeared in the log:
>> >> >> >> rsyslogd: Could not open dynamic file '/var/log/syslog-XXX' -
>> >> >> >> discarding message
>> >> >> > The dynamic files aren't created when rsyslog starts, so it
>> needs
>> >> >> > permission to write them after it drops permissions (the
>> default
>> >> >> > configuration in Ubuntu). If you try to write the file to
>> /var/log
>> >> >> > (which you are) you will get this error because /var/log is
>> owned
>> >> >> > by root and syslog cannot write new files there. At our site we
>> >> >> > work around this by creating a new folder owned by syslog.
>> >> >> >
>> >> >> >> It is an Ubuntu 10.04 LTS with the repo install of rsyslog
>> 4.2.0
>> >> >> > The important lines to note are these:
>> >> >> > $PrivDropToUser syslog
>> >> >> > $PrivDropToGroup syslog
>> >> >> >
>> >> >> > Which are why the file can't be created dynamically in
>> /var/log.
>> >> >> >
>> >> >> >> Do you have any idea what the problem with my original try and
>> why
>> >> >> >> rsyslog cannot open logfiles?
>> >> >> >
>> >> >> > -Ryan Kelly
>> >> >> > _______________________________________________
>> >> >> > rsyslog mailing list
>> >> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> >> > http://www.rsyslog.com
>> >> >> >
>> >> >> _______________________________________________
>> >> >> rsyslog mailing list
>> >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> >> http://www.rsyslog.com
>> >> > _______________________________________________
>> >> > rsyslog mailing list
>> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> > http://www.rsyslog.com
>> >> >
>> >> _______________________________________________
>> >> rsyslog mailing list
>> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >> http://www.rsyslog.com
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From horvath.peter77 at freemail.hu  Thu Oct 20 18:12:29 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Thu, 20 Oct 2011 17:12:29 +0100
Subject: [rsyslog] program name limitation for remote logging
Message-ID: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>

I am useing the following rules to send apache log perfectly tagged to
syslog and than to remote logging server over tcp 514.
On local servers syslog receives and stores messages perfectly however
on the remote server only the first 32 character of program name
appear in the syslog.

clients:
apache conf:
ErrorLog "|/usr/bin/logger -p local6.warn -t
httpd_eurwebtest01.eurweb.xxxxx.com"
CustomLog "|/usr/bin/logger -p local6.info -t
httpd_eurwebtest01.eurweb.xxxxx.com" combined

server:
rsyslog conf:
$template ApacheLogFormat,"%msg:2:10000%\n"
$template local6error,"/srv/log/apache2/%programname:7:50%_error_log.%$YEAR%%$MONTH%%$DAY%"
$template local6access,"/srv/log/apache2/%programname:7:50%_access_log.%$YEAR%%$MONTH%%$DAY%"

if $syslogfacility-text == 'local6' and $syslogseverity-text == 'warn'
and $programname startswith 'httpd' then -?local6error;ApacheLogFormat
#& ~

if $syslogfacility-text == 'local6' and $syslogseverity-text == 'info'
and $programname startswith 'httpd' then
-?local6access;ApacheLogFormat
#& ~

Any idea ? Is there a limitation on program name if you send it to a
remote host?

Thanks

Peter

From rgerhards at hq.adiscon.com  Thu Oct 20 18:43:47 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 18:43:47 +0200
Subject: [rsyslog] program name limitation for remote logging
In-Reply-To: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>
References: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281378@GRFEXC.intern.adiscon.com>



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> Sent: Thursday, October 20, 2011 6:12 PM
> To: rsyslog-users
> Subject: [rsyslog] program name limitation for remote logging
> 
> I am useing the following rules to send apache log perfectly tagged to
> syslog and than to remote logging server over tcp 514.
> On local servers syslog receives and stores messages perfectly however
> on the remote server only the first 32 character of program name
> appear in the syslog.
> 
> clients:
> apache conf:
> ErrorLog "|/usr/bin/logger -p local6.warn -t
> httpd_eurwebtest01.eurweb.xxxxx.com"
> CustomLog "|/usr/bin/logger -p local6.info -t
> httpd_eurwebtest01.eurweb.xxxxx.com" combined
> 
> server:
> rsyslog conf:
> $template ApacheLogFormat,"%msg:2:10000%\n"
> $template
> local6error,"/srv/log/apache2/%programname:7:50%_error_log.%$YEAR%%$MON
> TH%%$DAY%"
> $template
> local6access,"/srv/log/apache2/%programname:7:50%_access_log.%$YEAR%%$M
> ONTH%%$DAY%"
> 
> if $syslogfacility-text == 'local6' and $syslogseverity-text == 'warn'
> and $programname startswith 'httpd' then -?local6error;ApacheLogFormat
> #& ~
> 
> if $syslogfacility-text == 'local6' and $syslogseverity-text == 'info'
> and $programname startswith 'httpd' then
> -?local6access;ApacheLogFormat
> #& ~
> 
> Any idea ? Is there a limitation on program name if you send it to a
> remote host?

The relevant RFCs limit tag size to 32 octets. So the default template does
this as well. You can create a custom template that does not have this
limitation. Rsyslog can process longer tags.

Rainer

From david at lang.hm  Thu Oct 20 20:03:33 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 20 Oct 2011 11:03:33 -0700 (PDT)
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org>
	<9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1110201059560.32523@asgard.lang.hm>

I think there should be some way of getting the text without the numeric 
value as well.

I actually don't understand why anyone would want both. If you are having 
a system process it that understands the numbers, you don't need the text. 
The text is there to make it human friendly, and the numbers sure don't 
help that.

changing the exiting property to only be the text may break someone's 
stuff, but only if they are either filtering on it, or have a custom 
template. in either case it's a fairly small change to fix the 
configuration.

This would not be a change to make to an existing stable branch, but I 
could see it being reasonable as a cleanup in 6.x

David Lang


  On Thu, 20 Oct 2011, Rainer Gerhards wrote:

> Date: Thu, 20 Oct 2011 11:58:32 +0200
> From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog-users <rsyslog at lists.adiscon.com>
> Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Gregory K. Ruiz-Ade
>> Sent: Wednesday, October 19, 2011 3:21 AM
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] pri-text property incorrectly appending pri?
>>
>> I've been doing some testing and debugging on my rules in rsyslog
>> (5.9.5, built from sources on CentOS 5.6), and I've discovered that the
>> property "pri-text" doesn't just give you the textual form.
>>
>> using:
>>
>> -----
>> $template testfmt, "%pri-text%\n"
>> *.* /var/log/testing;testfmt
>> -----
>>
>> I end up with, for example:
>>
>> -----
>> local0.notice<133>
>> -----
>>
>> in my logfile.
>>
>>
>> On the bright side, I now know why my rules aren't working, like:
>>
>> -----
>> :pri-text, !isequal, "local0.err" ~
>> *.* /var/log/local0.err.log
>> -----
>>
>> Since there is always the contents of "%pri%" tacked on the end,
>> nothing will ever be equal to "local0.err" and my log file stays empty.
>> I can work around this for now by using startswith instead of isequal,
>> but the behavior still bugs me.
>>
>> Is this a bug, or intended behavior?
>
> That's a really ugly issue. I have checked to code, and it was this way all
> the time. It's intentional. While I don't remember introducing this property
> at all, I can see it is intentional by the way the property is formatted
> ("%s<%d>" basically). The doc does not have the number inside the spec, but
> clearly shows it in examples.
>
> Usually, I'd say I remove the numerical PRI, but I don't know who may be
> relying on it. Tough call. Even more ugly is introducing something like
> pri-text-alternate, but maybe that's the way to go.
>
> Anybody with suggestions?
>
>
> As a side-note, using
>
> local0.err /var/log/local0.err.log
>
> as a rule is much more efficient than what you wrote above.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From wma443 at herr-der-mails.de  Thu Oct 20 20:01:42 2011
From: wma443 at herr-der-mails.de (wma443 at herr-der-mails.de)
Date: Thu, 20 Oct 2011 20:01:42 +0200
Subject: [rsyslog] minimum gcc version required for compilation of
 rsyslog 4.8.0/5.8.5
In-Reply-To: <20111019134332.201190@gmx.net>
References: <20111019134332.201190@gmx.net>
Message-ID: <20111020180142.91090@gmx.net>

Dear list,

Do you know which is the minimum version of gcc that is needed to compile rsyslog successfully?

Kind regards,

Carl


-------- Original-Nachricht --------
> Datum: Wed, 19 Oct 2011 15:43:32 +0200
> Von: wma443 at herr-der-mails.de
> An: rsyslog at lists.adiscon.com
> Betreff: [rsyslog] minimum gcc version required for compilation of rsyslog	4.8.0/5.8.5

> Dear list,
> 
> Does anyone have experience in compiling one of the current rsyslog
> versions (4.8.0 or 5.8.5) with a version of gcc that is 4.1.0 or older?
> 
> Kind regards,
> 
> Carl
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Thu Oct 20 20:15:16 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Oct 2011 20:15:16 +0200
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <alpine.DEB.2.02.1110201059560.32523@asgard.lang.hm>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1110201059560.32523@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728137C@GRFEXC.intern.adiscon.com>



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, October 20, 2011 8:04 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> 
> I think there should be some way of getting the text without the
> numeric
> value as well.
> 
> I actually don't understand why anyone would want both. If you are
> having
> a system process it that understands the numbers, you don't need the
> text.
> The text is there to make it human friendly, and the numbers sure don't
> help that.
> 
> changing the exiting property to only be the text may break someone's
> stuff, but only if they are either filtering on it, or have a custom
> template. in either case it's a fairly small change to fix the
> configuration.
> 
> This would not be a change to make to an existing stable branch, but I
> could see it being reasonable as a cleanup in 6.x

That sounds like a good solution! Acutally, I would bet that nobody actually
uses this property -- otherwise the question should have come up before. But
you never know...

Rainer
> 
> David Lang
> 
> 
>   On Thu, 20 Oct 2011, Rainer Gerhards wrote:
> 
> > Date: Thu, 20 Oct 2011 11:58:32 +0200
> > From: Rainer Gerhards <rgerhards at hq.adiscon.com>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog-users <rsyslog at lists.adiscon.com>
> > Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Gregory K. Ruiz-Ade
> >> Sent: Wednesday, October 19, 2011 3:21 AM
> >> To: rsyslog at lists.adiscon.com
> >> Subject: [rsyslog] pri-text property incorrectly appending pri?
> >>
> >> I've been doing some testing and debugging on my rules in rsyslog
> >> (5.9.5, built from sources on CentOS 5.6), and I've discovered that
> the
> >> property "pri-text" doesn't just give you the textual form.
> >>
> >> using:
> >>
> >> -----
> >> $template testfmt, "%pri-text%\n"
> >> *.* /var/log/testing;testfmt
> >> -----
> >>
> >> I end up with, for example:
> >>
> >> -----
> >> local0.notice<133>
> >> -----
> >>
> >> in my logfile.
> >>
> >>
> >> On the bright side, I now know why my rules aren't working, like:
> >>
> >> -----
> >> :pri-text, !isequal, "local0.err" ~
> >> *.* /var/log/local0.err.log
> >> -----
> >>
> >> Since there is always the contents of "%pri%" tacked on the end,
> >> nothing will ever be equal to "local0.err" and my log file stays
> empty.
> >> I can work around this for now by using startswith instead of
> isequal,
> >> but the behavior still bugs me.
> >>
> >> Is this a bug, or intended behavior?
> >
> > That's a really ugly issue. I have checked to code, and it was this
> way all
> > the time. It's intentional. While I don't remember introducing this
> property
> > at all, I can see it is intentional by the way the property is
> formatted
> > ("%s<%d>" basically). The doc does not have the number inside the
> spec, but
> > clearly shows it in examples.
> >
> > Usually, I'd say I remove the numerical PRI, but I don't know who may
> be
> > relying on it. Tough call. Even more ugly is introducing something
> like
> > pri-text-alternate, but maybe that's the way to go.
> >
> > Anybody with suggestions?
> >
> >
> > As a side-note, using
> >
> > local0.err /var/log/local0.err.log
> >
> > as a rule is much more efficient than what you wrote above.
> >
> > Rainer
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From david at lang.hm  Thu Oct 20 20:16:40 2011
From: david at lang.hm (david at lang.hm)
Date: Thu, 20 Oct 2011 11:16:40 -0700 (PDT)
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728137C@GRFEXC.intern.adiscon.com>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1110201059560.32523@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA728137C@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1110201116160.32523@asgard.lang.hm>

On Thu, 20 Oct 2011, Rainer Gerhards wrote:

>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Thursday, October 20, 2011 8:04 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
>>
>> I think there should be some way of getting the text without the
>> numeric
>> value as well.
>>
>> I actually don't understand why anyone would want both. If you are
>> having
>> a system process it that understands the numbers, you don't need the
>> text.
>> The text is there to make it human friendly, and the numbers sure don't
>> help that.
>>
>> changing the exiting property to only be the text may break someone's
>> stuff, but only if they are either filtering on it, or have a custom
>> template. in either case it's a fairly small change to fix the
>> configuration.
>>
>> This would not be a change to make to an existing stable branch, but I
>> could see it being reasonable as a cleanup in 6.x
>
> That sounds like a good solution! Acutally, I would bet that nobody actually
> uses this property -- otherwise the question should have come up before. But
> you never know...

that's my buess as well.

David Lang

From gkra at unnerving.org  Fri Oct 21 01:13:06 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Thu, 20 Oct 2011 16:13:06 -0700
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org>
	<9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
Message-ID: <F862CE80-5DF5-46DF-AFBC-E114AA5EF089@unnerving.org>

On Oct 20, 2011, at 2:58 AM, Rainer Gerhards wrote:

>> -----
>> :pri-text, !isequal, "local0.err" ~
>> *.* /var/log/local0.err.log
>> -----
> 
> Anybody with suggestions?

I'm all for fixing it so the code only emitted "%s", but that's me. :)

> As a side-note, using
> 
> local0.err /var/log/local0.err.log


Oh, I understand. Above was basically a vast simplification of what was in my rules just to illustrate the point, and I could have even written it as:

:pri-text, isequal, "local0.err" /var/log/local0.err.log

for purposes of illustrating the bug as I found it. You are quite correct, however, that it's simply a more complicated (filter v. selector) way of doing:

local0.err /var/log/local0.err.log

To be perfectly honest, that set of rules has since been scrapped for a cleaner single if-statement filter at the end of a ruleset with it's own disk-backed queues.

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu




From rgerhards at hq.adiscon.com  Fri Oct 21 07:55:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Oct 2011 07:55:37 +0200
Subject: [rsyslog] minimum gcc version required for compilation of
	rsyslog 4.8.0/5.8.5
In-Reply-To: <20111020180142.91090@gmx.net>
References: <20111019134332.201190@gmx.net> <20111020180142.91090@gmx.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728137F@GRFEXC.intern.adiscon.com>

Just so that you get an (admittedly unhelpful) answer: I don't know.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of wma443 at herr-der-mails.de
> Sent: Thursday, October 20, 2011 8:02 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] minimum gcc version required for compilation of
> rsyslog 4.8.0/5.8.5
> 
> Dear list,
> 
> Do you know which is the minimum version of gcc that is needed to
> compile rsyslog successfully?
> 
> Kind regards,
> 
> Carl
> 
> 
> -------- Original-Nachricht --------
> > Datum: Wed, 19 Oct 2011 15:43:32 +0200
> > Von: wma443 at herr-der-mails.de
> > An: rsyslog at lists.adiscon.com
> > Betreff: [rsyslog] minimum gcc version required for compilation of
> rsyslog	4.8.0/5.8.5
> 
> > Dear list,
> >
> > Does anyone have experience in compiling one of the current rsyslog
> > versions (4.8.0 or 5.8.5) with a version of gcc that is 4.1.0 or
> older?
> >
> > Kind regards,
> >
> > Carl
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rosenski at wave-computer.de  Fri Oct 21 08:52:13 2011
From: rosenski at wave-computer.de (Axel Rosenski)
Date: Fri, 21 Oct 2011 08:52:13 +0200
Subject: [rsyslog] program name limitation for remote logging
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281378@GRFEXC.intern.adiscon.com>
References: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281378@GRFEXC.intern.adiscon.com>
Message-ID: <201110210852.13688.rosenski@wave-computer.de>

Hi, 

Am Donnerstag 20 Oktober 2011, 18:43:47 schrieb Rainer Gerhards:
> > Any idea ? Is there a limitation on program name if you send it to a
> > remote host?
> 
> The relevant RFCs limit tag size to 32 octets. So the default template does
> this as well. You can create a custom template that does not have this
> limitation. Rsyslog can process longer tags.

where do I have to define this custom template? It's not on the rsyslog server

I defined $template ApacheAccessLogFile, 
"/data/logs/apache2/%procid%/%procid%_access_log" and have the limitation of 
32 characters.

Wherelse can I define this template? 

Regards, 
Axel 

-- 
Axel Rosenski
- Administration -
______________________________
Wave Computersysteme GmbH
Philipp-Reis-Str. 1-3 / 9
35440 Linden

Gesch?ftsf?hrer: Carsten Kellmann
Registergericht Gie?en HRB 1823

Tel.: +49 (0)6403 / 9050 8317
Fax: +49 (0)6403 / 9050 5089
mailto:rosenski at wave-computer.de
http://www.wave-computer.de


From rgerhards at hq.adiscon.com  Fri Oct 21 09:48:04 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Oct 2011 09:48:04 +0200
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <F862CE80-5DF5-46DF-AFBC-E114AA5EF089@unnerving.org>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
	<F862CE80-5DF5-46DF-AFBC-E114AA5EF089@unnerving.org>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281380@GRFEXC.intern.adiscon.com>



> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Gregory K. Ruiz-Ade
> Sent: Friday, October 21, 2011 1:13 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> 
> On Oct 20, 2011, at 2:58 AM, Rainer Gerhards wrote:
> 
> >> -----
> >> :pri-text, !isequal, "local0.err" ~
> >> *.* /var/log/local0.err.log
> >> -----
> >
> > Anybody with suggestions?
> 
> I'm all for fixing it so the code only emitted "%s", but that's me. :)

I'll change that today in v6 as of David's suggestion.

> 
> > As a side-note, using
> >
> > local0.err /var/log/local0.err.log
> 
> 
> Oh, I understand. Above was basically a vast simplification of what was
> in my rules just to illustrate the point, 

I appreciate these simplifications, as they make helping you much easier ;) I
just wanted to make sure you are aware of the performance differences (many
folks are not).

> and I could have even written
> it as:
> 
> :pri-text, isequal, "local0.err" /var/log/local0.err.log

Yes, and it would be more efficient than the if -- but still less efficient
than the simple selector filter.

If you are up for speed, use
- traditional selector
- property 
- script
Filter in that order and go to the next one only if you actually need to. IF
speed doesn't matter, this is obviously unimportant.

Rainer

> 
> for purposes of illustrating the bug as I found it. You are quite
> correct, however, that it's simply a more complicated (filter v.
> selector) way of doing:
> 
> local0.err /var/log/local0.err.log
> 
> To be perfectly honest, that set of rules has since been scrapped for a
> cleaner single if-statement filter at the end of a ruleset with it's
> own disk-backed queues.
> 
> Gregory
> 
> --
> Gregory K. Ruiz-Ade <gkra at unnerving.org>
> OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu
> 
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From rgerhards at hq.adiscon.com  Fri Oct 21 11:01:37 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Oct 2011 11:01:37 +0200
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281380@GRFEXC.intern.adiscon.com>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com><F862CE80-5DF5-46DF-AFBC-E114AA5EF089@unnerving.org>
	<9B6E2A8877C38245BFB15CC491A11DA7281380@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281383@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, October 21, 2011 9:48 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> 
> 
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Gregory K. Ruiz-Ade
> > Sent: Friday, October 21, 2011 1:13 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> >
> > On Oct 20, 2011, at 2:58 AM, Rainer Gerhards wrote:
> >
> > >> -----
> > >> :pri-text, !isequal, "local0.err" ~
> > >> *.* /var/log/local0.err.log
> > >> -----
> > >
> > > Anybody with suggestions?
> >
> > I'm all for fixing it so the code only emitted "%s", but that's me.
> :)
> 
> I'll change that today in v6 as of David's suggestion.

Done:
http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=43d9709847de6d286825ed8e
b4db3ac702d8ab00

Rainer

From horvath.peter77 at freemail.hu  Fri Oct 21 11:02:00 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Fri, 21 Oct 2011 10:02:00 +0100
Subject: [rsyslog] program name limitation for remote logging
In-Reply-To: <201110210852.13688.rosenski@wave-computer.de>
References: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281378@GRFEXC.intern.adiscon.com>
	<201110210852.13688.rosenski@wave-computer.de>
Message-ID: <CAO9xhubDeELA3U_v-+6pnO8CuA-Q0_sV4vdi7BrMD4EOb8fCKw@mail.gmail.com>

My question is the same i've found the option to change date format
but not the message.
Could you point me to the right direction.

On 21 October 2011 07:52, Axel Rosenski <rosenski at wave-computer.de> wrote:
> Hi,
>
> Am Donnerstag 20 Oktober 2011, 18:43:47 schrieb Rainer Gerhards:
>> > Any idea ? Is there a limitation on program name if you send it to a
>> > remote host?
>>
>> The relevant RFCs limit tag size to 32 octets. So the default template does
>> this as well. You can create a custom template that does not have this
>> limitation. Rsyslog can process longer tags.
>
> where do I have to define this custom template? It's not on the rsyslog server
>
> I defined $template ApacheAccessLogFile,
> "/data/logs/apache2/%procid%/%procid%_access_log" and have the limitation of
> 32 characters.
>
> Wherelse can I define this template?
>
> Regards,
> Axel
>
> --
> Axel Rosenski
> - Administration -
> ______________________________
> Wave Computersysteme GmbH
> Philipp-Reis-Str. 1-3 / 9
> 35440 Linden
>
> Gesch?ftsf?hrer: Carsten Kellmann
> Registergericht Gie?en HRB 1823
>
> Tel.: +49 (0)6403 / 9050 8317
> Fax: +49 (0)6403 / 9050 5089
> mailto:rosenski at wave-computer.de
> http://www.wave-computer.de
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From teifler at adiscon.com  Fri Oct 21 16:46:57 2011
From: teifler at adiscon.com (Tim Eifler)
Date: Fri, 21 Oct 2011 16:46:57 +0200
Subject: [rsyslog] rsyslog 5.8.6 (v5-stable) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728138D@GRFEXC.intern.adiscon.com>

Hi all,

This is a maintenance release offering bug fixes. For example for a small bug
in property-based filter and a fix for $ActionExecOnlyOnce and more .For more
detailed information, please read the ChangeLog.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-8-6-v5-stable/
<http://www.rsyslog.com/changelog-for-5-8-6-v5-stable/> 

Download:

http://www.rsyslog.com/rsyslog-5-8-6-v5-stable/
<http://www.rsyslog.com/rsyslog-5-8-6-v5-stable/> 

As always, feedback is appreciated.

Best regards,

Tim Eifler

 


From rgerhards at hq.adiscon.com  Fri Oct 21 17:07:19 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Oct 2011 17:07:19 +0200
Subject: [rsyslog] program name limitation for remote logging
In-Reply-To: <CAO9xhubDeELA3U_v-+6pnO8CuA-Q0_sV4vdi7BrMD4EOb8fCKw@mail.gmail.com>
References: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA7281378@GRFEXC.intern.adiscon.com><201110210852.13688.rosenski@wave-computer.de>
	<CAO9xhubDeELA3U_v-+6pnO8CuA-Q0_sV4vdi7BrMD4EOb8fCKw@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728138E@GRFEXC.intern.adiscon.com>

Hi Peter and Axel,

I created a small article for you:

http://www.rsyslog.com/sende-messages-with-tags-larger-than-32-characters/

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
> Sent: Friday, October 21, 2011 11:02 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] program name limitation for remote logging
> 
> My question is the same i've found the option to change date format but not
> the message.
> Could you point me to the right direction.
> 
> On 21 October 2011 07:52, Axel Rosenski <rosenski at wave-computer.de>
> wrote:
> > Hi,
> >
> > Am Donnerstag 20 Oktober 2011, 18:43:47 schrieb Rainer Gerhards:
> >> > Any idea ? Is there a limitation on program name if you send it to
> >> > a remote host?
> >>
> >> The relevant RFCs limit tag size to 32 octets. So the default
> >> template does this as well. You can create a custom template that
> >> does not have this limitation. Rsyslog can process longer tags.
> >
> > where do I have to define this custom template? It's not on the
> > rsyslog server
> >
> > I defined $template ApacheAccessLogFile,
> > "/data/logs/apache2/%procid%/%procid%_access_log" and have the
> > limitation of
> > 32 characters.
> >
> > Wherelse can I define this template?
> >
> > Regards,
> > Axel
> >
> > --
> > Axel Rosenski
> > - Administration -
> > ______________________________
> > Wave Computersysteme GmbH
> > Philipp-Reis-Str. 1-3 / 9
> > 35440 Linden
> >
> > Gesch?ftsf?hrer: Carsten Kellmann
> > Registergericht Gie?en HRB 1823
> >
> > Tel.: +49 (0)6403 / 9050 8317
> > Fax: +49 (0)6403 / 9050 5089
> > mailto:rosenski at wave-computer.de
> > http://www.wave-computer.de
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From horvath.peter77 at freemail.hu  Fri Oct 21 18:19:31 2011
From: horvath.peter77 at freemail.hu (Peter Horvath)
Date: Fri, 21 Oct 2011 17:19:31 +0100
Subject: [rsyslog] program name limitation for remote logging
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728138E@GRFEXC.intern.adiscon.com>
References: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA7281378@GRFEXC.intern.adiscon.com>
	<201110210852.13688.rosenski@wave-computer.de>
	<CAO9xhubDeELA3U_v-+6pnO8CuA-Q0_sV4vdi7BrMD4EOb8fCKw@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA728138E@GRFEXC.intern.adiscon.com>
Message-ID: <CAO9xhuYz1Hv29hf-L5UXAC3XbNQ7j=vrqdUhuRvkzx7TKyqRyg@mail.gmail.com>

Working like a charm thank you very much.

Cheers,
Peter

On 21 October 2011 16:07, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
> Hi Peter and Axel,
>
> I created a small article for you:
>
> http://www.rsyslog.com/sende-messages-with-tags-larger-than-32-characters/
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Peter Horvath
>> Sent: Friday, October 21, 2011 11:02 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] program name limitation for remote logging
>>
>> My question is the same i've found the option to change date format but not
>> the message.
>> Could you point me to the right direction.
>>
>> On 21 October 2011 07:52, Axel Rosenski <rosenski at wave-computer.de>
>> wrote:
>> > Hi,
>> >
>> > Am Donnerstag 20 Oktober 2011, 18:43:47 schrieb Rainer Gerhards:
>> >> > Any idea ? Is there a limitation on program name if you send it to
>> >> > a remote host?
>> >>
>> >> The relevant RFCs limit tag size to 32 octets. So the default
>> >> template does this as well. You can create a custom template that
>> >> does not have this limitation. Rsyslog can process longer tags.
>> >
>> > where do I have to define this custom template? It's not on the
>> > rsyslog server
>> >
>> > I defined $template ApacheAccessLogFile,
>> > "/data/logs/apache2/%procid%/%procid%_access_log" and have the
>> > limitation of
>> > 32 characters.
>> >
>> > Wherelse can I define this template?
>> >
>> > Regards,
>> > Axel
>> >
>> > --
>> > Axel Rosenski
>> > - Administration -
>> > ______________________________
>> > Wave Computersysteme GmbH
>> > Philipp-Reis-Str. 1-3 / 9
>> > 35440 Linden
>> >
>> > Gesch?ftsf?hrer: Carsten Kellmann
>> > Registergericht Gie?en HRB 1823
>> >
>> > Tel.: +49 (0)6403 / 9050 8317
>> > Fax: +49 (0)6403 / 9050 5089
>> > mailto:rosenski at wave-computer.de
>> > http://www.wave-computer.de
>> >
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From david at lang.hm  Fri Oct 21 22:12:46 2011
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Oct 2011 13:12:46 -0700 (PDT)
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281380@GRFEXC.intern.adiscon.com>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com>
	<F862CE80-5DF5-46DF-AFBC-E114AA5EF089@unnerving.org>
	<9B6E2A8877C38245BFB15CC491A11DA7281380@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.02.1110211311510.13493@asgard.lang.hm>

On Fri, 21 Oct 2011, Rainer Gerhards wrote:

> Yes, and it would be more efficient than the if -- but still less efficient
> than the simple selector filter.
>
> If you are up for speed, use
> - traditional selector
> - property
> - script
> Filter in that order and go to the next one only if you actually need to. IF
> speed doesn't matter, this is obviously unimportant.

It would be handy to have some idea of how big the speed differences are 
between these (if anyone has some time to run benchmarks)

David Lang

From rgerhards at hq.adiscon.com  Sat Oct 22 12:00:36 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 22 Oct 2011 12:00:36 +0200
Subject: [rsyslog] pri-text property incorrectly appending pri?
In-Reply-To: <alpine.DEB.2.02.1110211311510.13493@asgard.lang.hm>
References: <C3750B8A-943E-4F41-B95F-DCD4389F1AF1@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA728136E@GRFEXC.intern.adiscon.com><F862CE80-5DF5-46DF-AFBC-E114AA5EF089@unnerving.org><9B6E2A8877C38245BFB15CC491A11DA7281380@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.02.1110211311510.13493@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281390@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, October 21, 2011 10:13 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] pri-text property incorrectly appending pri?
> 
> On Fri, 21 Oct 2011, Rainer Gerhards wrote:
> 
> > Yes, and it would be more efficient than the if -- but still less
> efficient
> > than the simple selector filter.
> >
> > If you are up for speed, use
> > - traditional selector
> > - property
> > - script
> > Filter in that order and go to the next one only if you actually need
> to. IF
> > speed doesn't matter, this is obviously unimportant.
> 
> It would be handy to have some idea of how big the speed differences
> are
> between these (if anyone has some time to run benchmarks)

I have never benchmarked selectors vs. properties. There is some difference
(selectors require exactly one memory access!), but I'd assume it usually is
not that big (except, of course, for regex matches). The script filters in v4
and v5 are really slow, order of 10+ times slower than selectors. In v6.3+
script-based filters have much better efficiency, I'd expect (but did not yet
benchmark) lightly more overhead than property based filters.

Rainer

From rosenski at wave-computer.de  Mon Oct 24 09:54:38 2011
From: rosenski at wave-computer.de (Axel Rosenski)
Date: Mon, 24 Oct 2011 09:54:38 +0200
Subject: [rsyslog] program name limitation for remote logging
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728138E@GRFEXC.intern.adiscon.com>
References: <CAO9xhuauoM6n6FTB5ZfZOVVGTKPgOsSqx707oyCRbD3Kuo+QbA@mail.gmail.com>
	<CAO9xhubDeELA3U_v-+6pnO8CuA-Q0_sV4vdi7BrMD4EOb8fCKw@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA728138E@GRFEXC.intern.adiscon.com>
Message-ID: <201110240954.38585.rosenski@wave-computer.de>

Hi Rainer, 

thanks, thats great! 

Am Freitag 21 Oktober 2011, 17:07:19 schrieb Rainer Gerhards:
> Hi Peter and Axel,
> 
> I created a small article for you:
> http://www.rsyslog.com/sende-messages-with-tags-larger-than-32-characters/

Regards, 
Axel

-- 
Axel Rosenski
- Administration -
______________________________
Wave Computersysteme GmbH
Philipp-Reis-Str. 1-3 / 9
35440 Linden

Gesch?ftsf?hrer: Carsten Kellmann
Registergericht Gie?en HRB 1823

Tel.: +49 (0)6403 / 9050 8317
Fax: +49 (0)6403 / 9050 5089
mailto:rosenski at wave-computer.de
http://www.wave-computer.de


From kyle.hubbard at Q1Labs.com  Mon Oct 24 16:02:45 2011
From: kyle.hubbard at Q1Labs.com (Kyle Hubbard)
Date: Mon, 24 Oct 2011 14:02:45 +0000
Subject: [rsyslog] tcp syslog payload getting broken up with newlines
Message-ID: <AF3432BB227ACE4C895D3A01FD12076B37A8213A@Q1EXCH03.Q1LABS.INC>

Hi everyone,

I'm having an issue where I'm trying to send a relatively long payload through tcp rsyslog, and it appears to be getting split up, with new lines being inserted into the payload.

<158>Oct  5 11:39:38 webgate logger: 10.100.100.100 - - [05/Oct/2011:11:39:30 -0400] "GET http://google.com/sakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaj\sakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaskjdsakldjaskljdlaskjdlasjdlasjdlajdlaj HTTP/1.1" 403 1492 000 0 0 0 1314 190 0 0 0

Above is my event (a junk event created to test a firewall). Below is the relevant entry in /etc/rsyslog.confg:

local3.*                                                @@172.16.160.154

a tcupdump on the box receiving the events shows that my event is getting truncated and sent incorrectly. Is there a payload size limit to rsyslog that is configurable? Any relevant gotchas when sending relatively large payloads via tcp?

From rpkelly22 at gmail.com  Mon Oct 24 16:22:10 2011
From: rpkelly22 at gmail.com (Ryan Kelly)
Date: Mon, 24 Oct 2011 10:22:10 -0400
Subject: [rsyslog] tcp syslog payload getting broken up with newlines
In-Reply-To: <AF3432BB227ACE4C895D3A01FD12076B37A8213A@Q1EXCH03.Q1LABS.INC>
References: <AF3432BB227ACE4C895D3A01FD12076B37A8213A@Q1EXCH03.Q1LABS.INC>
Message-ID: <20111024142210.GA14943@llserver.lakeliving.com>

> a tcupdump on the box receiving the events shows that my event is
> getting truncated and sent incorrectly. Is there a payload size limit
> to rsyslog that is configurable? Any relevant gotchas when sending
> relatively large payloads via tcp?
Have a look at http://www.rsyslog.com/doc/rsyslog_conf_global.html
$MaxMessageSize

-Ryan Kelly

From kyle.hubbard at Q1Labs.com  Mon Oct 24 16:57:28 2011
From: kyle.hubbard at Q1Labs.com (Kyle Hubbard)
Date: Mon, 24 Oct 2011 14:57:28 +0000
Subject: [rsyslog] tcp syslog payload getting broken up with newlines
In-Reply-To: <20111024142210.GA14943@llserver.lakeliving.com>
References: <AF3432BB227ACE4C895D3A01FD12076B37A8213A@Q1EXCH03.Q1LABS.INC>,
	<20111024142210.GA14943@llserver.lakeliving.com>
Message-ID: <AF3432BB227ACE4C895D3A01FD12076B37A8219C@Q1EXCH03.Q1LABS.INC>

Thanks, I changed $MaxMessageSize to 64k, restarted rsyslog, but I'm still seeing the same behaviour. 

I should mention that I'm using /bin/logger to send the syslog messages. Is this the proper executable to use?

/var/log/messages on the rsyslog box shows the same truncated payloads that I'm seeing on the destination box.


>> a tcupdump on the box receiving the events shows that my event is
>> getting truncated and sent incorrectly. Is there a payload size limit
>> to rsyslog that is configurable? Any relevant gotchas when sending
>> relatively large payloads via tcp?
>Have a look at http://www.rsyslog.com/doc/rsyslog_conf_global.html
>$MaxMessageSize

>-Ryan Kelly

From f at zz.de  Mon Oct 24 16:59:58 2011
From: f at zz.de (Florian Lohoff)
Date: Mon, 24 Oct 2011 16:59:58 +0200
Subject: [rsyslog] Syslog message stripped chars at the front
Message-ID: <20111024145958.GB5353@pax.zz.de>


Hi,
i am seeing a strange problem where syslog messages received from remote
get stripped a couple of chars at the front. This seems to happen
depending on the source of the message:

Config:

$template DynFile,"/var/log/remote/%fromhost%-%timegenerated:1:10:date-rfc3339%"
$template hostorip,"%TIMESTAMP:::date-rfc3339% %fromhost% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
:source , !isequal , "localhost" ?DynFile
:source , !isequal , "localhost" /var/log/remote.log;hostorip

Source A (Redback SE600)

	14:51:25.364326 IP (tos 0x0, ttl 63, id 49237, offset 0, flags [none], proto UDP (17), length 107)
	    172.30.0.7.63458 > 172.30.16.16.514: SYSLOG, length: 79
		Facility local7 (23), Severity info (6)
		Msg: Oct 24 14:51:25.351: %CSM-6-PORT: ethernet 2/1 link state UP, admin is UP\0x0a

	2011-10-24T14:51:25.373047+00:00 frnk1-bras1 24 14:51:25.351: %CSM-6-PORT: ethernet 2/1 link state UP, admin is UP

As one can see the "Oct " is stripped of the beginning of the message. Logging "rawmsg" shows
the full message including 
A different source (Cisco ASR9k):

	14:51:34.487744 IP (tos 0x0, ttl 30, id 64430, offset 0, flags [none], proto UDP (17), length 189)
	    172.30.0.2.514 > 172.30.16.16.514: SYSLOG, length: 161
		Facility local7 (23), Severity info (6)
		Msg: 467: LC/0/0/CPU0:Oct 24 14:51:34.476 : bfd_agent[123]: %L2-BFD-6-SESSION_STATE_UP : BFD session to neighbor 172.30.16.42 on interface TenGigE0/0/0/6 is up \0x0a

	2011-10-24T14:51:34.487834+00:00 frnk1-cr2 467: LC/0/0/CPU0:Oct 24 14:51:34.476 : bfd_agent[123]: %L2-BFD-6-SESSION_STATE_UP : BFD session to neighbor 172.30.16.42 on interface TenGigE0/0/0/6 is up 

Nothing stripped. I dont see anything obvious ....

Flo
-- 
Florian Lohoff                                                 f at zz.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111024/35c240fd/attachment.pgp>

From rgerhards at hq.adiscon.com  Mon Oct 24 17:13:52 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Oct 2011 17:13:52 +0200
Subject: [rsyslog] tcp syslog payload getting broken up with newlines
In-Reply-To: <AF3432BB227ACE4C895D3A01FD12076B37A8219C@Q1EXCH03.Q1LABS.INC>
References: <AF3432BB227ACE4C895D3A01FD12076B37A8213A@Q1EXCH03.Q1LABS.INC>,
	<20111024142210.GA14943@llserver.lakeliving.com>
	<AF3432BB227ACE4C895D3A01FD12076B37A8219C@Q1EXCH03.Q1LABS.INC>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728139E@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Kyle Hubbard
> Sent: Monday, October 24, 2011 4:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] tcp syslog payload getting broken up with
> newlines
> 
> Thanks, I changed $MaxMessageSize to 64k, restarted rsyslog, but I'm
> still seeing the same behaviour.

It's important to set this directive right at the top of rsyslog.conf. Based
on how pre v6-config works, only modules get the new limit that are loaded
after it is set.

Rainer
> 
> I should mention that I'm using /bin/logger to send the syslog
> messages. Is this the proper executable to use?
> 
> /var/log/messages on the rsyslog box shows the same truncated payloads
> that I'm seeing on the destination box.
> 
> 
> >> a tcupdump on the box receiving the events shows that my event is
> >> getting truncated and sent incorrectly. Is there a payload size
> limit
> >> to rsyslog that is configurable? Any relevant gotchas when sending
> >> relatively large payloads via tcp?
> >Have a look at http://www.rsyslog.com/doc/rsyslog_conf_global.html
> >$MaxMessageSize
> 
> >-Ryan Kelly
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From kyle.hubbard at Q1Labs.com  Mon Oct 24 19:37:28 2011
From: kyle.hubbard at Q1Labs.com (Kyle Hubbard)
Date: Mon, 24 Oct 2011 17:37:28 +0000
Subject: [rsyslog] tcp syslog payload getting broken up with newlines
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728139E@GRFEXC.intern.adiscon.com>
References: <AF3432BB227ACE4C895D3A01FD12076B37A8213A@Q1EXCH03.Q1LABS.INC>,<
	20111024142210.GA14943@llserver.lakeliving.com><AF3432BB227ACE4C895D3A01FD12076B37A8219C@Q1EXCH03.Q1LABS.INC>,
	<9B6E2A8877C38245BFB15CC491A11DA728139E@GRFEXC.intern.adiscon.com>
Message-ID: <AF3432BB227ACE4C895D3A01FD12076B37A8222A@Q1EXCH03.Q1LABS.INC>

I moved $MaxMessageSize 64k to the top of /etc/rsyslog.conf, still no change in behaviour. Here is an excerpt from the rsyslog debug log:

8256.650601925:main queue:Reg/w0: TCP sent 1061 bytes, requested 1061

It is consistently cutting off my event after the 1061st byte, even if the event characters are changed or moved around.

8256.650271286:imuxsock.c: logmsg: flags 4, from 'webgate', msg [msg omitted]^@^@^N?r?l<86>^H^@^@^@^@^H?a at r^A^E^H<90><9d>^G^H^D^@^@^@Xa<84>^H?l<86>^H<88>g<86>^H<85>g<85>^H^H?a at g?^E^H?l<86>^H<85>g<85>^H^XMessage has legacy syslog format.

Is that gibberish at the end a problem? It is not in the original event.

Thanks for the assistance thus far.

>> Thanks, I changed $MaxMessageSize to 64k, restarted rsyslog, but I'm
>> still seeing the same behaviour.

>It's important to set this directive right at the top of rsyslog.conf. Based
>on how pre v6-config works, only modules get the new limit that are loaded
>after it is set.

>Rainer


From a.piesk at gmx.net  Mon Oct 24 22:41:09 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Mon, 24 Oct 2011 22:41:09 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <4E9DC017.7070100@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>	<4E9C6D4B.4000602@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>
	<4E9DC017.7070100@gmx.net>
Message-ID: <4EA5CD65.9090905@gmx.net>

On 18.10.2011 20:06, Andreas Piesk wrote:
> On 18.10.2011 08:39, Rainer Gerhards wrote:
>>
>> Indeed and that is another story. Of course, that should not be blocked.
>> Trying to dig deeper into the issue: what happens if you do not monitor the
>> file but just use syslog() input. Does it then block?
>>
> 
> no, without any file monitors it does not block.
> even if i have a file monitor configured but do not write into the monitored file is does not block.
> 

any news on this issue?

in the meantime i tested your idea of changing imfile's flow control type:

# cat rsyslog-filedelay.patch
diff -Purp rsyslog-5.8.6/plugins/imfile/imfile.c rsyslog-5.8.6.mod/plugins/imfile/imfile.c
--- rsyslog-5.8.6/plugins/imfile/imfile.c	2011-10-21 11:53:02.000000000 +0200
+++ rsyslog-5.8.6.mod/plugins/imfile/imfile.c	2011-10-24 21:05:30.000000000 +0200
@@ -113,7 +113,7 @@ static rsRetVal enqLine(fileInfo_t *pInf
 	}

 	CHKiRet(msgConstruct(&pMsg));
-	MsgSetFlowControlType(pMsg, eFLOWCTL_FULL_DELAY);
+	MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY);
 	MsgSetInputName(pMsg, pInputName);
 	MsgSetRawMsg(pMsg, (char*)rsCStrGetSzStr(cstrLine), cstrLen(cstrLine));
 	MsgSetMSGoffs(pMsg, 0);	/* we do not have a header... */

the good news: it doesn't block anymore.
the bad news: the message order changes when the backlog is transferred to the logserver.

this is how i tested it using the latest 5.8.6 on 64bit with the same config as before:

the message generator generates messages with a timestamp and a counter:

# typeset -i i=0; while true; do echo "$(date) : $i"; echo "$(date) : $i" >> /var/log/test.log; echo
"$(date) : $i" | logger; i=$i+1; sleep 1; done

after re-enabling the network access to the syslog server the incoming messages have not the same
order as they were generated, this happens for both syslog and imfile messages. the '..' denotes
blocks with sequential counter values.

messages written by syslog():

2011-10-24T21:36:09.876045+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:08 CEST 2011 : 26
2011-10-24T21:36:09.876152+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:09 CEST 2011 : 27
<network blocked, re-enabled>
2011-10-24T21:36:19.876794+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:10 CEST 2011 : 28
2011-10-24T21:36:19.876825+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:11 CEST 2011 : 29
..
2011-10-24T21:38:58.194452+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:38:58 CEST 2011 : 190
2011-10-24T21:38:59.230003+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:38:59 CEST 2011 : 191
2011-10-24T21:44:31.663370+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:44:31 CEST 2011 : 512
2011-10-24T21:44:32.697829+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:44:32 CEST 2011 : 513
..
2011-10-24T21:47:41.191575+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:47:41 CEST 2011 : 695
2011-10-24T21:47:42.225602+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:47:42 CEST 2011 : 696
2011-10-24T21:39:00.266370+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:39:00 CEST 2011 : 192
2011-10-24T21:39:01.301951+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:39:01 CEST 2011 : 193
..
2011-10-24T21:44:29.592028+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:44:29 CEST 2011 : 510
2011-10-24T21:44:30.629308+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:44:30 CEST 2011 : 511
2011-10-24T21:47:45.341005+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:47:45 CEST 2011 : 699
2011-10-24T21:47:46.376808+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:47:46 CEST 2011 : 700
2011-10-24T21:47:47.412188+02:00 gold-centos5-x86_64 logger: Mon Oct 24 21:47:47 CEST 2011 : 701
<in sync with generator>

same for messages collected by imfile:

2011-10-24T21:36:09.876045+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:08 CEST 2011 : 26
2011-10-24T21:36:09.876152+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:09 CEST 2011 : 27
<network blocked, re-enabled>
2011-10-24T21:36:19.876794+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:10 CEST 2011 : 28
2011-10-24T21:36:19.876825+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:36:11 CEST 2011 : 29
..
2011-10-24T21:38:59.893033+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:38:58 CEST 2011 : 190
2011-10-24T21:38:59.893036+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:38:59 CEST 2011 : 191
2011-10-24T21:44:39.925036+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:44:30 CEST 2011 : 511
2011-10-24T21:44:39.925052+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:44:31 CEST 2011 : 512
..
2011-10-24T21:47:39.942124+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:47:38 CEST 2011 : 692
2011-10-24T21:47:39.942127+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:47:39 CEST 2011 : 693
2011-10-24T21:39:09.893706+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:39:00 CEST 2011 : 192
2011-10-24T21:39:09.893718+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:39:01 CEST 2011 : 193
..
2011-10-24T21:44:29.923811+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:44:28 CEST 2011 : 509
2011-10-24T21:44:29.923814+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:44:29 CEST 2011 : 510
2011-10-24T21:47:49.943314+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:47:40 CEST 2011 : 694
2011-10-24T21:47:49.943328+02:00 gold-centos5-x86_64 test: Mon Oct 24 21:47:41 CEST 2011 : 695
<in sync with generator>

doesn't rsyslog transfer the spool file first and then the new messages to preserve the order of the
messages?

i ran the same test with only syslog() writing messages and got a similar but worse result:

2011-10-24T22:16:19.968852+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:16:19 CEST 2011 : 29
2011-10-24T22:16:20.997981+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:16:20 CEST 2011 : 30
<network blocked, re-enabled>
2011-10-24T22:18:51.246842+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:18:51 CEST 2011 : 176
2011-10-24T22:18:52.274474+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:18:52 CEST 2011 : 177
..
2011-10-24T22:19:13.907656+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:19:13 CEST 2011 : 198
2011-10-24T22:19:14.939596+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:19:14 CEST 2011 : 199
2011-10-24T22:30:03.772399+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:30:03 CEST 2011 : 828
2011-10-24T22:30:04.793107+02:00 gold-centos5-x86_64 logger: Mon Oct 24 22:30:04 CEST 2011 : 829
<and so on>

the missing messages 31-175, almost 2min, disturb me the most.

any ideas what happened and why?

i will run some more tests with a disk queue instead of a DA queue.

regards,
-ap

From david at lang.hm  Tue Oct 25 07:17:26 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 24 Oct 2011 22:17:26 -0700 (PDT)
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <4EA5CD65.9090905@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>
	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>
	<4E9C6D4B.4000602@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>
	<4E9DC017.7070100@gmx.net> <4EA5CD65.9090905@gmx.net>
Message-ID: <alpine.DEB.2.02.1110242208420.9854@asgard.lang.hm>

On Mon, 24 Oct 2011, Andreas Piesk wrote:

> 
> On 18.10.2011 20:06, Andreas Piesk wrote:
>> On 18.10.2011 08:39, Rainer Gerhards wrote:
>>>
>>> Indeed and that is another story. Of course, that should not be blocked.
>>> Trying to dig deeper into the issue: what happens if you do not monitor the
>>> file but just use syslog() input. Does it then block?
>>>
>>
>> no, without any file monitors it does not block.
>> even if i have a file monitor configured but do not write into the monitored file is does not block.
>>
>
> the bad news: the message order changes when the backlog is transferred to the logserver.

the order of syslog messages is not maintained. Even in the simplest, most 
generic case it is possible for the network packets to pass one another 
between the source and the destination.

rsyslog used to put a lot more effort into maintaining the order of the 
logs, but it turns out that this effort was slowing things down 
significantly, and still couldn't provide the guarantee that it was 
assuming was needed.

As a result of that discussion, many new features have been implemented in 
rsyslog that have provided very significant speedups, but they also 
provide more ways that the logs can get out of order.

David Lang

From david at lang.hm  Tue Oct 25 07:23:49 2011
From: david at lang.hm (david at lang.hm)
Date: Mon, 24 Oct 2011 22:23:49 -0700 (PDT)
Subject: [rsyslog] Syslog message stripped chars at the front
In-Reply-To: <20111024145958.GB5353@pax.zz.de>
References: <20111024145958.GB5353@pax.zz.de>
Message-ID: <alpine.DEB.2.02.1110242223050.9854@asgard.lang.hm>

it looks to me like rsyslog is not recognizing the timestamp, probably due 
to the decimal seconds.

could you post the rawlog of the message?

David Lang
-------------- next part --------------

Hi,
i am seeing a strange problem where syslog messages received from remote
get stripped a couple of chars at the front. This seems to happen
depending on the source of the message:

Config:

$template DynFile,"/var/log/remote/%fromhost%-%timegenerated:1:10:date-rfc3339%"
$template hostorip,"%TIMESTAMP:::date-rfc3339% %fromhost% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
:source , !isequal , "localhost" ?DynFile
:source , !isequal , "localhost" /var/log/remote.log;hostorip

Source A (Redback SE600)

	14:51:25.364326 IP (tos 0x0, ttl 63, id 49237, offset 0, flags [none], proto UDP (17), length 107)
	    172.30.0.7.63458 > 172.30.16.16.514: SYSLOG, length: 79
		Facility local7 (23), Severity info (6)
		Msg: Oct 24 14:51:25.351: %CSM-6-PORT: ethernet 2/1 link state UP, admin is UP\0x0a

	2011-10-24T14:51:25.373047+00:00 frnk1-bras1 24 14:51:25.351: %CSM-6-PORT: ethernet 2/1 link state UP, admin is UP

As one can see the "Oct " is stripped of the beginning of the message. Logging "rawmsg" shows
the full message including 
A different source (Cisco ASR9k):

	14:51:34.487744 IP (tos 0x0, ttl 30, id 64430, offset 0, flags [none], proto UDP (17), length 189)
	    172.30.0.2.514 > 172.30.16.16.514: SYSLOG, length: 161
		Facility local7 (23), Severity info (6)
		Msg: 467: LC/0/0/CPU0:Oct 24 14:51:34.476 : bfd_agent[123]: %L2-BFD-6-SESSION_STATE_UP : BFD session to neighbor 172.30.16.42 on interface TenGigE0/0/0/6 is up \0x0a

	2011-10-24T14:51:34.487834+00:00 frnk1-cr2 467: LC/0/0/CPU0:Oct 24 14:51:34.476 : bfd_agent[123]: %L2-BFD-6-SESSION_STATE_UP : BFD session to neighbor 172.30.16.42 on interface TenGigE0/0/0/6 is up 

Nothing stripped. I dont see anything obvious ....

Flo
-- 
Florian Lohoff                                                 f at zz.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111024/090f5e56/attachment.pgp>
-------------- next part --------------
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From f at zz.de  Tue Oct 25 09:33:20 2011
From: f at zz.de (Florian Lohoff)
Date: Tue, 25 Oct 2011 09:33:20 +0200
Subject: [rsyslog] Syslog message stripped chars at the front
In-Reply-To: <alpine.DEB.2.02.1110242223050.9854@asgard.lang.hm>
References: <20111024145958.GB5353@pax.zz.de>
	<alpine.DEB.2.02.1110242223050.9854@asgard.lang.hm>
Message-ID: <20111025073320.GC21842@pax.zz.de>

On Mon, Oct 24, 2011 at 10:23:49PM -0700, david at lang.hm wrote:
> it looks to me like rsyslog is not recognizing the timestamp,
> probably due to the decimal seconds.
> 
> could you post the rawlog of the message?

Example message:

$template hostorip,"%TIMESTAMP:1:23:date-rfc3339% %fromhost% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n

2011-10-25T07:31:21.731 frnk1-bras1 25 07:31:21.742: [0001]: %LDP-6-INFO: ldp_rcv_notify: 172.30.0.2 rcv NOTIFICATION, 18 bytes, status data 0x9(Hold Timer Expired),fatal bit 1, forwar bit 0

$template raw,"%rawmsg%\n"

<190>Oct 25 07:31:21.742: [0001]: %LDP-6-INFO: ldp_rcv_notify: 172.30.0.2 rcv NOTIFICATION, 18 bytes, status data 0x9(Hold Timer Expired),fatal bit 1, forwar bit 0

Flo
-- 
Florian Lohoff                                                 f at zz.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111025/02d0ced0/attachment-0001.pgp>

From rgerhards at hq.adiscon.com  Tue Oct 25 14:59:58 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 25 Oct 2011 14:59:58 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
	remote destination is unresponsive
In-Reply-To: <4EA5CD65.9090905@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>	<4E9C6D4B.4000602@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com><4E9DC017.7070100@gmx.net>
	<4EA5CD65.9090905@gmx.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72813AA@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andreas Piesk
> Sent: Monday, October 24, 2011 10:41 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslog stops local logging and local logging
hangsif
> remote destination is unresponsive
> 
> On 18.10.2011 20:06, Andreas Piesk wrote:
> > On 18.10.2011 08:39, Rainer Gerhards wrote:
> >>
> >> Indeed and that is another story. Of course, that should not be blocked.
> >> Trying to dig deeper into the issue: what happens if you do not
> >> monitor the file but just use syslog() input. Does it then block?
> >>
> >
> > no, without any file monitors it does not block.
> > even if i have a file monitor configured but do not write into the
monitored
> file is does not block.
> >
> 
> any news on this issue?

Unfortunately, not, I did not yet find time to reproduce the issue.

Rainer

> 
> in the meantime i tested your idea of changing imfile's flow control type:
> 
> # cat rsyslog-filedelay.patch
> diff -Purp rsyslog-5.8.6/plugins/imfile/imfile.c rsyslog-
> 5.8.6.mod/plugins/imfile/imfile.c
> --- rsyslog-5.8.6/plugins/imfile/imfile.c	2011-10-21 11:53:02.000000000
+0200
> +++ rsyslog-5.8.6.mod/plugins/imfile/imfile.c	2011-10-24 21:05:30.000000000
> +0200
> @@ -113,7 +113,7 @@ static rsRetVal enqLine(fileInfo_t *pInf
>  	}
> 
>  	CHKiRet(msgConstruct(&pMsg));
> -	MsgSetFlowControlType(pMsg, eFLOWCTL_FULL_DELAY);
> +	MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY);
>  	MsgSetInputName(pMsg, pInputName);
>  	MsgSetRawMsg(pMsg, (char*)rsCStrGetSzStr(cstrLine),
> cstrLen(cstrLine));
>  	MsgSetMSGoffs(pMsg, 0);	/* we do not have a header... */
> 
> the good news: it doesn't block anymore.
> the bad news: the message order changes when the backlog is transferred to
> the logserver.
> 
> this is how i tested it using the latest 5.8.6 on 64bit with the same
config as
> before:
> 
> the message generator generates messages with a timestamp and a counter:
> 
> # typeset -i i=0; while true; do echo "$(date) : $i"; echo "$(date) : $i"
>>
> /var/log/test.log; echo
> "$(date) : $i" | logger; i=$i+1; sleep 1; done
> 
> after re-enabling the network access to the syslog server the incoming
> messages have not the same order as they were generated, this happens for
> both syslog and imfile messages. the '..' denotes blocks with sequential
> counter values.
> 
> messages written by syslog():
> 
> 2011-10-24T21:36:09.876045+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:08 CEST 2011 : 26
> 2011-10-24T21:36:09.876152+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:09 CEST 2011 : 27 <network blocked, re-enabled>
> 2011-10-24T21:36:19.876794+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:10 CEST 2011 : 28
> 2011-10-24T21:36:19.876825+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:11 CEST 2011 : 29 ..
> 2011-10-24T21:38:58.194452+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:38:58 CEST 2011 : 190
> 2011-10-24T21:38:59.230003+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:38:59 CEST 2011 : 191
> 2011-10-24T21:44:31.663370+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:44:31 CEST 2011 : 512
> 2011-10-24T21:44:32.697829+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:44:32 CEST 2011 : 513 ..
> 2011-10-24T21:47:41.191575+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:47:41 CEST 2011 : 695
> 2011-10-24T21:47:42.225602+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:47:42 CEST 2011 : 696
> 2011-10-24T21:39:00.266370+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:39:00 CEST 2011 : 192
> 2011-10-24T21:39:01.301951+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:39:01 CEST 2011 : 193 ..
> 2011-10-24T21:44:29.592028+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:44:29 CEST 2011 : 510
> 2011-10-24T21:44:30.629308+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:44:30 CEST 2011 : 511
> 2011-10-24T21:47:45.341005+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:47:45 CEST 2011 : 699
> 2011-10-24T21:47:46.376808+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:47:46 CEST 2011 : 700
> 2011-10-24T21:47:47.412188+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 21:47:47 CEST 2011 : 701 <in sync with generator>
> 
> same for messages collected by imfile:
> 
> 2011-10-24T21:36:09.876045+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:08 CEST 2011 : 26
> 2011-10-24T21:36:09.876152+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:09 CEST 2011 : 27 <network blocked, re-enabled>
> 2011-10-24T21:36:19.876794+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:10 CEST 2011 : 28
> 2011-10-24T21:36:19.876825+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:36:11 CEST 2011 : 29 ..
> 2011-10-24T21:38:59.893033+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:38:58 CEST 2011 : 190
> 2011-10-24T21:38:59.893036+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:38:59 CEST 2011 : 191
> 2011-10-24T21:44:39.925036+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:44:30 CEST 2011 : 511
> 2011-10-24T21:44:39.925052+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:44:31 CEST 2011 : 512 ..
> 2011-10-24T21:47:39.942124+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:47:38 CEST 2011 : 692
> 2011-10-24T21:47:39.942127+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:47:39 CEST 2011 : 693
> 2011-10-24T21:39:09.893706+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:39:00 CEST 2011 : 192
> 2011-10-24T21:39:09.893718+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:39:01 CEST 2011 : 193 ..
> 2011-10-24T21:44:29.923811+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:44:28 CEST 2011 : 509
> 2011-10-24T21:44:29.923814+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:44:29 CEST 2011 : 510
> 2011-10-24T21:47:49.943314+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:47:40 CEST 2011 : 694
> 2011-10-24T21:47:49.943328+02:00 gold-centos5-x86_64 test: Mon Oct 24
> 21:47:41 CEST 2011 : 695 <in sync with generator>
> 
> doesn't rsyslog transfer the spool file first and then the new messages to
> preserve the order of the messages?
> 
> i ran the same test with only syslog() writing messages and got a similar
but
> worse result:
> 
> 2011-10-24T22:16:19.968852+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:16:19 CEST 2011 : 29
> 2011-10-24T22:16:20.997981+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:16:20 CEST 2011 : 30 <network blocked, re-enabled>
> 2011-10-24T22:18:51.246842+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:18:51 CEST 2011 : 176
> 2011-10-24T22:18:52.274474+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:18:52 CEST 2011 : 177 ..
> 2011-10-24T22:19:13.907656+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:19:13 CEST 2011 : 198
> 2011-10-24T22:19:14.939596+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:19:14 CEST 2011 : 199
> 2011-10-24T22:30:03.772399+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:30:03 CEST 2011 : 828
> 2011-10-24T22:30:04.793107+02:00 gold-centos5-x86_64 logger: Mon Oct 24
> 22:30:04 CEST 2011 : 829 <and so on>
> 
> the missing messages 31-175, almost 2min, disturb me the most.
> 
> any ideas what happened and why?
> 
> i will run some more tests with a disk queue instead of a DA queue.
> 
> regards,
> -ap
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From a.piesk at gmx.net  Tue Oct 25 19:39:57 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Tue, 25 Oct 2011 19:39:57 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <alpine.DEB.2.02.1110242208420.9854@asgard.lang.hm>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>	<4E9C6D4B.4000602@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>	<4E9DC017.7070100@gmx.net>
	<4EA5CD65.9090905@gmx.net>
	<alpine.DEB.2.02.1110242208420.9854@asgard.lang.hm>
Message-ID: <4EA6F46D.9040500@gmx.net>

On 25.10.2011 07:17, david at lang.hm wrote:
> On Mon, 24 Oct 2011, Andreas Piesk wrote:
>>
>> the bad news: the message order changes when the backlog is transferred to the logserver.
> 
> the order of syslog messages is not maintained. Even in the simplest, most generic case it is
> possible for the network packets to pass one another between the source and the destination.
> 
> rsyslog used to put a lot more effort into maintaining the order of the logs, but it turns out that
> this effort was slowing things down significantly, and still couldn't provide the guarantee that it
> was assuming was needed.
> 
> As a result of that discussion, many new features have been implemented in rsyslog that have
> provided very significant speedups, but they also provide more ways that the logs can get out of order.
> 

thank you for the info.

I'm right assuming that using a disk queue would preserve the order (the possibility of packets
passing each other on network ignored) because all messages have to go though the queue? i know,
disk queues are not great for performance but speed is not so important for me.
the test with a disk queue is still on my list, but maybe someone can share first-hand experience.

regards,
-ap

From david at lang.hm  Tue Oct 25 19:50:04 2011
From: david at lang.hm (david at lang.hm)
Date: Tue, 25 Oct 2011 10:50:04 -0700 (PDT)
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <4EA6F46D.9040500@gmx.net>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>
	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>
	<4E9C6D4B.4000602@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com>
	<4E9DC017.7070100@gmx.net> <4EA5CD65.9090905@gmx.net>
	<alpine.DEB.2.02.1110242208420.9854@asgard.lang.hm>
	<4EA6F46D.9040500@gmx.net>
Message-ID: <alpine.DEB.2.02.1110251046140.6316@asgard.lang.hm>

On Tue, 25 Oct 2011, Andreas Piesk wrote:

> On 25.10.2011 07:17, david at lang.hm wrote:
>> On Mon, 24 Oct 2011, Andreas Piesk wrote:
>>>
>>> the bad news: the message order changes when the backlog is transferred to the logserver.
>>
>> the order of syslog messages is not maintained. Even in the simplest, most generic case it is
>> possible for the network packets to pass one another between the source and the destination.
>>
>> rsyslog used to put a lot more effort into maintaining the order of the logs, but it turns out that
>> this effort was slowing things down significantly, and still couldn't provide the guarantee that it
>> was assuming was needed.
>>
>> As a result of that discussion, many new features have been implemented in rsyslog that have
>> provided very significant speedups, but they also provide more ways that the logs can get out of order.
>>
>
> thank you for the info.
>
> I'm right assuming that using a disk queue would preserve the order (the possibility of packets
> passing each other on network ignored) because all messages have to go though the queue? i know,
> disk queues are not great for performance but speed is not so important for me.
> the test with a disk queue is still on my list, but maybe someone can share first-hand experience.

I think that adding disk queues increases the likelyhood of messages 
getting out of order.

the packets passing each other on the network was just an example of one 
way that they can get out of order, no matter how hard rsyslog tries to 
keep them in order. But since they can get out of order anyway, rsyslog 
stopped trying really hard to keep them in order.

for disk queues, I think that the memory part of the queue is serviced 
first, and only after it's drained do the older messages from the disk 
queue get sent. If I am correct, the logic behind this is that since the 
disk is so much slower than memory, it's better to process the memory ones 
first because if the disk ones were serviced first it's possible that you 
could not process them fast enough, and therefor new messages would need 
to get added to the disk queue, which sould slow processing down further, 
in sort of a death spiral.

David Lang

From a.piesk at gmx.net  Tue Oct 25 19:54:01 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Tue, 25 Oct 2011 19:54:01 +0200
Subject: [rsyslog] rsyslog stops local logging and local logging hangsif
 remote destination is unresponsive
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72813AA@GRFEXC.intern.adiscon.com>
References: <005b01cc8a9b$e92e3920$100013ac@intern.adiscon.com><4E987DF0.2070707@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA7281338@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281339@GRFEXC.intern.adiscon.com>	<4E9B27C9.1090006@gmx.net><9B6E2A8877C38245BFB15CC491A11DA728133D@GRFEXC.intern.adiscon.com>	<4E9C6D4B.4000602@gmx.net>	<9B6E2A8877C38245BFB15CC491A11DA728134D@GRFEXC.intern.adiscon.com><4E9DC017.7070100@gmx.net>	<4EA5CD65.9090905@gmx.net>
	<9B6E2A8877C38245BFB15CC491A11DA72813AA@GRFEXC.intern.adiscon.com>
Message-ID: <4EA6F7B9.9030200@gmx.net>

On 25.10.2011 14:59, Rainer Gerhards wrote:
> 
> Unfortunately, not, I did not yet find time to reproduce the issue.
> 

OK, does changing the flow control type of imfile has any negative side-effects besides the
increased storage requirements for the queue and the increased hard disk I/O?

i'm thinking of putting the patched version in production after adding a configuration parameter to
easily switch this "store-and-forward" feature on and off.

regards,
-ap

From nicol at selex-si-us.com  Thu Oct 27 22:29:36 2011
From: nicol at selex-si-us.com (David Nicol)
Date: Thu, 27 Oct 2011 20:29:36 +0000
Subject: [rsyslog] configuring behavior on disk-full conditions
Message-ID: <41622511EEE16441BC479723EAD7E0162ECE51A2@SELEX02.asii.local>

I'd like to trigger logrotate to run off its schedule in the unlikely event that rsyslog gets a DISK FULL error while appending to a log.

I haven't found any documentation of handling exceptions in the man page, documentation, or wiki.

Please advise?

From rgerhards at hq.adiscon.com  Fri Oct 28 07:16:35 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Oct 2011 07:16:35 +0200
Subject: [rsyslog] configuring behavior on disk-full conditions
In-Reply-To: <41622511EEE16441BC479723EAD7E0162ECE51A2@SELEX02.asii.local>
References: <41622511EEE16441BC479723EAD7E0162ECE51A2@SELEX02.asii.local>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72813DC@GRFEXC.intern.adiscon.com>

There is currently no way to handle that, but I would integrate a patch if
you intend to write it :)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of David Nicol
> Sent: Thursday, October 27, 2011 10:30 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] configuring behavior on disk-full conditions
> 
> I'd like to trigger logrotate to run off its schedule in the unlikely
> event that rsyslog gets a DISK FULL error while appending to a log.
> 
> I haven't found any documentation of handling exceptions in the man
> page, documentation, or wiki.
> 
> Please advise?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From mikehale at gmail.com  Fri Oct 28 07:25:52 2011
From: mikehale at gmail.com (Michael Hale)
Date: Fri, 28 Oct 2011 01:25:52 -0400
Subject: [rsyslog] outchannel blocking
Message-ID: <CAGF8HbpkGUB9+47X12iQw6LhA4jVoR4++HmyP1Hz1ZbnTyYkzA@mail.gmail.com>

Hi I am attempting to use outchannel to determisticly control the size
of my log files. However, when my log file reaches the limit rsyslog
simply blocks. Here is my configuration:

$outchannel o_messages, /var/log/messages, 3000, /usr/bin/savelog -pc
3 /var/log/messages
*.* $o_messages

I captured some debug output at the time when the file passes the
threshold with rsyslogd -c4 -dn, but I can't tell what is going on
other than my action is failing which I assume means there is some
kind of problem with the outchannel definition or the command... but
when I run the command outside rsyslog everything works fine. Any
ideas?

9172.503877007:b6682b70: Message from UNIX socket: #3
9172.503906379:b6682b70: main Q: entry added, size now log 1, phys 1 entries
9172.503921278:b6682b70: main Q: EnqueueMsg advised worker start
9172.503940494:b6682b70: --------imuxsock calling select, active file
descriptors (max 4): 3 4
9172.504014396:b7683b70: wti 0x9563758: worker awoke from idle processing
9172.504030616:b7683b70: we deleted 0 objects and enqueued 0 objects
9172.504038292:b7683b70: delete batch from store, new sizes: log 1, phys 1
9172.504058362:b7683b70: msg parser: flags 14, from 'ip-10-114-31-58',
msg '<13>Oct 28 05:19:32 logger: Fri Oct 28 05:19:32 UTC 2011'
9172.504067145:b7683b70: parse using parser list 0x955d808 (the default list).
9172.504075889:b7683b70: Parser 'rsyslog.rfc5424' returned -2160
9172.504084762:b7683b70: Message will now be parsed by the legacy
syslog parser (one size fits all... ;)).
9172.504093723:b7683b70: Parser 'rsyslog.rfc3164' returned 0
9172.504101318:b7683b70: ZZZ: processBatch: batch of 1 elements must
be processed
9172.504109167:b7683b70: Processing next rule
9172.504135121:b7683b70: result of expression evaluation: 0
9172.504144284:b7683b70: Processing next action
9172.504152949:b7683b70: Called action(NotAllMark), logging to builtin-file
9172.504160450:b7683b70: Called action(Batch), logging to builtin-file
9172.504172414:b7683b70: actionTryResume: action state: susp, next
retry (if applicable): 1319779178 [now 1319779172]
9172.504181078:b7683b70: actionTryResume: action state: susp, next
retry (if applicable): 1319779178 [now 1319779172]
9172.504188712:b7683b70: ruleset: get iRet 0 from rule.ProcessMsg()
9172.504196252:b7683b70: Processing next rule
9172.504221064:b7683b70: result of expression evaluation: 1
9172.504228755:b7683b70: Processing next action
9172.504235997:b7683b70: Called action(NotAllMark), logging to builtin-file
9172.504243333:b7683b70: Called action(Batch), logging to builtin-file
9172.504256393:b7683b70: Action 0x956a408 transitioned to state: itx
9172.504264293:b7683b70: entering actionCalldoAction(), state: itx
9172.504271861:b7683b70: file to log to: /var/log/messages
9172.504279522:b7683b70: write to stream, pData->pStrm 0x9569e80, lenBuf 87
9172.504286911:b7683b70: Action 0x956a408 transitioned to state: rtry
9172.504294143:b7683b70: action call returned -2007
9172.504301762:b7683b70: tryDoAction: unexpected error code
-2007[nElem 1, Commited UpTo 0], finalizing
9172.504309984:b7683b70: Action 0x956a408 transitioned to state: rdy
9172.504317073:b7683b70: Action 0x956a408 transitioned to state: itx
9172.504324084:b7683b70: entering actionCalldoAction(), state: itx
9172.504330966:b7683b70: file to log to: /var/log/messages
9172.504338463:b7683b70: write to stream, pData->pStrm 0x9569e80, lenBuf 87
9172.504345439:b7683b70: Action 0x956a408 transitioned to state: rtry
9172.504352265:b7683b70: action call returned -2007
9172.504359362:b7683b70: tryDoAction: unexpected error code
-2007[nElem 1, Commited UpTo 0], finalizing
9172.504366987:b7683b70: Action 0x956a408 transitioned to state: rdy
9172.504373995:b7683b70: Action 0x956a408 transitioned to state: itx
9172.504381016:b7683b70: entering actionCalldoAction(), state: itx
9172.504387832:b7683b70: file to log to: /var/log/messages

From teifler at adiscon.com  Fri Oct 28 15:13:12 2011
From: teifler at adiscon.com (Tim Eifler)
Date: Fri, 28 Oct 2011 15:13:12 +0200
Subject: [rsyslog] outchannel blocking
In-Reply-To: <CAGF8HbpkGUB9+47X12iQw6LhA4jVoR4++HmyP1Hz1ZbnTyYkzA@mail.gmail.com>
References: <CAGF8HbpkGUB9+47X12iQw6LhA4jVoR4++HmyP1Hz1ZbnTyYkzA@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72813E4@GRFEXC.intern.adiscon.com>

Hi, take a look on SELinux. Because SELinux can block Rsyslog. So rsyslog has
not enough rights to open another program.
Can you please send the complete debug log. Because the debug log not contain
the main problem.
Another problem could be that the user account have not enough rights. 

-----Urspr?ngliche Nachricht-----
Von: rsyslog-bounces at lists.adiscon.com
[mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Michael Hale
Gesendet: Freitag, 28. Oktober 2011 07:26
An: rsyslog at lists.adiscon.com
Betreff: [rsyslog] outchannel blocking

Hi I am attempting to use outchannel to determisticly control the size
of my log files. However, when my log file reaches the limit rsyslog
simply blocks. Here is my configuration:

$outchannel o_messages, /var/log/messages, 3000, /usr/bin/savelog -pc
3 /var/log/messages
*.* $o_messages

I captured some debug output at the time when the file passes the
threshold with rsyslogd -c4 -dn, but I can't tell what is going on
other than my action is failing which I assume means there is some
kind of problem with the outchannel definition or the command... but
when I run the command outside rsyslog everything works fine. Any
ideas?

9172.503877007:b6682b70: Message from UNIX socket: #3
9172.503906379:b6682b70: main Q: entry added, size now log 1, phys 1 entries
9172.503921278:b6682b70: main Q: EnqueueMsg advised worker start
9172.503940494:b6682b70: --------imuxsock calling select, active file
descriptors (max 4): 3 4
9172.504014396:b7683b70: wti 0x9563758: worker awoke from idle processing
9172.504030616:b7683b70: we deleted 0 objects and enqueued 0 objects
9172.504038292:b7683b70: delete batch from store, new sizes: log 1, phys 1
9172.504058362:b7683b70: msg parser: flags 14, from 'ip-10-114-31-58',
msg '<13>Oct 28 05:19:32 logger: Fri Oct 28 05:19:32 UTC 2011'
9172.504067145:b7683b70: parse using parser list 0x955d808 (the default
list).
9172.504075889:b7683b70: Parser 'rsyslog.rfc5424' returned -2160
9172.504084762:b7683b70: Message will now be parsed by the legacy
syslog parser (one size fits all... ;)).
9172.504093723:b7683b70: Parser 'rsyslog.rfc3164' returned 0
9172.504101318:b7683b70: ZZZ: processBatch: batch of 1 elements must
be processed
9172.504109167:b7683b70: Processing next rule
9172.504135121:b7683b70: result of expression evaluation: 0
9172.504144284:b7683b70: Processing next action
9172.504152949:b7683b70: Called action(NotAllMark), logging to builtin-file
9172.504160450:b7683b70: Called action(Batch), logging to builtin-file
9172.504172414:b7683b70: actionTryResume: action state: susp, next
retry (if applicable): 1319779178 [now 1319779172]
9172.504181078:b7683b70: actionTryResume: action state: susp, next
retry (if applicable): 1319779178 [now 1319779172]
9172.504188712:b7683b70: ruleset: get iRet 0 from rule.ProcessMsg()
9172.504196252:b7683b70: Processing next rule
9172.504221064:b7683b70: result of expression evaluation: 1
9172.504228755:b7683b70: Processing next action
9172.504235997:b7683b70: Called action(NotAllMark), logging to builtin-file
9172.504243333:b7683b70: Called action(Batch), logging to builtin-file
9172.504256393:b7683b70: Action 0x956a408 transitioned to state: itx
9172.504264293:b7683b70: entering actionCalldoAction(), state: itx
9172.504271861:b7683b70: file to log to: /var/log/messages
9172.504279522:b7683b70: write to stream, pData->pStrm 0x9569e80, lenBuf 87
9172.504286911:b7683b70: Action 0x956a408 transitioned to state: rtry
9172.504294143:b7683b70: action call returned -2007
9172.504301762:b7683b70: tryDoAction: unexpected error code
-2007[nElem 1, Commited UpTo 0], finalizing
9172.504309984:b7683b70: Action 0x956a408 transitioned to state: rdy
9172.504317073:b7683b70: Action 0x956a408 transitioned to state: itx
9172.504324084:b7683b70: entering actionCalldoAction(), state: itx
9172.504330966:b7683b70: file to log to: /var/log/messages
9172.504338463:b7683b70: write to stream, pData->pStrm 0x9569e80, lenBuf 87
9172.504345439:b7683b70: Action 0x956a408 transitioned to state: rtry
9172.504352265:b7683b70: action call returned -2007
9172.504359362:b7683b70: tryDoAction: unexpected error code
-2007[nElem 1, Commited UpTo 0], finalizing
9172.504366987:b7683b70: Action 0x956a408 transitioned to state: rdy
9172.504373995:b7683b70: Action 0x956a408 transitioned to state: itx
9172.504381016:b7683b70: entering actionCalldoAction(), state: itx
9172.504387832:b7683b70: file to log to: /var/log/messages
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com

From mikehale at gmail.com  Fri Oct 28 18:07:08 2011
From: mikehale at gmail.com (Michael Hale)
Date: Fri, 28 Oct 2011 12:07:08 -0400
Subject: [rsyslog] outchannel blocking
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72813E4@GRFEXC.intern.adiscon.com>
References: <CAGF8HbpkGUB9+47X12iQw6LhA4jVoR4++HmyP1Hz1ZbnTyYkzA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA72813E4@GRFEXC.intern.adiscon.com>
Message-ID: <CAGF8Hbo22CmBSB07Ez02cZZ28hqTE2XyWx7B1g16Cd979q1Mrg@mail.gmail.com>

I'm running on Ubuntu 10.04 LTS without SELinux and rsyslog-5.6.5.
Just for grins I disabled PrivDropToUser and PrivDropToGroup so
rsyslog is running as root, but that configuration is still unable to
rotate the logfiles. I'll try updating to the latest stable: 5.8.6 and
see if that helps.


On Fri, Oct 28, 2011 at 9:13 AM, Tim Eifler <teifler at adiscon.com> wrote:
> Hi, take a look on SELinux. Because SELinux can block Rsyslog. So rsyslog has
> not enough rights to open another program.
> Can you please send the complete debug log. Because the debug log not contain
> the main problem.
> Another problem could be that the user account have not enough rights.
>
> -----Urspr?ngliche Nachricht-----
> Von: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-bounces at lists.adiscon.com] Im Auftrag von Michael Hale
> Gesendet: Freitag, 28. Oktober 2011 07:26
> An: rsyslog at lists.adiscon.com
> Betreff: [rsyslog] outchannel blocking
>
> Hi I am attempting to use outchannel to determisticly control the size
> of my log files. However, when my log file reaches the limit rsyslog
> simply blocks. Here is my configuration:
>
> $outchannel o_messages, /var/log/messages, 3000, /usr/bin/savelog -pc
> 3 /var/log/messages
> *.* $o_messages
>
> I captured some debug output at the time when the file passes the
> threshold with rsyslogd -c4 -dn, but I can't tell what is going on
> other than my action is failing which I assume means there is some
> kind of problem with the outchannel definition or the command... but
> when I run the command outside rsyslog everything works fine. Any
> ideas?
>
> 9172.503877007:b6682b70: Message from UNIX socket: #3
> 9172.503906379:b6682b70: main Q: entry added, size now log 1, phys 1 entries
> 9172.503921278:b6682b70: main Q: EnqueueMsg advised worker start
> 9172.503940494:b6682b70: --------imuxsock calling select, active file
> descriptors (max 4): 3 4
> 9172.504014396:b7683b70: wti 0x9563758: worker awoke from idle processing
> 9172.504030616:b7683b70: we deleted 0 objects and enqueued 0 objects
> 9172.504038292:b7683b70: delete batch from store, new sizes: log 1, phys 1
> 9172.504058362:b7683b70: msg parser: flags 14, from 'ip-10-114-31-58',
> msg '<13>Oct 28 05:19:32 logger: Fri Oct 28 05:19:32 UTC 2011'
> 9172.504067145:b7683b70: parse using parser list 0x955d808 (the default
> list).
> 9172.504075889:b7683b70: Parser 'rsyslog.rfc5424' returned -2160
> 9172.504084762:b7683b70: Message will now be parsed by the legacy
> syslog parser (one size fits all... ;)).
> 9172.504093723:b7683b70: Parser 'rsyslog.rfc3164' returned 0
> 9172.504101318:b7683b70: ZZZ: processBatch: batch of 1 elements must
> be processed
> 9172.504109167:b7683b70: Processing next rule
> 9172.504135121:b7683b70: result of expression evaluation: 0
> 9172.504144284:b7683b70: Processing next action
> 9172.504152949:b7683b70: Called action(NotAllMark), logging to builtin-file
> 9172.504160450:b7683b70: Called action(Batch), logging to builtin-file
> 9172.504172414:b7683b70: actionTryResume: action state: susp, next
> retry (if applicable): 1319779178 [now 1319779172]
> 9172.504181078:b7683b70: actionTryResume: action state: susp, next
> retry (if applicable): 1319779178 [now 1319779172]
> 9172.504188712:b7683b70: ruleset: get iRet 0 from rule.ProcessMsg()
> 9172.504196252:b7683b70: Processing next rule
> 9172.504221064:b7683b70: result of expression evaluation: 1
> 9172.504228755:b7683b70: Processing next action
> 9172.504235997:b7683b70: Called action(NotAllMark), logging to builtin-file
> 9172.504243333:b7683b70: Called action(Batch), logging to builtin-file
> 9172.504256393:b7683b70: Action 0x956a408 transitioned to state: itx
> 9172.504264293:b7683b70: entering actionCalldoAction(), state: itx
> 9172.504271861:b7683b70: file to log to: /var/log/messages
> 9172.504279522:b7683b70: write to stream, pData->pStrm 0x9569e80, lenBuf 87
> 9172.504286911:b7683b70: Action 0x956a408 transitioned to state: rtry
> 9172.504294143:b7683b70: action call returned -2007
> 9172.504301762:b7683b70: tryDoAction: unexpected error code
> -2007[nElem 1, Commited UpTo 0], finalizing
> 9172.504309984:b7683b70: Action 0x956a408 transitioned to state: rdy
> 9172.504317073:b7683b70: Action 0x956a408 transitioned to state: itx
> 9172.504324084:b7683b70: entering actionCalldoAction(), state: itx
> 9172.504330966:b7683b70: file to log to: /var/log/messages
> 9172.504338463:b7683b70: write to stream, pData->pStrm 0x9569e80, lenBuf 87
> 9172.504345439:b7683b70: Action 0x956a408 transitioned to state: rtry
> 9172.504352265:b7683b70: action call returned -2007
> 9172.504359362:b7683b70: tryDoAction: unexpected error code
> -2007[nElem 1, Commited UpTo 0], finalizing
> 9172.504366987:b7683b70: Action 0x956a408 transitioned to state: rdy
> 9172.504373995:b7683b70: Action 0x956a408 transitioned to state: itx
> 9172.504381016:b7683b70: entering actionCalldoAction(), state: itx
> 9172.504387832:b7683b70: file to log to: /var/log/messages
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From gkra at unnerving.org  Fri Oct 28 21:42:57 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Fri, 28 Oct 2011 12:42:57 -0700
Subject: [rsyslog] configuring behavior on disk-full conditions
In-Reply-To: <41622511EEE16441BC479723EAD7E0162ECE51A2@SELEX02.asii.local>
References: <41622511EEE16441BC479723EAD7E0162ECE51A2@SELEX02.asii.local>
Message-ID: <743B544A-F3E6-426C-857A-5E19ABD68D87@unnerving.org>

On Oct 27, 2011, at 1:29 PM, David Nicol wrote:

> I'd like to trigger logrotate to run off its schedule in the unlikely event that rsyslog gets a DISK FULL error while appending to a log.
> 
> I haven't found any documentation of handling exceptions in the man page, documentation, or wiki.


Thinking outside of rsyslog, you *could* write a script to use via omprog, and then set up a filter so that any "DISK FULL" error gets logged to that script. Then, if the script ever receives input via the omprog action, it could simply trigger logrotate to do its thing.

No patching to rsyslog required.

Heck, you don't even need to use omprog, you could use the traditional pipe action:

:msg, contains, "DISK FULL" |/usr/local/bin/kick_logrotate

Obviously, you'd need to tune the rule to the actual error message rsyslog emits if it's not "DISK FULL", but that'll get you most of the way there.

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu




From gkra at unnerving.org  Fri Oct 28 21:45:30 2011
From: gkra at unnerving.org (Gregory K. Ruiz-Ade)
Date: Fri, 28 Oct 2011 12:45:30 -0700
Subject: [rsyslog] outchannel blocking
In-Reply-To: <CAGF8Hbo22CmBSB07Ez02cZZ28hqTE2XyWx7B1g16Cd979q1Mrg@mail.gmail.com>
References: <CAGF8HbpkGUB9+47X12iQw6LhA4jVoR4++HmyP1Hz1ZbnTyYkzA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA72813E4@GRFEXC.intern.adiscon.com>
	<CAGF8Hbo22CmBSB07Ez02cZZ28hqTE2XyWx7B1g16Cd979q1Mrg@mail.gmail.com>
Message-ID: <1F2DF9E6-4537-4191-AE38-B54C09B023EF@unnerving.org>

On Oct 28, 2011, at 9:07 AM, Michael Hale wrote:

> I'm running on Ubuntu 10.04 LTS without SELinux and rsyslog-5.6.5.
> Just for grins I disabled PrivDropToUser and PrivDropToGroup so
> rsyslog is running as root, but that configuration is still unable to
> rotate the logfiles. I'll try updating to the latest stable: 5.8.6 and
> see if that helps.


For Ubuntu, the thing to look at is AppArmor. There are some reasonably good docs on Ubuntu's site and the Ubuntu Community site, otherwise start digging through /etc/apparmor.d for anything related to syslog or rsyslog.

Gregory

-- 
Gregory K. Ruiz-Ade <gkra at unnerving.org>
OpenPGP Key ID: EAF4844B  keyserver: pgpkeys.mit.edu




From mikehale at gmail.com  Fri Oct 28 23:45:48 2011
From: mikehale at gmail.com (Michael Hale)
Date: Fri, 28 Oct 2011 17:45:48 -0400
Subject: [rsyslog] outchannel blocking
In-Reply-To: <1F2DF9E6-4537-4191-AE38-B54C09B023EF@unnerving.org>
References: <CAGF8HbpkGUB9+47X12iQw6LhA4jVoR4++HmyP1Hz1ZbnTyYkzA@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA72813E4@GRFEXC.intern.adiscon.com>
	<CAGF8Hbo22CmBSB07Ez02cZZ28hqTE2XyWx7B1g16Cd979q1Mrg@mail.gmail.com>
	<1F2DF9E6-4537-4191-AE38-B54C09B023EF@unnerving.org>
Message-ID: <CAGF8Hbr6CQw8m3+TknK3a4Pk9idiRE-SbeL+FuKL-oRyEZDv-A@mail.gmail.com>

It seems that rsyslog was not handling the params to savelog as I
would have expected.

/usr/bin/savelog -pc 3 /var/log/messages ended up being
/usr/bin/savelog '-pc 3 /var/log/messages', which is not a file that
exists. Also /var/log needed to be owned by the PrivDropToUser.

FWIW
  strace -fp rsyslogd -c5 -dn 2>&1 |tee rsyslogd.debug
wait for the file to not rotate, then kill rsyslogd and look for errors:
  grep -B 1 ENO rsyslogd.debug

Was very helpful in finding the problem.

On Fri, Oct 28, 2011 at 3:45 PM, Gregory K. Ruiz-Ade <gkra at unnerving.org> wrote:
> On Oct 28, 2011, at 9:07 AM, Michael Hale wrote:
>
>> I'm running on Ubuntu 10.04 LTS without SELinux and rsyslog-5.6.5.
>> Just for grins I disabled PrivDropToUser and PrivDropToGroup so
>> rsyslog is running as root, but that configuration is still unable to
>> rotate the logfiles. I'll try updating to the latest stable: 5.8.6 and
>> see if that helps.
>
>
> For Ubuntu, the thing to look at is AppArmor. There are some reasonably good docs on Ubuntu's site and the Ubuntu Community site, otherwise start digging through /etc/apparmor.d for anything related to syslog or rsyslog.
>
> Gregory
>
> --
> Gregory K. Ruiz-Ade <gkra at unnerving.org>
> OpenPGP Key ID: EAF4844B ?keyserver: pgpkeys.mit.edu
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>

From a.piesk at gmx.net  Sun Oct 30 22:02:47 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Sun, 30 Oct 2011 22:02:47 +0100
Subject: [rsyslog] rsyslog hangs at startup
Message-ID: <4EADBB77.5040801@gmx.net>

Hello,

i noticed that rsyslog sometimes hangs at startup.

i'm using rsyslog 5.8.6 64bit on RHEL5 and the attached config.

i think, it happens when the network is stopped before rsyslogd and rsyslog spooled messages to
disk, but i'm not sure.

i managed to get an debug log and rsyslogd hangs at

6822.575991000:42648940: entering actionCalldoAction(), state: itx
6822.576008000:42648940:  10.10.0.254
6822.576039000:42648940: caller requested object 'nsd_ptcp', not found (iRet -3003)
6822.576063000:42648940: Requested to load module 'lmnsd_ptcp'
<hang>

i've attached the complete debug log.

the requested module is where all the other modules (which have been loaded without problems) are:

# ls -l /lib64/rsyslog/
total 572
-rwxr-xr-x 1 root root  15368 Oct 24 21:08 imfile.so
-rwxr-xr-x 1 root root  27152 Oct 24 21:08 imklog.so
-rwxr-xr-x 1 root root   6392 Oct 24 21:08 immark.so
-rwxr-xr-x 1 root root  11000 Oct 24 21:08 imtcp.so
-rwxr-xr-x 1 root root  15136 Oct 24 21:08 imudp.so
-rwxr-xr-x 1 root root 328200 Oct 24 21:08 imuxsock.so
-rwxr-xr-x 1 root root  22808 Oct 24 21:08 lmnet.so
-rwxr-xr-x 1 root root  15960 Oct 24 21:08 lmnetstrms.so
-rwxr-xr-x 1 root root  24248 Oct 24 21:08 lmnsd_ptcp.so
-rwxr-xr-x 1 root root   6056 Oct 24 21:08 lmregexp.so
-rwxr-xr-x 1 root root  20240 Oct 24 21:08 lmstrmsrv.so
-rwxr-xr-x 1 root root  10152 Oct 24 21:08 lmtcpclt.so
-rwxr-xr-x 1 root root  24768 Oct 24 21:08 lmtcpsrv.so
-rwxr-xr-x 1 root root   6056 Oct 24 21:08 lmzlibw.so
-rwxr-xr-x 1 root root  10312 Oct 24 21:08 omruleset.so
-rwxr-xr-x 1 root root  10152 Oct 24 21:08 omtesting.so

any idea what's going wrong?

regards,
-ap
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: forwarder.conf
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111030/531b8edc/attachment.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pickup.conf
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111030/531b8edc/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: rsyslog.conf
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111030/531b8edc/attachment-0001.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: rsyslog_debug.log
URL: <http://lists.adiscon.net/pipermail/rsyslog/attachments/20111030/531b8edc/attachment-0001.txt>

From rodney.mckee at gmail.com  Mon Oct 31 06:27:28 2011
From: rodney.mckee at gmail.com (Rodney McKee)
Date: Mon, 31 Oct 2011 16:27:28 +1100 (EST)
Subject: [rsyslog] remote systems not re-connecting after extender outage of
	receiver
In-Reply-To: <3f0c34f9-b3e7-41bc-aa82-e9ef2edf2027@wsrmckee>
Message-ID: <58c74783-b412-4a81-96f3-30e4824fd96e@wsrmckee>

Currently noticed an issue with several remote servers not re-establishing the transfer of logs after we had an extended outage on the receiving server during maintenance. I did restart rsyslog on one remote and the exported queue value dropped to (3) but the files still remain. 

Any suggestion on what action is needed to get the logs transferring again without loss or how I can get some data for debugging the issue. 

From rgerhards at hq.adiscon.com  Mon Oct 31 10:06:08 2011
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 31 Oct 2011 10:06:08 +0100
Subject: [rsyslog] remote systems not re-connecting after extender
	outage ofreceiver
In-Reply-To: <58c74783-b412-4a81-96f3-30e4824fd96e@wsrmckee>
References: <3f0c34f9-b3e7-41bc-aa82-e9ef2edf2027@wsrmckee>
	<58c74783-b412-4a81-96f3-30e4824fd96e@wsrmckee>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72813EF@GRFEXC.intern.adiscon.com>

It's important to know that the longer the outage, the less frequent rsyslog
will retry (to save ressources). So you may simply have become impatient. It
is also important to run current builds, as a couple of fixes were done in
this area. If that does not help, we need a debug log to see what actually
happens.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rodney McKee
> Sent: Monday, October 31, 2011 6:27 AM
> To: rsyslog-users
> Subject: [rsyslog] remote systems not re-connecting after extender
> outage ofreceiver
> 
> Currently noticed an issue with several remote servers not re-
> establishing the transfer of logs after we had an extended outage on
> the receiving server during maintenance. I did restart rsyslog on one
> remote and the exported queue value dropped to (3) but the files still
> remain.
> 
> Any suggestion on what action is needed to get the logs transferring
> again without loss or how I can get some data for debugging the issue.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

From a.piesk at gmx.net  Mon Oct 31 14:09:04 2011
From: a.piesk at gmx.net (Andreas Piesk)
Date: Mon, 31 Oct 2011 14:09:04 +0100
Subject: [rsyslog] rsyslog hangs at startup
In-Reply-To: <4EADBB77.5040801@gmx.net>
References: <4EADBB77.5040801@gmx.net>
Message-ID: <4EAE9DF0.8080703@gmx.net>

On 30.10.2011 22:02, Andreas Piesk wrote:
> 
> any idea what's going wrong?
> 

after looking at runtime/modules.c i suspected that the mutex doesn't get unlocked.

after inserting some more debug printf:

2426.965520000:2ad3d4b29320: cfline: '$ModLoad imfile'
2426.965545000:2ad3d4b29320: Requested to load module 'imfile'
2426.965548000:2ad3d4b29320: setting module load/unlock lock
2426.965551000:2ad3d4b29320: module load/unload lock set
2426.965555000:2ad3d4b29320: loading module '/lib64/rsyslog/imfile.so'
2426.965591000:419d5940: strm 0x2ad3e9475220: file 8 read 537 bytes
2426.965733000:419d5940: MsgSetTAG in: len 9, pszBuf: rsyslogd:
2426.965740000:419d5940: MsgSetTAG exit: pMsg->iLenTAG 9, pMsg->TAG.szBuf: rsyslogd:
2426.965776000:419d5940: XXXXX:  tryDoAction 0x2ad3e94705a0, pnElem 1, nElem 1
2426.965780000:419d5940: Action 0x2ad3e94705a0 transitioned to state: itx
2426.965783000:419d5940: entering actionCalldoAction(), state: itx
2426.965797000:419d5940:  10.10.0.254
2426.965818000:419d5940: caller requested object 'nsd_ptcp', not found (iRet -3003)
2426.965843000:419d5940: Requested to load module 'lmnsd_ptcp'
2426.965846000:419d5940: setting module load/unlock lock

i don't see "imfile: version %s initializing\n", so the problem seems to be imfile because it
doesn't get initialized and blocks the mutex.

regards,
-ap