From mikel at irontec.com  Mon Nov  3 14:14:50 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Mon, 03 Nov 2008 14:14:50 +0100
Subject: [rsyslog] filter expression dude
Message-ID: <490EF94A.70704@irontec.com>

hello

Is this correct?

if $msg contains 'GUI_set' and not $msg contains 'VALIDATOR' then 
@@192.1.100.1


I want that if message contains GUI_set and no VALIDATOR logs remotelly 
in 192.1.100.1 via TCP


Thanks


From hks.private at gmail.com  Mon Nov  3 16:51:01 2008
From: hks.private at gmail.com ((private) HKS)
Date: Mon, 3 Nov 2008 10:51:01 -0500
Subject: [rsyslog] filter expression dude
In-Reply-To: <490EF94A.70704@irontec.com>
References: <490EF94A.70704@irontec.com>
Message-ID: <ac9ded160811030751v7220f0eas69a25b5faa926da7@mail.gmail.com>

Yes, that should work as expected.

-HKS

On Mon, Nov 3, 2008 at 8:14 AM, Mikel Jimenez <mikel at irontec.com> wrote:
> hello
>
> Is this correct?
>
> if $msg contains 'GUI_set' and not $msg contains 'VALIDATOR' then
> @@192.1.100.1
>
>
> I want that if message contains GUI_set and no VALIDATOR logs remotelly
> in 192.1.100.1 via TCP
>
>
> Thanks
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Wed Nov  5 17:46:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 5 Nov 2008 17:46:22 +0100
Subject: [rsyslog] IPv6 problem (was:  new v3-stable release candidate)
In-Reply-To: <alpine.DEB.1.10.0810281023570.635@asgard.lang.hm>
References: <577465F99B41C842AAFBE9ED71E70ABA44F4BB@grfint2.intern.adiscon.com>
	<alpine.DEB.1.10.0810281023570.635@asgard.lang.hm>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F555@grfint2.intern.adiscon.com>

David,

I had some time to set up a new test environment today and try to
reproduce the problem. Unfortunately, I still did not see it. If you
have any new information, that would be appreciated.

I would also appreciate if someone else could try rsyslog under the
conditions told by David.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, October 28, 2008 6:26 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] new v3-stable release candidate
> 
> On Tue, 28 Oct 2008, Rainer Gerhards wrote:
> 
> > Hi all,
> >
> > I am about to switch the v3-stable release to 3.20, which means it
> will
> > be based on the 3.19.x beta branch. I have created a test tarball,
> > available at:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-3.20.0.tar.gz
> >
> > I would appreciate if some could try it out and report any potential
> > issues they see. I intend to release this tomorrow or Thursday if
> > nothing bad happens.
> 
> one problem I have been seeing in the nextmaster branch is that with
> imudp
> on a box that has IPv6 in the kernel, but not IPv6 IP addresses on the
> box
> rsyslog goes into a loop (eating 100% cpu, but not doing anything)
when
> it
> recieves a IPv4 syslog packet.
> 
> Rainer has not been able to duplicate the problem, could someone else
> test
> this?
> 
> I've been seeing it on Debian amd64 boxes.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From friedl at hq.adiscon.com  Thu Nov  6 16:47:53 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Thu, 6 Nov 2008 16:47:53 +0100
Subject: [rsyslog] rsyslog 3.20.0 (stable) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F566@grfint2.intern.adiscon.com>

Hi all,

we have released rsyslog 3.20.0, the next iteration of the rsyslog
v3-stable branch. Most importantly, TLS support is now available in the
stable branches. This version includes all features of the current beta
plus a bugfix. For users of the v3-stable branch, it brings many feature
enhancements, too many to list all of them here. This is a recommended
update for all v3-stable branch users.

Changelog:

http://www.rsyslog.com/Article302.phtml

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-137.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .



From rgerhards at hq.adiscon.com  Thu Nov  6 17:53:27 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 6 Nov 2008 17:53:27 +0100
Subject: [rsyslog] rsyslog regular expressen checker/generator
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F56C@grfint2.intern.adiscon.com>

Hi all,

regular expressions are quite powerful, but the syntax in rsyslog is,
well, not easy to use. Also, as we have seen, the usual regex check
tools don't work always well with rsyslog's POSIX expressions. I have
created a web-based checker/generator today. It is more or less
finished, but of course needs fine-tuning. I would appreciate if some of
you could have a quick look and comment with whatever suggestion they
have. The URL is:

http://www.rsyslog.com/tool-regex

If this tool turns out to be useful (judging on comments and access
count over the next weeks), I will probably do some other online tools
aiding in config generation.

Thanks,
Rainer


From rgerhards at hq.adiscon.com  Tue Nov 11 17:41:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Nov 2008 17:41:44 +0100
Subject: [rsyslog] rsyslog 3.21.7 released, new beta
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F5E0@grfint2.intern.adiscon.com>

Hi all,

I have just released rsyslog 3.21.7, the new beta branch. This version
brings all features of the current development to the beta branch. It
also contains one new addition, that is the ability to replace a not
found regular expression match with a zero. This can be useful for
storing numerical values into databases. This is a recommended update
for all beta branch users. 

Please note that a new devel branch will follow shortly. Currently,
devel is *older* than the beta.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-138.phtml

Change Log:

http://www.rsyslog.com/Article305.phtml

As always, feedback is appreciated.

Best regards,
Rainer Gerhards


From patrick.shen at net-m.de  Wed Nov 12 11:06:03 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Wed, 12 Nov 2008 11:06:03 +0100
Subject: [rsyslog] FQDN with rsyslogd
Message-ID: <491AAA8B.5010805@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear Rainer,

Have a look at online man doc, maybe it's outdated. I find one thing I
really care about.

######################################################################
If the remote host is located in the same domain  as  the  host,
rsyslogd  is running on, only the simple hostname will be logged instead
of the whole fqdn.
######################################################################

Currently we are using rsyslog as client to send logs to central
loghost, actually in the same domain. But I wanna need FQDN in logs, not
simple hostname.

How need I do?

Many thanks,

- --
Patrick Shen
Operations Engineer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJGqqLkHhYtFevC+MRAoMlAJ9M1CeXQf27KTz0/GznkreUFYcb7QCeKjl6
8R3DzRHtpDZU+a0/B1XQny4=
=RUGG
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Wed Nov 12 11:51:53 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Nov 2008 11:51:53 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491AAA8B.5010805@net-m.de>
References: <491AAA8B.5010805@net-m.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>

Hi Patrick,

this is part of the sysklogd legacy code. While there is no longer much
of sysklogd in rsyslog, this part actually is. I need to figure out if
you can turn it off, I remember to have added such an option.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> Sent: Wednesday, November 12, 2008 11:06 AM
> To: rsyslog-users
> Subject: [rsyslog] FQDN with rsyslogd
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dear Rainer,
> 
> Have a look at online man doc, maybe it's outdated. I find one thing I
> really care about.
> 
> ######################################################################
> If the remote host is located in the same domain  as  the  host,
> rsyslogd  is running on, only the simple hostname will be logged
> instead
> of the whole fqdn.
> ######################################################################
> 
> Currently we are using rsyslog as client to send logs to central
> loghost, actually in the same domain. But I wanna need FQDN in logs,
> not
> simple hostname.
> 
> How need I do?
> 
> Many thanks,
> 
> - --
> Patrick Shen
> Operations Engineer
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJGqqLkHhYtFevC+MRAoMlAJ9M1CeXQf27KTz0/GznkreUFYcb7QCeKjl6
> 8R3DzRHtpDZU+a0/B1XQny4=
> =RUGG
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From patrick.shen at net-m.de  Thu Nov 13 03:11:36 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Thu, 13 Nov 2008 03:11:36 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
References: <491AAA8B.5010805@net-m.de>
	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
Message-ID: <491B8CD8.3020701@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Rainer,

I will appreciate if you could show me the magic code.

Best regards,
Patrick

Rainer Gerhards wrote:
> Hi Patrick,
> 
> this is part of the sysklogd legacy code. While there is no longer much
> of sysklogd in rsyslog, this part actually is. I need to figure out if
> you can turn it off, I remember to have added such an option.
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
>> Sent: Wednesday, November 12, 2008 11:06 AM
>> To: rsyslog-users
>> Subject: [rsyslog] FQDN with rsyslogd
>>
> Dear Rainer,
> 
> Have a look at online man doc, maybe it's outdated. I find one thing I
> really care about.
> 
> ######################################################################
> If the remote host is located in the same domain  as  the  host,
> rsyslogd  is running on, only the simple hostname will be logged
> instead
> of the whole fqdn.
> ######################################################################
> 
> Currently we are using rsyslog as client to send logs to central
> loghost, actually in the same domain. But I wanna need FQDN in logs,
> not
> simple hostname.
> 
> How need I do?
> 
> Many thanks,
> 
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJG4zYkHhYtFevC+MRAhtxAKCFQABa96nbmKFTtrhNN1scqz769gCeLI+J
MHPb4zA20/cupkxuCL6uqQ4=
=TyV0
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Thu Nov 13 09:56:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Nov 2008 09:56:22 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491B8CD8.3020701@net-m.de>
References: <491AAA8B.5010805@net-m.de>
	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
	<491B8CD8.3020701@net-m.de>
Message-ID: <1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>

Hi Patrick,

the code in question is this:

http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008aac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171

doesn't look hard to add a global option to prevent it, I'll see when I
can do it. In the meantime, you can simply comment out the strip of the
local domain (line 1174 I guess, but you need to check if it works).

Rainer

On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Rainer,
> 
> I will appreciate if you could show me the magic code.
> 
> Best regards,
> Patrick
> 
> Rainer Gerhards wrote:
> > Hi Patrick,
> > 
> > this is part of the sysklogd legacy code. While there is no longer much
> > of sysklogd in rsyslog, this part actually is. I need to figure out if
> > you can turn it off, I remember to have added such an option.
> > 
> > Rainer
> > 
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> >> Sent: Wednesday, November 12, 2008 11:06 AM
> >> To: rsyslog-users
> >> Subject: [rsyslog] FQDN with rsyslogd
> >>
> > Dear Rainer,
> > 
> > Have a look at online man doc, maybe it's outdated. I find one thing I
> > really care about.
> > 
> > ######################################################################
> > If the remote host is located in the same domain  as  the  host,
> > rsyslogd  is running on, only the simple hostname will be logged
> > instead
> > of the whole fqdn.
> > ######################################################################
> > 
> > Currently we are using rsyslog as client to send logs to central
> > loghost, actually in the same domain. But I wanna need FQDN in logs,
> > not
> > simple hostname.
> > 
> > How need I do?
> > 
> > Many thanks,
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJG4zYkHhYtFevC+MRAhtxAKCFQABa96nbmKFTtrhNN1scqz769gCeLI+J
> MHPb4zA20/cupkxuCL6uqQ4=
> =TyV0
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From mikel at irontec.com  Thu Nov 13 12:37:55 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 13 Nov 2008 12:37:55 +0100
Subject: [rsyslog] filter expression dude
Message-ID: <491C1193.3060609@irontec.com>

Hello
I usually use startswitch 'xxxxx' but is anything similar to "endswith" ?
Or regex that makes "endwith" function?

Would be appreciated an example please

Thanks!!

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Thu Nov 13 13:55:55 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Nov 2008 13:55:55 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C1193.3060609@irontec.com>
References: <491C1193.3060609@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>

You can use 'xxxxx$' as a regular expression. For the property replacer,
I have recently written a tool where you can try this out:

http://www.rsyslog.com/tool-regex

Note, though, that the syntax for conditions is different. But you get
the idea...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Thursday, November 13, 2008 12:38 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] filter expression dude
> 
> Hello
> I usually use startswitch 'xxxxx' but is anything similar to
"endswith"
> ?
> Or regex that makes "endwith" function?
> 
> Would be appreciated an example please
> 
> Thanks!!
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mikel at irontec.com  Thu Nov 13 14:00:05 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 13 Nov 2008 14:00:05 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
References: <491C1193.3060609@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
Message-ID: <491C24D5.4000607@irontec.com>

Rainer Gerhards escribi?:
> You can use 'xxxxx$' as a regular expression. For the property replacer,
> I have recently written a tool where you can try this out:
>
> http://www.rsyslog.com/tool-regex
>
> Note, though, that the syntax for conditions is different. But you get
> the idea...
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Thursday, November 13, 2008 12:38 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] filter expression dude
>>
>> Hello
>> I usually use startswitch 'xxxxx' but is anything similar to
>>     
> "endswith"
>   
>> ?
>> Or regex that makes "endwith" function?
>>
>> Would be appreciated an example please
>>
>> Thanks!!
>>
>> --
>> Mikel Jimenez Fernandez
>> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
>> +34 94.404.81.82
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
ok!
And in full example?
I have this
if $FROMHOST startswith 'deusto' then 
:ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
& ~

And I want al "fromhost" that ends with 'sto'. I dont understand how to 
do this with regex tool...
For example:
esto
ppesto
rtisto

Can you write me these practical example?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From r.bhatia at ipax.at  Thu Nov 13 13:45:34 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Thu, 13 Nov 2008 13:45:34 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C1193.3060609@irontec.com>
References: <491C1193.3060609@irontec.com>
Message-ID: <491C216E.4020101@ipax.at>

Mikel Jimenez wrote:
> Hello
> I usually use startswitch 'xxxxx' but is anything similar to "endswith" ?
> Or regex that makes "endwith" function?
> 
> Would be appreciated an example please

thou i do not know about the actual implementation in rsyslog,
you can match an end-of-line with: "$".

e.g.
> raoul $ echo "my test"|grep test$
> my test
> raoul $ echo "my test2"|grep test$
> raoul $

maybe [1] might be of use.

cheers,
raoul
[1] http://www.addedbytes.com/cheat-sheets/regular-expressions-cheat-sheet/
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From hks.private at gmail.com  Thu Nov 13 15:17:58 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 13 Nov 2008 09:17:58 -0500
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C24D5.4000607@irontec.com>
References: <491C1193.3060609@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
	<491C24D5.4000607@irontec.com>
Message-ID: <ac9ded160811130617s4926431ewc177f8137b52299@mail.gmail.com>

> ok!
> And in full example?
> I have this
> if $FROMHOST startswith 'deusto' then
> :ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
> & ~
>
> And I want al "fromhost" that ends with 'sto'. I dont understand how to
> do this with regex tool...
> For example:
> esto
> ppesto
> rtisto
>
> Can you write me these practical example?
>


:fromhost, regex, "sto$"
:ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
& ~

-HKS


From patrick.shen at net-m.de  Fri Nov 14 03:48:42 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Fri, 14 Nov 2008 03:48:42 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
References: <491AAA8B.5010805@net-m.de>	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>	<491B8CD8.3020701@net-m.de>
	<1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
Message-ID: <491CE70A.8050108@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Rainer,

It works. Thank you very much. But I also find an issue:

###################################################################
2008-11-14T03:32:47.137221+01:00 hydra-t kernel: imklog 3.18.4, log
source = /proc/kmsg started.
2008-11-14T03:32:47.137273+01:00 hydra-t rsyslogd: [origin
software="rsyslogd" swVersion="3.18.4" x-pid="651"
x-info="http://www.rsyslog.com"] restart
...
2008-11-14T03:34:51.680382+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,679 [INFO] [main] [:] [] [] [] []
[] [] [] [] mbg.servlet.BillingGateway:
/opt/as/APP/mbg/WEB-INF/conf/MBGBillingProviderConfig.xml loaded
2008-11-14T03:34:51.785187+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,784 [INFO] [main] [:] [] [] [] []
[] [] [] [] base.db.MBGCounterManager: Starting MBGCounterManager
[dataSourceName:jdbc/dataSource/factory]
2008-11-14T03:34:51.785724+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,785 [INFO] [main] [:] [] [] [] []
[] [] [] [] mbg.servlet.BillingGateway: Initialization finish. Start
waiting for request ...
...
2008-11-14T03:37:15.555764+01:00 hydra-t snmpd[1699]: Connection from
UDP: [192.168.4.15]:34136
2008-11-14T03:37:15.555829+01:00 hydra-t snmpd[1699]: Received SNMP
packet(s) from UDP: [192.168.4.15]:34136
2008-11-14T03:37:16.357732+01:00 hydra-t snmpd[1699]: Connection from
UDP: [192.168.4.15]:34136
#####################################################################

We use "Log4j" to produce application(MBG) logs to hydra-t.test.xxx.xxx
(localhost) via TCP. Every line of them is tagged with
hydra-t.test.xxx.xxx. That's working after you show me the code.

But some other logs still use shortname of FQDN (hydra-t), like rsyslog
itself and snmpd.

Is it relative to $template in rsyslog config file? We use "$template
RSYSLOG_TraditionalForwardFormat" currently.

Many thanks,
Patrick


Rainer Gerhards wrote:
> Hi Patrick,
> 
> the code in question is this:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008aac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171
> 
> doesn't look hard to add a global option to prevent it, I'll see when I
> can do it. In the meantime, you can simply comment out the strip of the
> local domain (line 1174 I guess, but you need to check if it works).
> 
> Rainer
> 
> On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> Hi Rainer,
> 
> I will appreciate if you could show me the magic code.
> 
> Best regards,
> Patrick
> 
> Rainer Gerhards wrote:
>>>> Hi Patrick,
>>>>
>>>> this is part of the sysklogd legacy code. While there is no longer much
>>>> of sysklogd in rsyslog, this part actually is. I need to figure out if
>>>> you can turn it off, I remember to have added such an option.
>>>>
>>>> Rainer
>>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
>>>>> Sent: Wednesday, November 12, 2008 11:06 AM
>>>>> To: rsyslog-users
>>>>> Subject: [rsyslog] FQDN with rsyslogd
>>>>>
>>>> Dear Rainer,
>>>>
>>>> Have a look at online man doc, maybe it's outdated. I find one thing I
>>>> really care about.
>>>>
>>>> ######################################################################
>>>> If the remote host is located in the same domain  as  the  host,
>>>> rsyslogd  is running on, only the simple hostname will be logged
>>>> instead
>>>> of the whole fqdn.
>>>> ######################################################################
>>>>
>>>> Currently we are using rsyslog as client to send logs to central
>>>> loghost, actually in the same domain. But I wanna need FQDN in logs,
>>>> not
>>>> simple hostname.
>>>>
>>>> How need I do?
>>>>
>>>> Many thanks,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJHOcKkHhYtFevC+MRAjafAJ9aSXu91K9Hj6xj+tGs0SNWjk/vswCgiE9C
c6P4BTbHAnCa8+/hcw/MkGc=
=GeS4
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Fri Nov 14 08:24:21 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Nov 2008 08:24:21 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491CE70A.8050108@net-m.de>
References: <491AAA8B.5010805@net-m.de>	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>	<491B8CD8.3020701@net-m.de><1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
	<491CE70A.8050108@net-m.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F621@grfint2.intern.adiscon.com>

Hi Patrick,

I think it would be good if you file a feature request in bugzilla. I
will try to have a look, but these messages (or more precisely
hostnames) stem back to some places where the hostname is generated. I
need to review the code more in-depth to see if there is one ultimate
place and, if not, if there is a reason for multiple places or if they
could be restructured to a single code module (very likely!).

There are many other things in the queue, but I'll check if that's
something easy to do to shuffle in between...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> Sent: Friday, November 14, 2008 3:49 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] FQDN with rsyslogd
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Rainer,
> 
> It works. Thank you very much. But I also find an issue:
> 
> ###################################################################
> 2008-11-14T03:32:47.137221+01:00 hydra-t kernel: imklog 3.18.4, log
> source = /proc/kmsg started.
> 2008-11-14T03:32:47.137273+01:00 hydra-t rsyslogd: [origin
> software="rsyslogd" swVersion="3.18.4" x-pid="651"
> x-info="http://www.rsyslog.com"] restart
> ...
> 2008-11-14T03:34:51.680382+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,679 [INFO] [main] [:] [] [] [] []
> [] [] [] [] mbg.servlet.BillingGateway:
> /opt/as/APP/mbg/WEB-INF/conf/MBGBillingProviderConfig.xml loaded
> 2008-11-14T03:34:51.785187+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,784 [INFO] [main] [:] [] [] [] []
> [] [] [] [] base.db.MBGCounterManager: Starting MBGCounterManager
> [dataSourceName:jdbc/dataSource/factory]
> 2008-11-14T03:34:51.785724+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,785 [INFO] [main] [:] [] [] [] []
> [] [] [] [] mbg.servlet.BillingGateway: Initialization finish. Start
> waiting for request ...
> ...
> 2008-11-14T03:37:15.555764+01:00 hydra-t snmpd[1699]: Connection from
> UDP: [192.168.4.15]:34136
> 2008-11-14T03:37:15.555829+01:00 hydra-t snmpd[1699]: Received SNMP
> packet(s) from UDP: [192.168.4.15]:34136
> 2008-11-14T03:37:16.357732+01:00 hydra-t snmpd[1699]: Connection from
> UDP: [192.168.4.15]:34136
> #####################################################################
> 
> We use "Log4j" to produce application(MBG) logs to
hydra-t.test.xxx.xxx
> (localhost) via TCP. Every line of them is tagged with
> hydra-t.test.xxx.xxx. That's working after you show me the code.
> 
> But some other logs still use shortname of FQDN (hydra-t), like
rsyslog
> itself and snmpd.
> 
> Is it relative to $template in rsyslog config file? We use "$template
> RSYSLOG_TraditionalForwardFormat" currently.
> 
> Many thanks,
> Patrick
> 
> 
> Rainer Gerhards wrote:
> > Hi Patrick,
> >
> > the code in question is this:
> >
> >
>
http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008a
> ac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171
> >
> > doesn't look hard to add a global option to prevent it, I'll see
when
> I
> > can do it. In the meantime, you can simply comment out the strip of
> the
> > local domain (line 1174 I guess, but you need to check if it works).
> >
> > Rainer
> >
> > On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> > Hi Rainer,
> >
> > I will appreciate if you could show me the magic code.
> >
> > Best regards,
> > Patrick
> >
> > Rainer Gerhards wrote:
> >>>> Hi Patrick,
> >>>>
> >>>> this is part of the sysklogd legacy code. While there is no
longer
> much
> >>>> of sysklogd in rsyslog, this part actually is. I need to figure
> out if
> >>>> you can turn it off, I remember to have added such an option.
> >>>>
> >>>> Rainer
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>>>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> >>>>> Sent: Wednesday, November 12, 2008 11:06 AM
> >>>>> To: rsyslog-users
> >>>>> Subject: [rsyslog] FQDN with rsyslogd
> >>>>>
> >>>> Dear Rainer,
> >>>>
> >>>> Have a look at online man doc, maybe it's outdated. I find one
> thing I
> >>>> really care about.
> >>>>
> >>>>
> ######################################################################
> >>>> If the remote host is located in the same domain  as  the  host,
> >>>> rsyslogd  is running on, only the simple hostname will be logged
> >>>> instead
> >>>> of the whole fqdn.
> >>>>
> ######################################################################
> >>>>
> >>>> Currently we are using rsyslog as client to send logs to central
> >>>> loghost, actually in the same domain. But I wanna need FQDN in
> logs,
> >>>> not
> >>>> simple hostname.
> >>>>
> >>>> How need I do?
> >>>>
> >>>> Many thanks,
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJHOcKkHhYtFevC+MRAjafAJ9aSXu91K9Hj6xj+tGs0SNWjk/vswCgiE9C
> c6P4BTbHAnCa8+/hcw/MkGc=
> =GeS4
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Nov 18 10:41:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 10:41:35 +0100
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>

Hi all,

I have been asked if it is possible that the property replacer provides
an option of emitting a Unix timestamp for timestamp fields. This sounds
reasonable, but there are a number of subtleties. They are outlined in
the forum thread, with this post:

http://kb.monitorware.com/post14667.html#p14667

being probably a good wrap-up of the problems.

I would appreciate if someone from the mailing list could point me to a
resource, or share an idea, or whatever, on how I may tackle the
problem. Note that the primary concern is that I need to generate the
correct timestamp for messages from different time zones (problems
outlined in mentioned post).

Any feedback is highly welcome.

Thanks,
Rainer


From denemici at libero.it  Tue Nov 18 10:57:04 2008
From: denemici at libero.it (denemici at libero.it)
Date: Tue, 18 Nov 2008 10:57:04 +0100 (CET)
Subject: [rsyslog] undefined symbol: mysql_init
Message-ID: <19844959.335851227002224564.JavaMail.defaultUser@defaultHost>

Hi all,
I have upgrade Rsyslog from the 1.13.1 version (perfectly working with 
mysql) to the 3.20.0.
I have installed the new version from the source package, 
i have followed this step,
1) ./configure --enable-mysql
2) make
3) make 
install

Installation went fine but when i try to run rsyslog i have this error


Could not load module '/usr/local/lib/rsyslog/ommysql.so', dlopen: 
/usr/local/lib/rsyslog/ommysql.so: undefined symbol: mysql_init

If useful this 
a piece of configure phase:
.....
checking mysql/mysql.h usability... yes

checking mysql/mysql.h presence... yes
checking for mysql/mysql.h... yes

checking for mysql_config... yes
checking for mysql_init in -lmysqlclient... 
yes
...
Rsyslog is installed on Red Hat Enterprise Linux AS U4

Linux hostname 
2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64 
GNU/Linux

I try to find this error on the forum but i haven't found nothing :(


Thanks for any help.

Bye
denemici


From mikel at irontec.com  Tue Nov 18 14:10:20 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Tue, 18 Nov 2008 14:10:20 +0100
Subject: [rsyslog] milliseconds timestamp
Message-ID: <4922BEBC.3050606@irontec.com>

Hello
I am using phplogcon viewer and I have loosed milliseconds timestamps.

I watched that "DATE" in mysql databases is date-time type.

Does rsyslog gives the date in milliseconds? is possible to see this 
milliseconds din phplogcon?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From hks.private at gmail.com  Tue Nov 18 15:46:32 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 09:46:32 -0500
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>

On Tue, Nov 18, 2008 at 4:41 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Hi all,
>
> I have been asked if it is possible that the property replacer provides
> an option of emitting a Unix timestamp for timestamp fields. This sounds
> reasonable, but there are a number of subtleties. They are outlined in
> the forum thread, with this post:
>
> http://kb.monitorware.com/post14667.html#p14667
>
> being probably a good wrap-up of the problems.
>
> I would appreciate if someone from the mailing list could point me to a
> resource, or share an idea, or whatever, on how I may tackle the
> problem. Note that the primary concern is that I need to generate the
> correct timestamp for messages from different time zones (problems
> outlined in mentioned post).


Perhaps I'm overlooking a subtlety of the calculation, but why not
convert the time with respect to the host's locale, then apply the the
offset in seconds?

On the subject of leap seconds: epoch time has no such concept. This
is one of the (many) reasons it's crap and should be deserted in favor
of proper ISO timestamps or something similar, but it at least makes
your job a bit easier. To epoch time, every day is 86400 seconds, no
matter what.

-HKS


From rgerhards at hq.adiscon.com  Tue Nov 18 15:49:02 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 15:49:02 +0100
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>

> Perhaps I'm overlooking a subtlety of the calculation, but why not
> convert the time with respect to the host's locale, then apply the the
> offset in seconds?

Lol, there is a saying in German "you don't see the forest because of
the trees" which means you don't see the obvious thing, because it is so
obvious you are overlooking it. This sounds like a perfect case. Of
course, this solution is quite simple. Lol, at least I am glad I asked
;)

Thanks for hinting me.

Rainer


From hks.private at gmail.com  Tue Nov 18 15:51:29 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 09:51:29 -0500
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160811180651l72d85273if6ebea5c979e0c7b@mail.gmail.com>

Glad to help.

-HKS


On Tue, Nov 18, 2008 at 9:49 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
>> Perhaps I'm overlooking a subtlety of the calculation, but why not
>> convert the time with respect to the host's locale, then apply the the
>> offset in seconds?
>
> Lol, there is a saying in German "you don't see the forest because of
> the trees" which means you don't see the obvious thing, because it is so
> obvious you are overlooking it. This sounds like a perfect case. Of
> course, this solution is quite simple. Lol, at least I am glad I asked
> ;)
>
> Thanks for hinting me.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Tue Nov 18 16:38:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 16:38:56 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <4922BEBC.3050606@irontec.com>
References: <4922BEBC.3050606@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>

Hi Mikel,

I have talked to Andre, phpLogCon's lead developer. He says MySQL does
not support millisecond resolution in timestamps. So a work-around would
be to store them to another field and add this as a second field to the
phpLogCon view. I think it would probably smart to enable phpLogCon to
combine two such fields, but I leave this to Andre...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Tuesday, November 18, 2008 2:10 PM
> To: rsyslog-users
> Subject: [rsyslog] milliseconds timestamp
> 
> Hello
> I am using phplogcon viewer and I have loosed milliseconds timestamps.
> 
> I watched that "DATE" in mysql databases is date-time type.
> 
> Does rsyslog gives the date in milliseconds? is possible to see this
> milliseconds din phplogcon?
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From alorbach at ro1.adiscon.com  Tue Nov 18 17:56:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 18 Nov 2008 17:56:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>

I think it wouldn't be so difficult to implement, if the milliseconds
were simply stored in a second field.
We can add custom new fields into phpLogCon very easily. 

best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Dienstag, 18. November 2008 16:39
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> Hi Mikel,
> 
> I have talked to Andre, phpLogCon's lead developer. He says MySQL does
> not support millisecond resolution in timestamps. So a work-around
> would
> be to store them to another field and add this as a second field to
the
> phpLogCon view. I think it would probably smart to enable phpLogCon to
> combine two such fields, but I leave this to Andre...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > Sent: Tuesday, November 18, 2008 2:10 PM
> > To: rsyslog-users
> > Subject: [rsyslog] milliseconds timestamp
> >
> > Hello
> > I am using phplogcon viewer and I have loosed milliseconds
> timestamps.
> >
> > I watched that "DATE" in mysql databases is date-time type.
> >
> > Does rsyslog gives the date in milliseconds? is possible to see this
> > milliseconds din phplogcon?
> >
> > --
> > Mikel Jimenez Fernandez
> > Irontec, Internet y Sistemas sobre GNU/LinuX -
http://www.irontec.com
> > +34 94.404.81.82
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Nov 18 18:00:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 18:00:04 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>

Andre and all,

the rsyslog engine has the necessary plumbing and I know of at least one
user who stores subsecond timestamps in this way. As far as rsyslog is
concerned, this just requires a special timestamp. So if you would
support that in phpLogCon (what I would find a good idea ;)), I would
include a standard template into rsyslog (hardcoded). That would make it
easier for people to work with it. But maybe we get along by just
properly documenting it, because the schema must be modified in any
case.

Another alternative would be to update the default schema, but I am not
sure it that is actually a good idea. On the other hand, high-precision
timestamps hopefully get into more widespread use...

Thoughts from the community are appreciated.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Tuesday, November 18, 2008 5:56 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> I think it wouldn't be so difficult to implement, if the milliseconds
> were simply stored in a second field.
> We can add custom new fields into phpLogCon very easily.
> 
> best regards,
> Andre Lorbach
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Dienstag, 18. November 2008 16:39
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > Hi Mikel,
> >
> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
> does
> > not support millisecond resolution in timestamps. So a work-around
> > would
> > be to store them to another field and add this as a second field to
> the
> > phpLogCon view. I think it would probably smart to enable phpLogCon
> to
> > combine two such fields, but I leave this to Andre...
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > To: rsyslog-users
> > > Subject: [rsyslog] milliseconds timestamp
> > >
> > > Hello
> > > I am using phplogcon viewer and I have loosed milliseconds
> > timestamps.
> > >
> > > I watched that "DATE" in mysql databases is date-time type.
> > >
> > > Does rsyslog gives the date in milliseconds? is possible to see
> this
> > > milliseconds din phplogcon?
> > >
> > > --
> > > Mikel Jimenez Fernandez
> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> http://www.irontec.com
> > > +34 94.404.81.82
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From friedl at hq.adiscon.com  Tue Nov 18 18:03:16 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Tue, 18 Nov 2008 18:03:16 +0100
Subject: [rsyslog] rsyslog 4.1.0 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F681@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 4.1.0 today. This is the initial development
branch version of the upcoming v4 series. The main theme is performance
enhancement, 4.1.0 contains many enhancements which make the engine
process many more events per second than the previous versions. This was
made possible thanks to large changes inside the rsyslog core, and
consequently there is ample room for bugs. So this version should be
used with care. Note that it also contains some additional bugfixes and
small feature enhancements which are not found in other versions.

Please report any issue you see when working with that version. We are
especially interested in any anomaly on IPv6 enabled systems without
IPv6 addresses. There is a problem report for such an environment, which
we could not yet reproduce.

In case you wonder: there is no explicit schedule for the first
v4-stable version. But expect it no earlier than February 2009.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-139.phtml

Changelog:

http://www.rsyslog.com/Article308.phtml

Feedback is very appreciated for this new release.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .



From aoz.syn at gmail.com  Tue Nov 18 18:30:11 2008
From: aoz.syn at gmail.com (RB)
Date: Tue, 18 Nov 2008 10:30:11 -0700
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
Message-ID: <4255c2570811180930m77862fddu984f045127d7116f@mail.gmail.com>

> On the subject of leap seconds: epoch time has no such concept. This
> is one of the (many) reasons it's crap and should be deserted in favor
> of proper ISO timestamps or something similar, but it at least makes
> your job a bit easier. To epoch time, every day is 86400 seconds, no
> matter what.

I think your statement is mildly misleading.  As I understand it,
POSIX.1 epoch (as opposed to TAI) is ambiguous when it comes to leap
seconds, since certain timestamps can be interpreted as two separate
ISO timestamps - i.e. some values are repeated.  Therefore, POSIX.1
epochs most certainly account for leap seconds, but make conversions
of certain stamps difficult.

Regardless, you're right - ISO stamps (or TAI with a leap-second
table) is the way to go when your timestamps must be absolutely
precise across leap boundaries.


From mikel at irontec.com  Tue Nov 18 21:06:04 2008
From: mikel at irontec.com (mikel)
Date: Tue, 18 Nov 2008 21:06:04 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
Message-ID: <a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>


Dear Andre

So, in near future will be possible?

Or in far future? 

Thanx

On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
<rgerhards at hq.adiscon.com> wrote:
> Andre and all,
> 
> the rsyslog engine has the necessary plumbing and I know of at least one
> user who stores subsecond timestamps in this way. As far as rsyslog is
> concerned, this just requires a special timestamp. So if you would
> support that in phpLogCon (what I would find a good idea ;)), I would
> include a standard template into rsyslog (hardcoded). That would make it
> easier for people to work with it. But maybe we get along by just
> properly documenting it, because the schema must be modified in any
> case.
> 
> Another alternative would be to update the default schema, but I am not
> sure it that is actually a good idea. On the other hand, high-precision
> timestamps hopefully get into more widespread use...
> 
> Thoughts from the community are appreciated.
> 
> Thanks,
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>> Sent: Tuesday, November 18, 2008 5:56 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] milliseconds timestamp
>>
>> I think it wouldn't be so difficult to implement, if the milliseconds
>> were simply stored in a second field.
>> We can add custom new fields into phpLogCon very easily.
>>
>> best regards,
>> Andre Lorbach
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> > Sent: Dienstag, 18. November 2008 16:39
>> > To: rsyslog-users
>> > Subject: Re: [rsyslog] milliseconds timestamp
>> >
>> > Hi Mikel,
>> >
>> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
>> does
>> > not support millisecond resolution in timestamps. So a work-around
>> > would
>> > be to store them to another field and add this as a second field to
>> the
>> > phpLogCon view. I think it would probably smart to enable phpLogCon
>> to
>> > combine two such fields, but I leave this to Andre...
>> >
>> > Rainer
>> >
>> > > -----Original Message-----
>> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> > > Sent: Tuesday, November 18, 2008 2:10 PM
>> > > To: rsyslog-users
>> > > Subject: [rsyslog] milliseconds timestamp
>> > >
>> > > Hello
>> > > I am using phplogcon viewer and I have loosed milliseconds
>> > timestamps.
>> > >
>> > > I watched that "DATE" in mysql databases is date-time type.
>> > >
>> > > Does rsyslog gives the date in milliseconds? is possible to see
>> this
>> > > milliseconds din phplogcon?
>> > >
>> > > --
>> > > Mikel Jimenez Fernandez
>> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
>> http://www.irontec.com
>> > > +34 94.404.81.82
>> > >
>> > >
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > http://www.rsyslog.com
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From rsyslog at clark-communications.com  Tue Nov 18 21:23:51 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 12:23:51 -0800
Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
Message-ID: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>


Hello,

I have motivated the creation of a port of rsyslog to OpenBSD.
The port is intended to fully support the OpenBSD ports framework, and  
will build an OpenBSD binary package
suitable for installation via pkg_add.  See the message below for a  
copy of the port.

I have done some testing of this port on OpenBSD 4.4 (stable) under  
amd64.

I would very much appreciate any additional testing and/or comments  
from the rsyslog community.

Best regards,

Don


Begin forwarded message:
> From: Don Jackson <don.jackson at gmail.com>
> Date: November 18, 2008 11:07:45 AM PST
> To: ports at openbsd.org
> Subject: NEW: sysutils/rsyslog-3.20.0
>
>
> Tested on 4.4/amd64, definitely interested in more testing.
>
> $ cat ./pkg/DESCR
>
> A syslogd replacement
>
>
>
>
>
>
>


From rsyslog at clark-communications.com  Tue Nov 18 21:57:02 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 12:57:02 -0800
Subject: [rsyslog] rsyslogd security questions/comments
Message-ID: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>


Hello,

I was reading the man page for rsyslogd today, and saw:

      SECURITY THREATS

      There  is the potential for the rsyslogd daemon to be used
      as a conduit for a denial of service attack.  A rogue pro-
      gram(mer) could very easily flood the rsyslogd daemon with
      syslog messages resulting in the log files  consuming  all
      the remaining space on the filesystem.  Activating logging
      over the inet domain sockets will of course expose a  sys-
      tem  to  risks  outside  of programs or individuals on the
      local machine.

      There are a number of methods of protecting a machine:

      1.     Implement kernel firewalling to limit  which  hosts
             or networks have access to the 514/UDP socket.

      2.     Logging  can be directed to an isolated or non-root
             filesystem which, if filled, will  not  impair  the
             machine.

      3.     The  ext2  filesystem can be used which can be con-
             figured to limit a certain percentage of a filesys-
             tem  to  usage  by  root only.  NOTE that this will
             require rsyslogd to be run as a  non-root  process.
             ALSO  NOTE  that  this will prevent usage of remote
             logging on the default port since rsyslogd will  be
             unable to bind to the 514/UDP socket.

I had the following questions:

Would it be possible (optionally) to have rsyslogd chroot to a  
particular directory on startup?

That seems the safest.  One could configure a disk partition for log  
messages, configure rsyslogd to log there,
and also chroot to a directory on that partition, so if the rsyslogd  
process itself is compromised,
it can't do other damage.

There must be a way to have a daemon run as a non-root user, and also  
to open ports < 1024.
This seems to be done all the time on *bsd machines:

# ps -aux
USER       PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED       TIME  
COMMAND
root         1  0.0  0.0   428   356 ??  Is    Thu02PM    0:00.01 / 
sbin/init
_dhcp    22078  0.0  0.0   396   432 ??  Is    Thu03PM    0:00.01  
dhclient: bge0 (dhclient)
_syslogd 27943  0.0  0.0   452   812 ??  S     Thu03PM    0:00.19  
syslogd -a /var/empty/dev/log

I'm not sure how this is done, but it looks like chroot also supports  
changing the userid...

Just some thoughts,

Best regards,

Don









From david at lang.hm  Tue Nov 18 22:10:13 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 13:10:13 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
Message-ID: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, Don Jackson wrote:

> I had the following questions:
>
> Would it be possible (optionally) to have rsyslogd chroot to a
> particular directory on startup?
> That seems the safest.  One could configure a disk partition for log
> messages, configure rsyslogd to log there,

chroot doesn't help. if you have rsyslog set to log to a seperate 
partition it can only fill that partition, but it can fill _all of that 
partition even if you chroot into a subdirectory on that partition.

> and also chroot to a directory on that partition, so if the rsyslogd
> process itself is compromised,
> it can't do other damage.

that gives you protection if you are receiving logs from the network, but 
if you are receiving logs from /dev/log (local logs) you can't go into a 
chroot effectivly

> There must be a way to have a daemon run as a non-root user, and also
> to open ports < 1024.
> This seems to be done all the time on *bsd machines:

the POSIX standard still calls for ports < 1024 to require root to bind to 
them, different systems have different ways to be non-compliant with the 
standard and let you do so anyway. what OS are you using?

David Lang

> # ps -aux
> USER       PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED       TIME
> COMMAND
> root         1  0.0  0.0   428   356 ??  Is    Thu02PM    0:00.01 /
> sbin/init
> _dhcp    22078  0.0  0.0   396   432 ??  Is    Thu03PM    0:00.01
> dhclient: bge0 (dhclient)
> _syslogd 27943  0.0  0.0   452   812 ??  S     Thu03PM    0:00.19
> syslogd -a /var/empty/dev/log
>
> I'm not sure how this is done, but it looks like chroot also supports
> changing the userid...
>
> Just some thoughts,
>
> Best regards,
>
> Don
>
>
>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From hks.private at gmail.com  Tue Nov 18 22:57:58 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 16:57:58 -0500
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
Message-ID: <ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>

On Tue, Nov 18, 2008 at 4:10 PM,  <david at lang.hm> wrote:
> On Tue, 18 Nov 2008, Don Jackson wrote:
>
>> I had the following questions:
>>
>> Would it be possible (optionally) to have rsyslogd chroot to a
>> particular directory on startup?
>> That seems the safest.  One could configure a disk partition for log
>> messages, configure rsyslogd to log there,
>
> chroot doesn't help. if you have rsyslog set to log to a seperate
> partition it can only fill that partition, but it can fill _all of that
> partition even if you chroot into a subdirectory on that partition.

Yes, but chrooting precludes any possibility of a rogue syslog agent
filling up /other/ partitions. In the event that a security compromise
were found in the rsyslogd service itself, it also confines an
attacker's damage to the chrooted environment. Paired with non-root
privileges, that's decent insurance against a full-on machine ownage.


>> and also chroot to a directory on that partition, so if the rsyslogd
>> process itself is compromised,
>> it can't do other damage.
>
> that gives you protection if you are receiving logs from the network, but
> if you are receiving logs from /dev/log (local logs) you can't go into a
> chroot effectivly
>
>> There must be a way to have a daemon run as a non-root user, and also
>> to open ports < 1024.
>> This seems to be done all the time on *bsd machines:
>
> the POSIX standard still calls for ports < 1024 to require root to bind to
> them, different systems have different ways to be non-compliant with the
> standard and let you do so anyway. what OS are you using?

OpenBSD has solved problems like this with their own daemons by
following two (general) stages of startup:

 - Privileged, where the daemon reads its config file and opens the
necessary sockets (but doesn't yet listen on them). It then chroots
itself and drops its privileges:
 - Unprivileged, where the daemon begins communication with the rest
of the environment and performing whatever job is required of it

For a concise discussion of how this applies to Apache (the best
explanation I've found in their docs), see:
http://www.openbsd.org/faq/faq10.html#httpdchroot

-HKS


From rsyslog at clark-communications.com  Tue Nov 18 23:02:46 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 14:02:46 -0800
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
Message-ID: <85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>


On Nov 18, 2008, at 1:10 PM, david at lang.hm wrote:

> On Tue, 18 Nov 2008, Don Jackson wrote:
>
>> I had the following questions:
>>
>> Would it be possible (optionally) to have rsyslogd chroot to a
>> particular directory on startup?
>> That seems the safest.  One could configure a disk partition for log
>> messages, configure rsyslogd to log there,
>
> chroot doesn't help. if you have rsyslog set to log to a seperate
> partition it can only fill that partition, but it can fill _all of  
> that
> partition even if you chroot into a subdirectory on that partition.

Agree that file systems or partitions can get filled up no matter what  
you do.
But it seems to me the safest thing to do is to enable rsyslog to be  
configured to
use a specific file system on a dedicated partition, and then always  
monitor that.
Clearly that is possible with rsyslog now, but chrooting would make  
this even more restrictive/secure.

If the (r)syslog clients are also using rsyslog, then they could be  
configured for reliable delivery,
and queue log msgs in the event the rsyslog "server" gets hosed,so one  
you fixed your rsyslog server,
they would then be able to deliver their logs.

Seems like that would be the best overall approach.

>
>> and also chroot to a directory on that partition, so if the rsyslogd
>> process itself is compromised,
>> it can't do other damage.
>
> that gives you protection if you are receiving logs from the  
> network, but
> if you are receiving logs from /dev/log (local logs) you can't go  
> into a
> chroot effectivly

Protecting the "server" from attacks from networked clients seems like  
a good idea.

To answer your later question, I use OpenBSD a lot.
Here is how the stock syslogd on OpenBSD provides a /dev/log for each  
chrooted app:

NAME
      syslogd - log systems messages

SYNOPSIS
      syslogd [-dnu] [-a path] [-f config_file] [-m mark_interval]
              [-p log_socket] [-s reporting_socket]

DESCRIPTION
      syslogd reads and logs messages to the system console, log  
files, pipes
      to other programs, other machines and/or users as specified by  
its con-
      figuration file.

      The options are as follows:

      -a path
              Specify a location where syslogd should place an  
additional log
              socket.  Up to about 20 additional logging sockets can  
be speci-
              fied.  The primary use for this is to place additional  
log sock-
              ets in /dev/log of various chroot filespaces.

So you can arrange for each chroot'ed application to have it's own  
copy of /dev/log in its chroot.

It looks like syslogd on OpenBSD itself doesn't chroot.  I am still  
thinking it might be possible to do so (see below), but maybe I am  
totally wrong about this.

>
>> There must be a way to have a daemon run as a non-root user, and also
>> to open ports < 1024.
>> This seems to be done all the time on *bsd machines:
>
> the POSIX standard still calls for ports < 1024 to require root to  
> bind to
> them, different systems have different ways to be non-compliant with  
> the
> standard and let you do so anyway. what OS are you using?

Sorry, I was not being clear nor accurate here.

What I meant to say was, OK, start up as root, open the resources you  
need, and then chroot, both to change your userid to something that is  
non-root ( user _rsyslog ?), and also
to put rsyslog into a restricted subset file system.  Could you not  
open up the network ports needed, and also /dev/log (for the benefit  
of apps running on that machine that will
write there?) before the call to chroot?

If not, it looks like what syslogd on OpenBSD does is separate itself  
into to processes, the main process runs as root, and then it spawns a  
child that runs as user _syslogd,
it is this child process that listens to log sockets, while the parent  
process gives the child access to write log files.

I'm just brainstorming here.  Perhaps what I am thinking is not  
possible.

Don




From david at lang.hm  Tue Nov 18 23:06:48 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 14:06:48 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
Message-ID: <alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, (private) HKS wrote:

> On Tue, Nov 18, 2008 at 4:10 PM,  <david at lang.hm> wrote:
>> On Tue, 18 Nov 2008, Don Jackson wrote:
>>
>>> I had the following questions:
>>>
>>> Would it be possible (optionally) to have rsyslogd chroot to a
>>> particular directory on startup?
>>> That seems the safest.  One could configure a disk partition for log
>>> messages, configure rsyslogd to log there,
>>
>> chroot doesn't help. if you have rsyslog set to log to a seperate
>> partition it can only fill that partition, but it can fill _all of that
>> partition even if you chroot into a subdirectory on that partition.
>
> Yes, but chrooting precludes any possibility of a rogue syslog agent
> filling up /other/ partitions. In the event that a security compromise
> were found in the rsyslogd service itself, it also confines an
> attacker's damage to the chrooted environment. Paired with non-root
> privileges, that's decent insurance against a full-on machine ownage.

true, but that's a very different problem than the one that Don quoted in 
his message

>>> and also chroot to a directory on that partition, so if the rsyslogd
>>> process itself is compromised,
>>> it can't do other damage.
>>
>> that gives you protection if you are receiving logs from the network, but
>> if you are receiving logs from /dev/log (local logs) you can't go into a
>> chroot effectivly
>>
>>> There must be a way to have a daemon run as a non-root user, and also
>>> to open ports < 1024.
>>> This seems to be done all the time on *bsd machines:
>>
>> the POSIX standard still calls for ports < 1024 to require root to bind to
>> them, different systems have different ways to be non-compliant with the
>> standard and let you do so anyway. what OS are you using?
>
> OpenBSD has solved problems like this with their own daemons by
> following two (general) stages of startup:
>
> - Privileged, where the daemon reads its config file and opens the
> necessary sockets (but doesn't yet listen on them). It then chroots
> itself and drops its privileges:
> - Unprivileged, where the daemon begins communication with the rest
> of the environment and performing whatever job is required of it
>
> For a concise discussion of how this applies to Apache (the best
> explanation I've found in their docs), see:
> http://www.openbsd.org/faq/faq10.html#httpdchroot

Yes, many applications drop privilages (and optionally chroot) after 
starting up.

I suspect that changing rsyslog to do this would be a fair bit of work 
(among other things it can't do a full restart without full privilages, so 
the historic HUP behavior couldn't work)

David Lang


From david at lang.hm  Tue Nov 18 23:13:43 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 14:13:43 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
Message-ID: <alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, Don Jackson wrote:

> On Nov 18, 2008, at 1:10 PM, david at lang.hm wrote:
>
>> On Tue, 18 Nov 2008, Don Jackson wrote:
>>
>>
>>> and also chroot to a directory on that partition, so if the rsyslogd
>>> process itself is compromised,
>>> it can't do other damage.
>>
>> that gives you protection if you are receiving logs from the
>> network, but
>> if you are receiving logs from /dev/log (local logs) you can't go
>> into a
>> chroot effectivly
>
> Protecting the "server" from attacks from networked clients seems like
> a good idea.
>
> To answer your later question, I use OpenBSD a lot.
> Here is how the stock syslogd on OpenBSD provides a /dev/log for each
> chrooted app:

the issue isn't providing a /dev/log for each chroot app, that can be done 
easily from a syslogd that's not in the chroot by opening additional 
pipes, the problem is that if the syslogd itself is inside a chroot it 
cannot open anything _outside_ of that chroot, so it can't provide the 
/dev/log device for the main system.

and you don't want to open that before you go into the chroot becouse one 
of the ways to break out of a chroot involves having a file handle open to 
something outside of the chroot.

>>> There must be a way to have a daemon run as a non-root user, and also
>>> to open ports < 1024.
>>> This seems to be done all the time on *bsd machines:
>>
>> the POSIX standard still calls for ports < 1024 to require root to
>> bind to
>> them, different systems have different ways to be non-compliant with
>> the
>> standard and let you do so anyway. what OS are you using?
>
> Sorry, I was not being clear nor accurate here.
>
> What I meant to say was, OK, start up as root, open the resources you
> need, and then chroot, both to change your userid to something that is
> non-root ( user _rsyslog ?), and also
> to put rsyslog into a restricted subset file system.  Could you not
> open up the network ports needed, and also /dev/log (for the benefit
> of apps running on that machine that will
> write there?) before the call to chroot?

changing the userid will work, doing a chroot will not.

> If not, it looks like what syslogd on OpenBSD does is separate itself
> into to processes, the main process runs as root, and then it spawns a
> child that runs as user _syslogd,
> it is this child process that listens to log sockets, while the parent
> process gives the child access to write log files.

rsyslog is multi-threaded, not multi-process. all the threads have access 
to everything that any one thread has access to, so it wouldn't help to 
seperate things out without serious surgery to rsyslog.

you could do something like have one copy of rsyslog running inside a 
chroot, and another one running outside the chroot with the one outside 
the chroot not listening to the network but forwarding everything it 
receives to the one inside the chroot.

this may require a little bit of work to make work (depending on how 
outbound network connections are made from rsyslog)

David Lang


From hks.private at gmail.com  Tue Nov 18 23:29:43 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 17:29:43 -0500
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
Message-ID: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>

FWIW, while I believe this discussion is relevant and the security
changes proposed are important, I don't see this as a high priority
feature. The scripting language/syntax, greater performance, continued
stability enhancements, and true RELP support are all far more
important. As far as security goes, I would much rather see two-way
host authentication than a chroot/unprivilieged framework implemented.

Of course, all of the above would be just fine with me.

-HKS


From rgerhards at hq.adiscon.com  Wed Nov 19 09:39:18 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 09:39:18 +0100
Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
In-Reply-To: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>
References: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F683@grfint2.intern.adiscon.com>

Hi Don,

this is very good news. Please let me know if you run into any problems
that require an upstream fix. I have also seen your other message, but
will read through that thread before I reply.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Don Jackson
> Sent: Tuesday, November 18, 2008 9:24 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
> 
> 
> Hello,
> 
> I have motivated the creation of a port of rsyslog to OpenBSD.
> The port is intended to fully support the OpenBSD ports framework, and
> will build an OpenBSD binary package
> suitable for installation via pkg_add.  See the message below for a
> copy of the port.
> 
> I have done some testing of this port on OpenBSD 4.4 (stable) under
> amd64.
> 
> I would very much appreciate any additional testing and/or comments
> from the rsyslog community.
> 
> Best regards,
> 
> Don
> 
> 
> Begin forwarded message:
> > From: Don Jackson <don.jackson at gmail.com>
> > Date: November 18, 2008 11:07:45 AM PST
> > To: ports at openbsd.org
> > Subject: NEW: sysutils/rsyslog-3.20.0
> >
> >
> > Tested on 4.4/amd64, definitely interested in more testing.
> >
> > $ cat ./pkg/DESCR
> >
> > A syslogd replacement
> >
> >
> >
> >
> >
> >
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 09:46:58 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 09:46:58 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
Message-ID: <1227084418.19905.38.camel@rgf9dev.intern.adiscon.com>

Hi HKS,

just one point for now:

On Tue, 2008-11-18 at 17:29 -0500, (private) HKS wrote:
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

Two-way host authentication is available in the stable branch since
3.20.0. This is done by configuring the proper certficate settings in
TLS mode. As syslog-TLS will soon turn into a full-blown RFC, I expect
to see interoperable implementations in the not so distant future.

Rainer



From alorbach at ro1.adiscon.com  Wed Nov 19 10:37:15 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 19 Nov 2008 10:37:15 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
	<a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>

I think this would be possible yes, we need to define a common name for
the database and property field in rsyslog, then we can start adding
support for it in phplogcon. 

regards,
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of mikel
> Sent: Dienstag, 18. November 2008 21:06
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> 
> Dear Andre
> 
> So, in near future will be possible?
> 
> Or in far future?
> 
> Thanx
> 
> On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> <rgerhards at hq.adiscon.com> wrote:
> > Andre and all,
> >
> > the rsyslog engine has the necessary plumbing and I know of at least
> one
> > user who stores subsecond timestamps in this way. As far as rsyslog
> is
> > concerned, this just requires a special timestamp. So if you would
> > support that in phpLogCon (what I would find a good idea ;)), I
would
> > include a standard template into rsyslog (hardcoded). That would
make
> it
> > easier for people to work with it. But maybe we get along by just
> > properly documenting it, because the schema must be modified in any
> > case.
> >
> > Another alternative would be to update the default schema, but I am
> not
> > sure it that is actually a good idea. On the other hand, high-
> precision
> > timestamps hopefully get into more widespread use...
> >
> > Thoughts from the community are appreciated.
> >
> > Thanks,
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> >> Sent: Tuesday, November 18, 2008 5:56 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] milliseconds timestamp
> >>
> >> I think it wouldn't be so difficult to implement, if the
> milliseconds
> >> were simply stored in a second field.
> >> We can add custom new fields into phpLogCon very easily.
> >>
> >> best regards,
> >> Andre Lorbach
> >>
> >> > -----Original Message-----
> >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> >> > Sent: Dienstag, 18. November 2008 16:39
> >> > To: rsyslog-users
> >> > Subject: Re: [rsyslog] milliseconds timestamp
> >> >
> >> > Hi Mikel,
> >> >
> >> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
> >> does
> >> > not support millisecond resolution in timestamps. So a
work-around
> >> > would
> >> > be to store them to another field and add this as a second field
> to
> >> the
> >> > phpLogCon view. I think it would probably smart to enable
> phpLogCon
> >> to
> >> > combine two such fields, but I leave this to Andre...
> >> >
> >> > Rainer
> >> >
> >> > > -----Original Message-----
> >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> >> > > To: rsyslog-users
> >> > > Subject: [rsyslog] milliseconds timestamp
> >> > >
> >> > > Hello
> >> > > I am using phplogcon viewer and I have loosed milliseconds
> >> > timestamps.
> >> > >
> >> > > I watched that "DATE" in mysql databases is date-time type.
> >> > >
> >> > > Does rsyslog gives the date in milliseconds? is possible to see
> >> this
> >> > > milliseconds din phplogcon?
> >> > >
> >> > > --
> >> > > Mikel Jimenez Fernandez
> >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> >> http://www.irontec.com
> >> > > +34 94.404.81.82
> >> > >
> >> > >
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > > http://www.rsyslog.com
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 10:38:45 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 10:38:45 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>

Andre,

So do you think we should add this to the standard schema? I am a bit
hesitant, as not all folks will probably want that and it keeps backward
compatibility. Maybe it is better to just us a standard name but not
include it in the standard schema.

What do you (and others) think?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Wednesday, November 19, 2008 10:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> I think this would be possible yes, we need to define a common name
for
> the database and property field in rsyslog, then we can start adding
> support for it in phplogcon.
> 
> regards,
> Andre
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of mikel
> > Sent: Dienstag, 18. November 2008 21:06
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> >
> > Dear Andre
> >
> > So, in near future will be possible?
> >
> > Or in far future?
> >
> > Thanx
> >
> > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > <rgerhards at hq.adiscon.com> wrote:
> > > Andre and all,
> > >
> > > the rsyslog engine has the necessary plumbing and I know of at
> least
> > one
> > > user who stores subsecond timestamps in this way. As far as
rsyslog
> > is
> > > concerned, this just requires a special timestamp. So if you would
> > > support that in phpLogCon (what I would find a good idea ;)), I
> would
> > > include a standard template into rsyslog (hardcoded). That would
> make
> > it
> > > easier for people to work with it. But maybe we get along by just
> > > properly documenting it, because the schema must be modified in
any
> > > case.
> > >
> > > Another alternative would be to update the default schema, but I
am
> > not
> > > sure it that is actually a good idea. On the other hand, high-
> > precision
> > > timestamps hopefully get into more widespread use...
> > >
> > > Thoughts from the community are appreciated.
> > >
> > > Thanks,
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > >> To: rsyslog-users
> > >> Subject: Re: [rsyslog] milliseconds timestamp
> > >>
> > >> I think it wouldn't be so difficult to implement, if the
> > milliseconds
> > >> were simply stored in a second field.
> > >> We can add custom new fields into phpLogCon very easily.
> > >>
> > >> best regards,
> > >> Andre Lorbach
> > >>
> > >> > -----Original Message-----
> > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > >> > Sent: Dienstag, 18. November 2008 16:39
> > >> > To: rsyslog-users
> > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > >> >
> > >> > Hi Mikel,
> > >> >
> > >> > I have talked to Andre, phpLogCon's lead developer. He says
> MySQL
> > >> does
> > >> > not support millisecond resolution in timestamps. So a
> work-around
> > >> > would
> > >> > be to store them to another field and add this as a second
field
> > to
> > >> the
> > >> > phpLogCon view. I think it would probably smart to enable
> > phpLogCon
> > >> to
> > >> > combine two such fields, but I leave this to Andre...
> > >> >
> > >> > Rainer
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > >> > > To: rsyslog-users
> > >> > > Subject: [rsyslog] milliseconds timestamp
> > >> > >
> > >> > > Hello
> > >> > > I am using phplogcon viewer and I have loosed milliseconds
> > >> > timestamps.
> > >> > >
> > >> > > I watched that "DATE" in mysql databases is date-time type.
> > >> > >
> > >> > > Does rsyslog gives the date in milliseconds? is possible to
> see
> > >> this
> > >> > > milliseconds din phplogcon?
> > >> > >
> > >> > > --
> > >> > > Mikel Jimenez Fernandez
> > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > >> http://www.irontec.com
> > >> > > +34 94.404.81.82
> > >> > >
> > >> > >
> > >> > > _______________________________________________
> > >> > > rsyslog mailing list
> > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > > http://www.rsyslog.com
> > >> > _______________________________________________
> > >> > rsyslog mailing list
> > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > http://www.rsyslog.com
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From alorbach at ro1.adiscon.com  Wed Nov 19 10:48:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 19 Nov 2008 10:48:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>

By not adding it into the standard schema, you mean the default table
layout for the database? 
We may could reuse an existing unused field instead?

--
Andre 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Mittwoch, 19. November 2008 10:39
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> Andre,
> 
> So do you think we should add this to the standard schema? I am a bit
> hesitant, as not all folks will probably want that and it keeps
> backward
> compatibility. Maybe it is better to just us a standard name but not
> include it in the standard schema.
> 
> What do you (and others) think?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > Sent: Wednesday, November 19, 2008 10:37 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > I think this would be possible yes, we need to define a common name
> for
> > the database and property field in rsyslog, then we can start adding
> > support for it in phplogcon.
> >
> > regards,
> > Andre
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of mikel
> > > Sent: Dienstag, 18. November 2008 21:06
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] milliseconds timestamp
> > >
> > >
> > > Dear Andre
> > >
> > > So, in near future will be possible?
> > >
> > > Or in far future?
> > >
> > > Thanx
> > >
> > > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > > <rgerhards at hq.adiscon.com> wrote:
> > > > Andre and all,
> > > >
> > > > the rsyslog engine has the necessary plumbing and I know of at
> > least
> > > one
> > > > user who stores subsecond timestamps in this way. As far as
> rsyslog
> > > is
> > > > concerned, this just requires a special timestamp. So if you
> would
> > > > support that in phpLogCon (what I would find a good idea ;)), I
> > would
> > > > include a standard template into rsyslog (hardcoded). That would
> > make
> > > it
> > > > easier for people to work with it. But maybe we get along by
just
> > > > properly documenting it, because the schema must be modified in
> any
> > > > case.
> > > >
> > > > Another alternative would be to update the default schema, but I
> am
> > > not
> > > > sure it that is actually a good idea. On the other hand, high-
> > > precision
> > > > timestamps hopefully get into more widespread use...
> > > >
> > > > Thoughts from the community are appreciated.
> > > >
> > > > Thanks,
> > > > Rainer
> > > >
> > > >> -----Original Message-----
> > > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > > >> To: rsyslog-users
> > > >> Subject: Re: [rsyslog] milliseconds timestamp
> > > >>
> > > >> I think it wouldn't be so difficult to implement, if the
> > > milliseconds
> > > >> were simply stored in a second field.
> > > >> We can add custom new fields into phpLogCon very easily.
> > > >>
> > > >> best regards,
> > > >> Andre Lorbach
> > > >>
> > > >> > -----Original Message-----
> > > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > >> > Sent: Dienstag, 18. November 2008 16:39
> > > >> > To: rsyslog-users
> > > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > > >> >
> > > >> > Hi Mikel,
> > > >> >
> > > >> > I have talked to Andre, phpLogCon's lead developer. He says
> > MySQL
> > > >> does
> > > >> > not support millisecond resolution in timestamps. So a
> > work-around
> > > >> > would
> > > >> > be to store them to another field and add this as a second
> field
> > > to
> > > >> the
> > > >> > phpLogCon view. I think it would probably smart to enable
> > > phpLogCon
> > > >> to
> > > >> > combine two such fields, but I leave this to Andre...
> > > >> >
> > > >> > Rainer
> > > >> >
> > > >> > > -----Original Message-----
> > > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > >> > > To: rsyslog-users
> > > >> > > Subject: [rsyslog] milliseconds timestamp
> > > >> > >
> > > >> > > Hello
> > > >> > > I am using phplogcon viewer and I have loosed milliseconds
> > > >> > timestamps.
> > > >> > >
> > > >> > > I watched that "DATE" in mysql databases is date-time type.
> > > >> > >
> > > >> > > Does rsyslog gives the date in milliseconds? is possible to
> > see
> > > >> this
> > > >> > > milliseconds din phplogcon?
> > > >> > >
> > > >> > > --
> > > >> > > Mikel Jimenez Fernandez
> > > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > > >> http://www.irontec.com
> > > >> > > +34 94.404.81.82
> > > >> > >
> > > >> > >
> > > >> > > _______________________________________________
> > > >> > > rsyslog mailing list
> > > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > > http://www.rsyslog.com
> > > >> > _______________________________________________
> > > >> > rsyslog mailing list
> > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > http://www.rsyslog.com
> > > >> _______________________________________________
> > > >> rsyslog mailing list
> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 10:49:28 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 10:49:28 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F688@grfint2.intern.adiscon.com>

No, I mean that we define a field name, but leave it to the user if he
or she adds/creates the field to the schema (actually two, one for
received and one for generated).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Wednesday, November 19, 2008 10:48 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> By not adding it into the standard schema, you mean the default table
> layout for the database?
> We may could reuse an existing unused field instead?
> 
> --
> Andre
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Mittwoch, 19. November 2008 10:39
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > Andre,
> >
> > So do you think we should add this to the standard schema? I am a
bit
> > hesitant, as not all folks will probably want that and it keeps
> > backward
> > compatibility. Maybe it is better to just us a standard name but not
> > include it in the standard schema.
> >
> > What do you (and others) think?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > Sent: Wednesday, November 19, 2008 10:37 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] milliseconds timestamp
> > >
> > > I think this would be possible yes, we need to define a common
name
> > for
> > > the database and property field in rsyslog, then we can start
> adding
> > > support for it in phplogcon.
> > >
> > > regards,
> > > Andre
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of mikel
> > > > Sent: Dienstag, 18. November 2008 21:06
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] milliseconds timestamp
> > > >
> > > >
> > > > Dear Andre
> > > >
> > > > So, in near future will be possible?
> > > >
> > > > Or in far future?
> > > >
> > > > Thanx
> > > >
> > > > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > > > <rgerhards at hq.adiscon.com> wrote:
> > > > > Andre and all,
> > > > >
> > > > > the rsyslog engine has the necessary plumbing and I know of at
> > > least
> > > > one
> > > > > user who stores subsecond timestamps in this way. As far as
> > rsyslog
> > > > is
> > > > > concerned, this just requires a special timestamp. So if you
> > would
> > > > > support that in phpLogCon (what I would find a good idea ;)),
I
> > > would
> > > > > include a standard template into rsyslog (hardcoded). That
> would
> > > make
> > > > it
> > > > > easier for people to work with it. But maybe we get along by
> just
> > > > > properly documenting it, because the schema must be modified
in
> > any
> > > > > case.
> > > > >
> > > > > Another alternative would be to update the default schema, but
> I
> > am
> > > > not
> > > > > sure it that is actually a good idea. On the other hand, high-
> > > > precision
> > > > > timestamps hopefully get into more widespread use...
> > > > >
> > > > > Thoughts from the community are appreciated.
> > > > >
> > > > > Thanks,
> > > > > Rainer
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > > > >> To: rsyslog-users
> > > > >> Subject: Re: [rsyslog] milliseconds timestamp
> > > > >>
> > > > >> I think it wouldn't be so difficult to implement, if the
> > > > milliseconds
> > > > >> were simply stored in a second field.
> > > > >> We can add custom new fields into phpLogCon very easily.
> > > > >>
> > > > >> best regards,
> > > > >> Andre Lorbach
> > > > >>
> > > > >> > -----Original Message-----
> > > > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > > >> > Sent: Dienstag, 18. November 2008 16:39
> > > > >> > To: rsyslog-users
> > > > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > > > >> >
> > > > >> > Hi Mikel,
> > > > >> >
> > > > >> > I have talked to Andre, phpLogCon's lead developer. He says
> > > MySQL
> > > > >> does
> > > > >> > not support millisecond resolution in timestamps. So a
> > > work-around
> > > > >> > would
> > > > >> > be to store them to another field and add this as a second
> > field
> > > > to
> > > > >> the
> > > > >> > phpLogCon view. I think it would probably smart to enable
> > > > phpLogCon
> > > > >> to
> > > > >> > combine two such fields, but I leave this to Andre...
> > > > >> >
> > > > >> > Rainer
> > > > >> >
> > > > >> > > -----Original Message-----
> > > > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > > >> > > To: rsyslog-users
> > > > >> > > Subject: [rsyslog] milliseconds timestamp
> > > > >> > >
> > > > >> > > Hello
> > > > >> > > I am using phplogcon viewer and I have loosed
milliseconds
> > > > >> > timestamps.
> > > > >> > >
> > > > >> > > I watched that "DATE" in mysql databases is date-time
> type.
> > > > >> > >
> > > > >> > > Does rsyslog gives the date in milliseconds? is possible
> to
> > > see
> > > > >> this
> > > > >> > > milliseconds din phplogcon?
> > > > >> > >
> > > > >> > > --
> > > > >> > > Mikel Jimenez Fernandez
> > > > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > > > >> http://www.irontec.com
> > > > >> > > +34 94.404.81.82
> > > > >> > >
> > > > >> > >
> > > > >> > > _______________________________________________
> > > > >> > > rsyslog mailing list
> > > > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> > > http://www.rsyslog.com
> > > > >> > _______________________________________________
> > > > >> > rsyslog mailing list
> > > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> > http://www.rsyslog.com
> > > > >> _______________________________________________
> > > > >> rsyslog mailing list
> > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 13:32:08 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:32:08 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
	<alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68A@grfint2.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, November 18, 2008 11:07 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
[snip]

> Yes, many applications drop privilages (and optionally chroot) after
> starting up.
> 
> I suspect that changing rsyslog to do this would be a fair bit of work
> (among other things it can't do a full restart without full
privilages,
> so
> the historic HUP behavior couldn't work)

Yes, the traditional HUP behavior is simply incompatible with dropping
privileges. But I assume that someone who configures rsyslogd in such a
way knows what he does and thus changes config-reload scripts away from
HUP to a real reload.

Rainer


From rgerhards at hq.adiscon.com  Wed Nov 19 13:46:01 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:46:01 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
Message-ID: <1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>

Hi all,

So, let me use this reply as a warp-up of my thoughts. Basically, I
agree to HKS argument. I also agree to David Lang's argument that some
degree of design change would be necessary to get the requested features
done 100% correct.

But, of course, it all depends on the effort required. And more security
is always welcome. So I took time out today to think about the options
and experiment a bit. I came to the conclusion that some improvement can
probably be made in short time. I experimentally implemented a new
$PrivDropToUser config directive, which basically issues a setuid() call
with the user in question. It is far from being bullet-proof and will
never be totally save without architecture changes. But I think it still
can considerably improve security and when we can achieve this with a
few hours of work, it's probably worth the effort (a bit more work would
improve its security, but I don't do this unless I get feedback it makes
sense at all).

HOWEVER, there are also some things that simply don't work out. For
example, imklog reads the proc file system under Linux and, even though
it is opened as root, these reads fail after dropping to another user.
There may be other issues, I have not further investigated that.

I have created a bugzilla enhancement request for it:

http://bugzilla.adiscon.com/show_bug.cgi?id=105

You may want to subscribe to that bug for updates. 

The experimental code is available here:

http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/droppriv

I would appreciate feedback

a) on the general issue
b) on the experimental code itself

Many thanks,
Rainer

On Tue, 2008-11-18 at 17:29 -0500, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.
> 
> Of course, all of the above would be just fine with me.
> 
> -HKS
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From mbiebl at gmail.com  Wed Nov 19 13:53:12 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 13:53:12 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
Message-ID: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>

2008/11/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> So, let me use this reply as a warp-up of my thoughts. Basically, I
> agree to HKS argument. I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested features
> done 100% correct.
>
> But, of course, it all depends on the effort required. And more security
> is always welcome. So I took time out today to think about the options
> and experiment a bit. I came to the conclusion that some improvement can
> probably be made in short time. I experimentally implemented a new
> $PrivDropToUser config directive, which basically issues a setuid() call
> with the user in question. It is far from being bullet-proof and will
> never be totally save without architecture changes. But I think it still
> can considerably improve security and when we can achieve this with a
> few hours of work, it's probably worth the effort (a bit more work would
> improve its security, but I don't do this unless I get feedback it makes
> sense at all).
>
> HOWEVER, there are also some things that simply don't work out. For
> example, imklog reads the proc file system under Linux and, even though
> it is opened as root, these reads fail after dropping to another user.
> There may be other issues, I have not further investigated that.

/dev/klog could be made accessible via an ACL or a separate group.
(e.g. the init script of the distro could chgrp or setfacl /dev/klog).
Distros will also have to make sure, that the rsyslog daemon has write
access to /var/log (or wherever they put their log files by default,
but that is easily solvable too).

More tricky will be the already mentioned opening of privileged ports I think.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Wed Nov 19 13:55:29 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 13:55:29 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>

2008/11/19 Michael Biebl <mbiebl at gmail.com>:

>
> /dev/klog could be made accessible via an ACL or a separate group.
^^^^^^^^^^^^
Argh, /proc/kmsg, I mean.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 19 13:56:26 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:56:26 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68B@grfint2.intern.adiscon.com>

Hi Michael,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, November 19, 2008 1:53 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> 2008/11/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Hi all,
> >
> > So, let me use this reply as a warp-up of my thoughts. Basically, I
> > agree to HKS argument. I also agree to David Lang's argument that
> some
> > degree of design change would be necessary to get the requested
> features
> > done 100% correct.
> >
> > But, of course, it all depends on the effort required. And more
> security
> > is always welcome. So I took time out today to think about the
> options
> > and experiment a bit. I came to the conclusion that some improvement
> can
> > probably be made in short time. I experimentally implemented a new
> > $PrivDropToUser config directive, which basically issues a setuid()
> call
> > with the user in question. It is far from being bullet-proof and
will
> > never be totally save without architecture changes. But I think it
> still
> > can considerably improve security and when we can achieve this with
a
> > few hours of work, it's probably worth the effort (a bit more work
> would
> > improve its security, but I don't do this unless I get feedback it
> makes
> > sense at all).
> >
> > HOWEVER, there are also some things that simply don't work out. For
> > example, imklog reads the proc file system under Linux and, even
> though
> > it is opened as root, these reads fail after dropping to another
> user.
> > There may be other issues, I have not further investigated that.
> 
> /dev/klog could be made accessible via an ACL or a separate group.
> (e.g. the init script of the distro could chgrp or setfacl /dev/klog).
> Distros will also have to make sure, that the rsyslog daemon has write
> access to /var/log (or wherever they put their log files by default,
> but that is easily solvable too).
> 
> More tricky will be the already mentioned opening of privileged ports
I
> think.

The experimental code handles that. It drops privileges after processing
the config file. There is a window of insecurity between when the
listeners start and the privileges are dropped, and that can not be
solved without an architecture change. But in essence, the "port open"
problem is solved (of course, HUP will always be of non-restart type). I
think security is improved even with this Window of insecurity, so if
the rest works OK, I would consider the new code a valuable addition.

Rainer 


From mbiebl at gmail.com  Wed Nov 19 14:07:34 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 14:07:34 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
Message-ID: <aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>

2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
>
>>
>> /dev/klog could be made accessible via an ACL or a separate group.
> ^^^^^^^^^^^^
> Argh, /proc/kmsg, I mean.

Looks like one can't set an ACL on a procfs unfortunately.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 19 14:10:24 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 14:10:24 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
	<aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68D@grfint2.intern.adiscon.com>

An out-of-process implementation for the plugins would solve all such
issues. But that takes time... A work-around may be to run two rsyslogd
processes concurrently. Definitely not a mainstream or default config,
but could be done. Also, I wonder if the kernel log works on BSD (not
tested), in which case the experimental code would work there without
any bad side effects (better phrased: with side-effects yet to be seen,
as I said I did so far not invest any serious time in testing).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, November 19, 2008 2:08 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> > 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> >
> >>
> >> /dev/klog could be made accessible via an ACL or a separate group.
> > ^^^^^^^^^^^^
> > Argh, /proc/kmsg, I mean.
> 
> Looks like one can't set an ACL on a procfs unfortunately.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Thu Nov 20 16:14:50 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Thu, 20 Nov 2008 10:14:50 -0500
Subject: [rsyslog] documentation: <object>blah, what is object?
Message-ID: <49257EEA.2000706@canonical.com>

I've been looking over the rsyslog documentation and I see a lot of
directives that begin with the nomenclature "<object>".  I see examples
of this tag being assigned to "Action" and "MainMsg".

Is there an explanation of what <object> is all about?

Thanks,

Peter


From rgerhards at hq.adiscon.com  Thu Nov 20 17:10:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:10:04 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <49257EEA.2000706@canonical.com>
References: <49257EEA.2000706@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>

I guess you mean the queue documentation. It looks like I have forgotten
that piece of information ;) <object> is "Action" for action queues and
"MainMsg" for the main message queue. Will add that soon, thanks for
pointing out.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Thursday, November 20, 2008 4:15 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] documentation: <object>blah, what is object?
> 
> I've been looking over the rsyslog documentation and I see a lot of
> directives that begin with the nomenclature "<object>".  I see
examples
> of this tag being assigned to "Action" and "MainMsg".
> 
> Is there an explanation of what <object> is all about?
> 
> Thanks,
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Nov 20 17:11:39 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:11:39 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>

Lol, actually it is described at the *bottom* of the document. I will
add a refrecence to that at the front ;)

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, November 20, 2008 5:10 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> 
> I guess you mean the queue documentation. It looks like I have
> forgotten
> that piece of information ;) <object> is "Action" for action queues
and
> "MainMsg" for the main message queue. Will add that soon, thanks for
> pointing out.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> > Sent: Thursday, November 20, 2008 4:15 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] documentation: <object>blah, what is object?
> >
> > I've been looking over the rsyslog documentation and I see a lot of
> > directives that begin with the nomenclature "<object>".  I see
> examples
> > of this tag being assigned to "Action" and "MainMsg".
> >
> > Is there an explanation of what <object> is all about?
> >
> > Thanks,
> >
> > Peter
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Nov 20 17:16:49 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:16:49 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com><577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>

I "fixed" this. The bottom part could simply be moved up, I think it now
makes more sense. You may want to have a look at the online version:

http://www.rsyslog.com/doc-queues.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, November 20, 2008 5:12 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> 
> Lol, actually it is described at the *bottom* of the document. I will
> add a refrecence to that at the front ;)
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, November 20, 2008 5:10 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> >
> > I guess you mean the queue documentation. It looks like I have
> > forgotten
> > that piece of information ;) <object> is "Action" for action queues
> and
> > "MainMsg" for the main message queue. Will add that soon, thanks for
> > pointing out.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> > > Sent: Thursday, November 20, 2008 4:15 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] documentation: <object>blah, what is object?
> > >
> > > I've been looking over the rsyslog documentation and I see a lot
of
> > > directives that begin with the nomenclature "<object>".  I see
> > examples
> > > of this tag being assigned to "Action" and "MainMsg".
> > >
> > > Is there an explanation of what <object> is all about?
> > >
> > > Thanks,
> > >
> > > Peter
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Thu Nov 20 17:32:40 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Thu, 20 Nov 2008 11:32:40 -0500
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com><577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>
Message-ID: <49259128.2040506@canonical.com>

Thanks for your prompt response.

Peter

Rainer Gerhards wrote:
> I "fixed" this. The bottom part could simply be moved up, I think it now
> makes more sense. You may want to have a look at the online version:
> 
> http://www.rsyslog.com/doc-queues.html
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Thursday, November 20, 2008 5:12 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
>>
>> Lol, actually it is described at the *bottom* of the document. I will
>> add a refrecence to that at the front ;)
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>> Sent: Thursday, November 20, 2008 5:10 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
>>>
>>> I guess you mean the queue documentation. It looks like I have
>>> forgotten
>>> that piece of information ;) <object> is "Action" for action queues
>> and
>>> "MainMsg" for the main message queue. Will add that soon, thanks for
>>> pointing out.
>>>
>>> Rainer
>>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
>>>> Sent: Thursday, November 20, 2008 4:15 PM
>>>> To: rsyslog at lists.adiscon.com
>>>> Subject: [rsyslog] documentation: <object>blah, what is object?
>>>>
>>>> I've been looking over the rsyslog documentation and I see a lot
> of
>>>> directives that begin with the nomenclature "<object>".  I see
>>> examples
>>>> of this tag being assigned to "Action" and "MainMsg".
>>>>
>>>> Is there an explanation of what <object> is all about?
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From janfrode at tanso.net  Fri Nov 21 13:11:02 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 13:11:02 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>

For my usage, I need two modes of operation for syslog daemons. 

    1 - local syslog. Requires privileges to on local devices (/dev/log,
        /dev/klogd or similar), write to local log-files, and send to
        remote log server.

    2 - central log server. Only listen on the needed network ports
        (514/udp/tcp), and write to local log files (possibly also send
        to other remote log servers).

For #1 I think it's OK not being able to chroot, keep more privileges,
etc.,  as the attacks against it will mostly be from local processes.

#2 needs to be *very* openly available on the network ports, since all 
my servers needs to send logs to it. #2 will also be holding a lot more
sensitive data than #1, so I think this server needs to be protected as
much as possible. If chroot'ing, or dropping privileges prevents it from
reading from local /proc og /dev, I think that wouldn't matter much. One
could always run two instances on these few central servers, i.e. #1
sending to #2.


  -jf



From rgerhards at hq.adiscon.com  Fri Nov 21 14:50:28 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 14:50:28 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>

So it looks like the new idea, though not perfect, seems to provide some
value, at least under some circumstances. It also looks trivially to
implement. Most effort is probably to tell people precisely why it is
not a fully security guard. I'll see if I get this fully implemented
next week if nobody objects. I guess it's not more than a day's worth.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Friday, November 21, 2008 1:11 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> For my usage, I need two modes of operation for syslog daemons.
> 
>     1 - local syslog. Requires privileges to on local devices
> (/dev/log,
>         /dev/klogd or similar), write to local log-files, and send to
>         remote log server.
> 
>     2 - central log server. Only listen on the needed network ports
>         (514/udp/tcp), and write to local log files (possibly also
send
>         to other remote log servers).
> 
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
> 
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot
more
> sensitive data than #1, so I think this server needs to be protected
as
> much as possible. If chroot'ing, or dropping privileges prevents it
> from
> reading from local /proc og /dev, I think that wouldn't matter much.
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.
> 
> 
>   -jf
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Fri Nov 21 15:31:24 2008
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Nov 2008 06:31:24 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>

On Fri, 21 Nov 2008, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
>    1 - local syslog. Requires privileges to on local devices (/dev/log,
>        /dev/klogd or similar), write to local log-files, and send to
>        remote log server.
>
>    2 - central log server. Only listen on the needed network ports
>        (514/udp/tcp), and write to local log files (possibly also send
>        to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot more
> sensitive data than #1, so I think this server needs to be protected as
> much as possible.

just pointing out that on this central server the data will be in the same 
place as the daemon listening to the network, and accessable to that 
daemon, so chrooting, dropping privilages, etc won't protect the data that 
gets logged (it will allow you to protect other data that lives on the 
server, just not log data)

well, you could do something where rsyslog writes the file in the chroot, 
and when it rolls the file something outside the chroot picks it up and 
moves it elsewhere, but that's starting to get a little extreme.

> If chroot'ing, or dropping privileges prevents it from
> reading from local /proc og /dev, I think that wouldn't matter much. One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

correct.

David Lang


From david at lang.hm  Fri Nov 21 15:31:49 2008
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Nov 2008 06:31:49 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>
Message-ID: <alpine.DEB.1.10.0811210631270.28119@asgard.lang.hm>

On Fri, 21 Nov 2008, Rainer Gerhards wrote:

> So it looks like the new idea, though not perfect, seems to provide some
> value, at least under some circumstances. It also looks trivially to
> implement. Most effort is probably to tell people precisely why it is
> not a fully security guard. I'll see if I get this fully implemented
> next week if nobody objects. I guess it's not more than a day's worth.

I agree that even the limited version has benifits.

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
>> Sent: Friday, November 21, 2008 1:11 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] rsyslogd security questions/comments
>>
>> For my usage, I need two modes of operation for syslog daemons.
>>
>>     1 - local syslog. Requires privileges to on local devices
>> (/dev/log,
>>         /dev/klogd or similar), write to local log-files, and send to
>>         remote log server.
>>
>>     2 - central log server. Only listen on the needed network ports
>>         (514/udp/tcp), and write to local log files (possibly also
> send
>>         to other remote log servers).
>>
>> For #1 I think it's OK not being able to chroot, keep more privileges,
>> etc.,  as the attacks against it will mostly be from local processes.
>>
>> #2 needs to be *very* openly available on the network ports, since all
>> my servers needs to send logs to it. #2 will also be holding a lot
> more
>> sensitive data than #1, so I think this server needs to be protected
> as
>> much as possible. If chroot'ing, or dropping privileges prevents it
>> from
>> reading from local /proc og /dev, I think that wouldn't matter much.
>> One
>> could always run two instances on these few central servers, i.e. #1
>> sending to #2.
>>
>>
>>   -jf
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From peter.matulis at canonical.com  Fri Nov 21 15:33:07 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 21 Nov 2008 09:33:07 -0500
Subject: [rsyslog] available compare operations in properties-base filters
Message-ID: <4926C6A3.8030706@canonical.com>

For properties-based filtering, the documentation states that currently
there is only a single compare operation ('contains') available.  It
then goes on to describe additional ones.  So what operations are
implemented at this point?

Peter



From rgerhards at hq.adiscon.com  Fri Nov 21 15:38:16 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 15:38:16 +0100
Subject: [rsyslog] available compare operations in properties-base
	filters
In-Reply-To: <4926C6A3.8030706@canonical.com>
References: <4926C6A3.8030706@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D6@grfint2.intern.adiscon.com>

Peter,

all modes in the table are implemented. The "single compare available"
sentence is wrong, I think I simply forgot to update it as I added
modes. Will do that now :)

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 21, 2008 3:33 PM
> To: rsyslog-users
> Subject: [rsyslog] available compare operations in properties-base
> filters
> 
> For properties-based filtering, the documentation states that
currently
> there is only a single compare operation ('contains') available.  It
> then goes on to describe additional ones.  So what operations are
> implemented at this point?
> 
> Peter
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From aoz.syn at gmail.com  Fri Nov 21 16:26:14 2008
From: aoz.syn at gmail.com (RB)
Date: Fri, 21 Nov 2008 08:26:14 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
Message-ID: <4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>

On Fri, Nov 21, 2008 at 07:31,  <david at lang.hm> wrote:
> just pointing out that on this central server the data will be in the same
> place as the daemon listening to the network, and accessable to that
> daemon, so chrooting, dropping privilages, etc won't protect the data that
> gets logged (it will allow you to protect other data that lives on the
> server, just not log data)

As is the same with all chrooted processes - the chroot model prevents
untrusted processes from messing with each other's candy, not their
own.  If anyone that runs a chroot thinks otherwise, they're sorely
mistaken.  It's why web servers are often chrooted with no write
access: the only writing they can do is back through specific ports
(database, syslog, memcache, etc.)

> well, you could do something where rsyslog writes the file in the chroot,
> and when it rolls the file something outside the chroot picks it up and
> moves it elsewhere, but that's starting to get a little extreme.

I absolutely agree that approach is foolish - better to give the
process no write permissions at all and have it either log to a
database (INSERT-only user) or run a two-layered approach wherein the
chrooted log server passes the [scrubbed] logs it receives back over a
pipe or to a secondary syslog service.  Crash/subvert the front layer,
you have little place to go.

RB


From janfrode at tanso.net  Fri Nov 21 18:34:56 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 18:34:56 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
Message-ID: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, RB <aoz.syn at gmail.com> wrote:
> On Fri, Nov 21, 2008 at 07:31,  <david at lang.hm> wrote:
>> just pointing out that on this central server the data will be in the same
>> place as the daemon listening to the network, and accessable to that
>> daemon, so chrooting, dropping privilages, etc won't protect the data that
>> gets logged (it will allow you to protect other data that lives on the
>> server, just not log data)
>
> As is the same with all chrooted processes - the chroot model prevents
> untrusted processes from messing with each other's candy, not their
> own.  If anyone that runs a chroot thinks otherwise, they're sorely
> mistaken.

I think the chroot() limits the possibility of:

    - executing anything outside the chroot (f.ex. /bin/sh)
    - write a new file, and execute it (noexec on the chroot mountpoint)
    - overwrite something outside the chroot. I'm an itsy bit worried that 
      something like this:
    
        $template PerAppLogs,"/var/log/apps/%programname%.log"
        *.* -?PerAppLogs

      could be tricked to write to unexpected places if you mess with
      %programname%, or other similar variables that the sender has complete
      controle over.

so I would argue that my sensitive syslogged data is more secure in a
chroot environment where there are no other executables, than on a non-
chroot environment.



  -jf

-- 
To know recursion, you must first know recursion.



From rgerhards at hq.adiscon.com  Fri Nov 21 18:43:37 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 18:43:37 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com><alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm><4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D7@grfint2.intern.adiscon.com>

Just a quick note:

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Friday, November 21, 2008 6:35 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> I'm an itsy bit worried
> that
>       something like this:
> 
>         $template PerAppLogs,"/var/log/apps/%programname%.log"
>         *.* -?PerAppLogs
> 
>       could be tricked to write to unexpected places if you mess with
>       %programname%, or other similar variables that the sender has
> complete
>       controle over.

Definitely. Though not a complete cure, the secpath-* property replacer
options prevent at least slashes inside programname. I know this is not
100% bullet proof, but it should be applied. Otherwise, you don't need a
malicious user, a simple program error is sufficient to screw up things.

So the template is suggested to be

           $template
PerAppLogs,"/var/log/apps/%programname:::secpath-replace%.log"

>From the doc:
---
secpath-replace 	Replace slashes inside the field by an
underscore. (e.g. "a/b" becomes "a_b"). Useful for secure pathname
generation (with dynafiles).
---

Full source at http://www.rsyslog.com/doc-property_replacer.html

I just thought I mention it.

Rainer


From janfrode at tanso.net  Fri Nov 21 19:23:54 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 19:23:54 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6D7@grfint2.intern.adiscon.com>
Message-ID: <slrngidv5q.jbu.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>
> Definitely. Though not a complete cure, the secpath-* property replacer
> options prevent at least slashes inside programname. I know this is not
> 100% bullet proof, but it should be applied. Otherwise, you don't need a
> malicious user, a simple program error is sufficient to screw up things.


Oh, I didn't know about this one. Thanks!


  -jf



From aoz.syn at gmail.com  Fri Nov 21 20:08:51 2008
From: aoz.syn at gmail.com (RB)
Date: Fri, 21 Nov 2008 12:08:51 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>

On Fri, Nov 21, 2008 at 10:34, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> I think the chroot() limits the possibility of:
>
>    - executing anything outside the chroot (f.ex. /bin/sh)
>    - write a new file, and execute it (noexec on the chroot mountpoint)
>    - overwrite something outside the chroot. I'm an itsy bit worried that
>      something like this:

You are absolutely correct - an unbroken chroot() provides a security
boundary that prevents the process[es] therein from accessing
filesystem resources outside that boundary.  Permissions on the
directory structure at & below that point can provide additional
restrictions (like noexec).

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment where there are no other executables, than on a non-
> chroot environment.

Argue what you like, but your basis is flawed.  You seem to forget
that most exploits bring their own executable [shell]code with them
and often operate purely in-memory.  Therefore they don't strictly
need external executables; the lack thereof is more of a nuisance than
a roadblock.

System designers _must_ assume that any successful remote attacker may
access anything the application can, with the same permissions and
executing arbitrary code - chroot or not.  If the data you are logging
is sufficiently sensitive that no client should be able to read the
data and any possibility of their doing so would be catastrophic, you
are going to have to either establish a one-way transport out of that
syslog chroot() or limit your scope to the point that remote attackers
are no longer a possibility.


RB


From janfrode at tanso.net  Sat Nov 22 12:17:24 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Sat, 22 Nov 2008 12:17:24 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>
Message-ID: <slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, RB <aoz.syn at gmail.com> wrote:
>
>> so I would argue that my sensitive syslogged data is more secure in a
>> chroot environment where there are no other executables, than on a non-
>> chroot environment.
>
> Argue what you like, but your basis is flawed.  You seem to forget
> that most exploits bring their own executable [shell]code with them
> and often operate purely in-memory.  Therefore they don't strictly
> need external executables; the lack thereof is more of a nuisance than
> a roadblock.

Did you see the comment from Rainer about the secpath-replace property? 
I think that proves my basis is not flawed. The chroot here protects
against code flaws, or configuration flaws that could otherwise give
attacker the possibility of overwriting system files.

Also you pointing at "most exploits bring their own executable .. operate
purely in-memory" argument is flawed when I'm arguing for chroot being 
*more* secure. Not 100% secure against all flaws, but *more* secure than
a non-chrooted process.

.. but dropping privileges is higher up on my wishlist, than chroot()..


  -jf

-- 
http://xkcd.com/386/ ;-)



From rsyslog at clark-communications.com  Sat Nov 22 20:18:22 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Sat, 22 Nov 2008 11:18:22 -0800
Subject: [rsyslog] rsyslog support for other chrooted apps
Message-ID: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>


NB, in this email, I am *not* referring to any potential chroot  
support within rsyslog itself!

In an OS like OpenBSD, some/many daemons run in a chroot jail, two  
common examples include postfix and named.

The stock syslogd on OpenBSD supports this by a command line option "- 
a" that adds new /dev/log devices.

Thus, on one of my machines, syslogd is run like this:

	syslogd -a /var/spool/postfix/dev/log -a /var/named/dev/log -a /var/ 
empty/dev/log

Which adds the three additional /dev/logs to the standard /dev/log,  
and syslogd reads them all.

In further testing of the OpenBSD port of rsyslog that I posted  
earlier this week,

	http://lists.adiscon.net/pipermail/rsyslog/2008-November/001395.html

I finally realized that it is going to difficult to replace the stock  
OpenBSD syslogd with rsyslogd unless it supports something like this.

Questions,

Does rsyslog support anything like this already?

If not, how difficult would it be to add this, and do people here  
think this is valuable addition, or not?

Don



From rsyslog at clark-communications.com  Sat Nov 22 21:34:07 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Sat, 22 Nov 2008 12:34:07 -0800
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>


On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
>    1 - local syslog. Requires privileges to on local devices (/dev/ 
> log,
>        /dev/klogd or similar), write to local log-files, and send to
>        remote log server.
>
>    2 - central log server. Only listen on the needed network ports
>        (514/udp/tcp), and write to local log files (possibly also send
>        to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot  
> more
> sensitive data than #1, so I think this server needs to be protected  
> as
> much as possible. If chroot'ing, or dropping privileges prevents it  
> from
> reading from local /proc og /dev, I think that wouldn't matter much.  
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

Yes, your two modes as defined above are exactly how I run syslog, and  
I think the distinction between the two use cases
local-syslog and central-log-server is very useful and important in  
this discussion.

And I agree with your prioritization of security between the two  
modes, the local-syslog does not need to open a network port
to listen for log messages, and so it is only at risk from other  
processes on the same box.

The central-log-server does need to open and read a network port, and  
needs to be open to lots of devices on the network, and thus it is at  
the most risk.
That is the version that most needs privilege separation and chroot.   
And I agree that the server that runs the central-log-server could  
also run an instance of the local-syslog, the local-syslog would  
handle /dev/log on that machine, and forward msgs to be centrally  
logged onto the central-log-server running on the same machine.

 From another sub-thread:

> On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:

> Yes, the traditional HUP behavior is simply incompatible with dropping
> privileges. But I assume that someone who configures rsyslogd in  
> such a
> way knows what he does and thus changes config-reload scripts away  
> from
> HUP to a real reload.

I am not sure about the details of the implementation, but named on  
OpenBSD supports privilege separation, and it provides
some of your HUP functionality, although possibly in different ways:

Here is how named looks from ps:

	root     	23404  0.0  0.0  2148   880 ??  Is     2:19PM    0:00.00  
named: [priv] (named)
	named    21080  0.0  0.2  8744  9452 ??  I      2:19PM    0:00.24  
named -4

Here is what the named man page says about HUP:

   SIGNALS
        In routine operation, signals should not be used to con-
        trol the nameserver; rndc should be used instead.

        SIGHUP Force a reload of the server.

        SIGINT, SIGTERM
               Shut down the server.

        The result of sending any other signals to the server is
        undefined.

And, as stated above, rndc is the preferred way of controlling named  
(asking it to reload zone files, etc.)

The config file for rsyslogd could/should reside within the chroot, so  
it should always be accessible.


On yet another sub-thread:

On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
>  I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested  
> features
> done 100% correct.

It is generally difficult to add security ex post facto, and the more  
code that is written prior to the implementation of those design  
changes will tend to increase the difficultly/cost of that  
implementation.

On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

I don't know what the author's goals for rsyslog are, so it is  
difficult to prioritize.

What *I* would like rsyslog to be is the clear choice to replace  
syslog on all the machines in my network, on all the OSes I care about  
(OpenBSD, Solaris, FreeBSD, etc),
basically "total world domination of the syslog space" :-)

I would ask Ranier and others here to reflect on the need/importance  
of security vis-a-vis their ultimate goals for rsyslog, and if  
security is going to be important (I think it is crucial, but you need  
to come to that conclusion yourself), and the cost of implementing  
security design changes is going to increase over time, to consider  
making those design changes sooner rather than later.

In closing, I am learning a lot and enjoying the thoughtful and  
patient responses to my original email on this topic, kudos to the  
rsyslog community, it is a welcome change from some other mailing  
lists :-)

Best regards,

Don















From aoz.syn at gmail.com  Sat Nov 22 23:49:08 2008
From: aoz.syn at gmail.com (RB)
Date: Sat, 22 Nov 2008 15:49:08 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>
	<slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>

On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> Did you see the comment from Rainer about the secpath-replace property?
> I think that proves my basis is not flawed. The chroot here protects
> against code flaws, or configuration flaws that could otherwise give
> attacker the possibility of overwriting system files.

I most certainly did, but all the secpath-replace property protects
against is an administrator foolishly trusting unvalidated data from a
remote host to generate file paths.  Chroot does not protect against
code flaws, full stop.  That is the continuing flaw in your argument.
Chroot only prevents code flaws from affecting anything _outside_ of
the chroot environment.

> Also you pointing at "most exploits bring their own executable .. operate
> purely in-memory" argument is flawed when I'm arguing for chroot being
> *more* secure. Not 100% secure against all flaws, but *more* secure than
> a non-chrooted process.

You mistake what I am arguing - there is no question that chroot()
helps protect the host system's files.  What it does _not_ protect
against is abuse of anything within the chroot().  Your statement was:

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment

If your log data is being stored inside that chroot, you are utterly
wrong.  Please actually understand what chroot() does before you
continue making wild, unfounded statements.


From rgerhards at hq.adiscon.com  Mon Nov 24 11:08:16 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:08:16 +0100
Subject: [rsyslog] rsyslog support for other chrooted apps
In-Reply-To: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>
References: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DA@grfint2.intern.adiscon.com>

Hi Don,

I guess you were lost in the docs (it needs to be restrucuted some
day... I begun with that, but that didn't yet result in improved ease of
use, to phrase it politely ;)).

You use case is described in the relevant module's doc:

http://www.rsyslog.com/doc-imuxsock.html

Please let me know if that solves the issue.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Don Jackson
> Sent: Saturday, November 22, 2008 8:18 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog support for other chrooted apps
> 
> 
> NB, in this email, I am *not* referring to any potential chroot
> support within rsyslog itself!
> 
> In an OS like OpenBSD, some/many daemons run in a chroot jail, two
> common examples include postfix and named.
> 
> The stock syslogd on OpenBSD supports this by a command line option "-
> a" that adds new /dev/log devices.
> 
> Thus, on one of my machines, syslogd is run like this:
> 
> 	syslogd -a /var/spool/postfix/dev/log -a /var/named/dev/log -a
> /var/
> empty/dev/log
> 
> Which adds the three additional /dev/logs to the standard /dev/log,
> and syslogd reads them all.
> 
> In further testing of the OpenBSD port of rsyslog that I posted
> earlier this week,
> 
> 	http://lists.adiscon.net/pipermail/rsyslog/2008-
> November/001395.html
> 
> I finally realized that it is going to difficult to replace the stock
> OpenBSD syslogd with rsyslogd unless it supports something like this.
> 
> Questions,
> 
> Does rsyslog support anything like this already?
> 
> If not, how difficult would it be to add this, and do people here
> think this is valuable addition, or not?
> 
> Don
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dhaivat.desai at yahoo.com  Mon Nov 24 11:23:57 2008
From: dhaivat.desai at yahoo.com (Dhaivat Desai)
Date: Mon, 24 Nov 2008 15:53:57 +0530 (IST)
Subject: [rsyslog]  defining own input module in rsyslog
Message-ID: <10144.52888.qm@web94909.mail.in2.yahoo.com>

Hi,

I am using imuxsock module of rsyslog.
I want to make a new module with some changes in the existing one.
in imuxsock.c the macros and callbacks are very confusing.
are there any steps i need to follow for developing my own module working with RSyslog?

i want to change imuxsock module work with stream protocol. as current implementation doesnt support binary message logging, i want to add binary message logging also in the same.

moreover, in the rsyslog.config i want to add one more directive $BINARYFILEPATH which specifies the pathname for binary file which all the binary messages will be logged into.

what changes will i require to get my code working?

Thanks,
Dhaivat....









      Add more friends to your messenger and enjoy! Go to http://messenger.yahoo.com/invite/

From rgerhards at hq.adiscon.com  Mon Nov 24 11:41:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:41:25 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com><alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm><4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com><slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com><4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com><slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DC@grfint2.intern.adiscon.com>

Hi all,

please let me jump into the middle of that discussion.

Maybe it helps if someone else comments. From my (external) point of
view, it looks like your arguments hit more or less the same targets,
but it also looks like you seem to apply different co-notations to
things. So let me try to formalize things a bit.

Let me think of security as a probability of security breach. S_curr is
the security of the reference system without a root jail. S_total is the
security of a hypothetical system that is "totally secure" (knowing well
that no such system exists). In other words, the probability S_total
equals 0.

I think the common ground is that a root jail does not worsen security.
Note that I do not say it improves security, only that it does not
reduce a system's security. S_jail is the security of a system that is
otherwise identical to the reference system, but with a root jail. Than
S_jail <= s_curr, because we assume that the security of the system is
not reduced.

I think it is also common ground that the probability of a security
breach is reduced if the number of attack vectors is reduced, without
any new attack vectors being added. [There is one generic "attack
vector", the "thought of being secure and thus becoming careless" which
always increases as risk is reduced - I will not include that vector in
my thoughts]

We seem to be in agreement that a root jail is able to prevent some
attacks from being successful. I can't enumerate them and it is probably
useless to try to do so (because attackers invent new attacks each day),
but there exist some attacks which can be prevented by a root jail. I do
not try to weigh them by their importance.

For obvious reasons, there exist other attacks which are not affected by
the root jail. Some of them have been mentions, like the class of
in-memory based attacks, code injection and many more. 

I tend to think that the set of attack vectors that can be prevented by
a root jail is much smaller than the set of those which can not. I also
tend to think that the later class contains the more serious attack
vectors.

But even then, a root jail seems to remove a subset of the attack
vectors that otherwise exists and so it reduces the probably of security
breach. So it benefits security. We can only argue that it does not
benefit security if we can show that in all cases we can think of (and
those we can not), security is not improved. However, some cases have
been show, where it improves, so it can not be that security is not
improved in all cases. As such, a root jail improves security, or more
precisely the probability of a security breach is

0 < S_jail < S_curr

We can identify the benefit we gain is the difference between the
reference system's probability of security breach and the system with
the jail. Be S_impr this improvement, than

S_impr = S_curr - S_jail

Now the root jail is just one potential security measure. We could now
try to calculate S_impr for all kinds of security measures, for example
a privilege drop. I find it hard to do the actual probability
calculations, but I would guess that S_impr_privdop > S_impr_jail.

Based on the improvements, one may finally decide what to implement
first (either at the code or admin level), all of this of course
weighted with the importance of the numbers.

In any case, I think I have show that both is correct:

- the root jail is a security improvement
- there exist numerous other improvements, many of them probably more
efficient than the jail

And if I got you right, I think your arguments meant exactly this ;)

For rsyslog as a project, this also means we need to weigh security
measures against functionality. Based on this discussion, I wonder if it
would be a useful undertaking to try document at least the security
options at hand and try to provide some helpful notes for those that are
not so deeply knowledgeable about these issues (including me, someone
who is always surprised by the numerous ways people find to circumvent
what one has considered to be secure ;)).

Feedback is appreciated.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of RB
> Sent: Saturday, November 22, 2008 11:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust
<janfrode at tanso.net>
> wrote:
> > Did you see the comment from Rainer about the secpath-replace
> property?
> > I think that proves my basis is not flawed. The chroot here protects
> > against code flaws, or configuration flaws that could otherwise give
> > attacker the possibility of overwriting system files.
> 
> I most certainly did, but all the secpath-replace property protects
> against is an administrator foolishly trusting unvalidated data from a
> remote host to generate file paths.  Chroot does not protect against
> code flaws, full stop.  That is the continuing flaw in your argument.
> Chroot only prevents code flaws from affecting anything _outside_ of
> the chroot environment.
> 
> > Also you pointing at "most exploits bring their own executable ..
> operate
> > purely in-memory" argument is flawed when I'm arguing for chroot
> being
> > *more* secure. Not 100% secure against all flaws, but *more* secure
> than
> > a non-chrooted process.
> 
> You mistake what I am arguing - there is no question that chroot()
> helps protect the host system's files.  What it does _not_ protect
> against is abuse of anything within the chroot().  Your statement was:
> 
> > so I would argue that my sensitive syslogged data is more secure in
a
> > chroot environment
> 
> If your log data is being stored inside that chroot, you are utterly
> wrong.  Please actually understand what chroot() does before you
> continue making wild, unfounded statements.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 11:46:30 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:46:30 +0100
Subject: [rsyslog] defining own input module in rsyslog
In-Reply-To: <10144.52888.qm@web94909.mail.in2.yahoo.com>
References: <10144.52888.qm@web94909.mail.in2.yahoo.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DD@grfint2.intern.adiscon.com>

Hi,

I suggest that you have a look at the imtemplate modul, which was
specifically written to help understand the interface. Please come back
once you have reviewed it. I would also appreciate if you could do that
as a web-based forum thread, because that would make it easier for me to
refer other people with the same intention to that thread. The forum
link is

http://www.rsyslog.com/forum

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dhaivat Desai
> Sent: Monday, November 24, 2008 11:24 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] defining own input module in rsyslog
> 
> Hi,
> 
> I am using imuxsock module of rsyslog.
> I want to make a new module with some changes in the existing one.
> in imuxsock.c the macros and callbacks are very confusing.
> are there any steps i need to follow for developing my own module
> working with RSyslog?
> 
> i want to change imuxsock module work with stream protocol. as current
> implementation doesnt support binary message logging, i want to add
> binary message logging also in the same.
> 
> moreover, in the rsyslog.config i want to add one more directive
> $BINARYFILEPATH which specifies the pathname for binary file which all
> the binary messages will be logged into.
> 
> what changes will i require to get my code working?
> 
> Thanks,
> Dhaivat....
> 
> 
> 
> 
> 
> 
> 
> 
> 
>       Add more friends to your messenger and enjoy! Go to
> http://messenger.yahoo.com/invite/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 12:55:00 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 12:55:00 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>
Message-ID: <1227527700.5282.21.camel@rgf9dev.intern.adiscon.com>

Hi Don,

thanks for your careful thoughts. I would also appreciate if you could
try out the experimental branch, but on the other hand I think I will be
able to create a regular devel branch based on some of the discussion
this week.

On Sat, 2008-11-22 at 12:34 -0800, Don Jackson wrote:
> On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:
> 
> > For my usage, I need two modes of operation for syslog daemons.
> >
> >    1 - local syslog. Requires privileges to on local devices (/dev/ 
> > log,
> >        /dev/klogd or similar), write to local log-files, and send to
> >        remote log server.
> >
> >    2 - central log server. Only listen on the needed network ports
> >        (514/udp/tcp), and write to local log files (possibly also send
> >        to other remote log servers).
> >
> > For #1 I think it's OK not being able to chroot, keep more privileges,
> > etc.,  as the attacks against it will mostly be from local processes.
> >
> > #2 needs to be *very* openly available on the network ports, since all
> > my servers needs to send logs to it. #2 will also be holding a lot  
> > more
> > sensitive data than #1, so I think this server needs to be protected  
> > as
> > much as possible. If chroot'ing, or dropping privileges prevents it  
> > from
> > reading from local /proc og /dev, I think that wouldn't matter much.  
> > One
> > could always run two instances on these few central servers, i.e. #1
> > sending to #2.
> 
> Yes, your two modes as defined above are exactly how I run syslog, and  
> I think the distinction between the two use cases
> local-syslog and central-log-server is very useful and important in  
> this discussion.

Even further, I think this could be different (but compatible) use cases
in a to-be-written rsyslog security doc.

> 
> And I agree with your prioritization of security between the two  
> modes, the local-syslog does not need to open a network port
> to listen for log messages, and so it is only at risk from other  
> processes on the same box.
> 
> The central-log-server does need to open and read a network port, and  
> needs to be open to lots of devices on the network, and thus it is at  
> the most risk.
> That is the version that most needs privilege separation and chroot.   
> And I agree that the server that runs the central-log-server could  
> also run an instance of the local-syslog, the local-syslog would  
> handle /dev/log on that machine, and forward msgs to be centrally  
> logged onto the central-log-server running on the same machine.
> 
>  From another sub-thread:
> 
> > On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:
> 
> > Yes, the traditional HUP behavior is simply incompatible with dropping
> > privileges. But I assume that someone who configures rsyslogd in  
> > such a
> > way knows what he does and thus changes config-reload scripts away  
> > from
> > HUP to a real reload.
> 
> I am not sure about the details of the implementation, but named on  
> OpenBSD supports privilege separation, and it provides
> some of your HUP functionality, although possibly in different ways:
[snip]

It would be too much info to go in full detail, but HUP is ugly and it
is a full restart. I'd like to phase out that mode, but will probably
need to keep it for "admin compatibility". Anyhow, I do not see any
issue if, in a secure system, HUP does not work as a full restart. (One
way to make it work would be to have a master running all the time and
the actual rsyslogd being a child where the parent handles HUP and
restarts the "real" rsyslogd - but I don't like to make it so compliated
"just" to satisfy old habits... [who likes this can always contribute
the code ;)]).

> On yet another sub-thread:
> 
> On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
> >  I also agree to David Lang's argument that some
> > degree of design change would be necessary to get the requested  
> > features
> > done 100% correct.
> 
> It is generally difficult to add security ex post facto, and the more  
> code that is written prior to the implementation of those design  
> changes will tend to increase the difficultly/cost of that  
> implementation.

That's totally right, and rsyslog is a good example. First of all it is
important to remember that it was not designed from scratch. We forked
it off from sysklogd. And while almost no sysklogd code remains today,
we had a long chain of design decisions which were based on the need to
keep compatible with sysklogd. If you traverse that chain, you'll see
why some things are as they are. 

The security approach was not to be worse than sysklogd and better where
possible, but there always was a strong focus on deliverving something.
You may argue that it is foolish not doing the "right thing" right from
the beginning. My view is that we live in an imperfect (and risky) world
and it is important to get some things done. So I don't want to limit
myself in using a better solution because that better solution is not
better in some aspect than the previous solution, which I would be using
otherwise. 

In somewhat more formal words. If sysklogd's number if features is
F_sysklogd and it's probably of security breach is S_sysklog and the
exact same concepts for rsyslog are F_rsyslog and S_rsyslog, then I
don't see any reason not to use rsyslog over sysklogd if F_rsyslog >
F_sysklogd and R_rsyslog <= R_sysklogd. In even other words, if I would
not use rsyslog because it is not as secure as sysklogd, I would ditch
it without getting any benefit and I would also not get the benefits
from the new features it offers.

One may of course argue that rsyslog's addition feature inherently
increase the risk of using it, in which case S_rsyslog < S_sysklogd. The
only argument I have against this is that you can disable most of these
new features so that you can limit the potential risk exposure.

In any case, you are right that the ultimate goal should be to keep as
secure *as possible* and this is why I immediately followed up on your
questions. 

But I think it is still very important to not let security be a show
stopper in a case if it is not worsened. I think if I had started the
rsyslog project with a totally secure design, rsyslog would be obsolete
by now because anybody would already have moved over to syslog-ng and no
other solution would have a chance to rise.

I am not saying that syslog-ng is a bad project, nor am I saying it is
less secure than rsyslog (right now, it probably is more secure). But a
world with just syslog-ng (or one with just rsyslog!) is a monoculture a
monoculture has inherent and very large security risk.

I sincerely believe (bash me) that Windows is *not* bad software (or at
least not worse than Linux). It "just" is (well, "was" ;)) a monoculture
and that lead to much more interest in attacking it. The financial
benefit of attacking it was much greater than the benefit of attacking
Linux.

If you look from that angle (and I do), you'll find that a world with a
not-so-secure rsyslog AND syslog-ng is more secure than a world with a
totally-secure rsyslog (that nobody uses) and and always-present
syslog-ng (being a monoculture on the vast majority of systems). So less
security in a "subsystem" brought more security to the overall picture.

Still, I strongly favor an as-secure-as possible rsyslog, but I need to
abide to the constraints given (and with the financial crises and loss
of sponsorship these constraints for the rsyslog project unfortunately
have not become more permitting...).

> 
> On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> > FWIW, while I believe this discussion is relevant and the security
> > changes proposed are important, I don't see this as a high priority
> > feature. The scripting language/syntax, greater performance, continued
> > stability enhancements, and true RELP support are all far more
> > important. As far as security goes, I would much rather see two-way
> > host authentication than a chroot/unprivilieged framework implemented.
> 
> I don't know what the author's goals for rsyslog are, so it is  
> difficult to prioritize.
> 
> What *I* would like rsyslog to be is the clear choice to replace  
> syslog on all the machines in my network, on all the OSes I care about  
> (OpenBSD, Solaris, FreeBSD, etc),
> basically "total world domination of the syslog space" :-)

as said above, "total world dominition" is sweet fruit, but it has a big
risk associated with it...

> 
> I would ask Ranier and others here to reflect on the need/importance  
> of security vis-a-vis their ultimate goals for rsyslog, and if  
> security is going to be important (I think it is crucial, but you need  
> to come to that conclusion yourself), and the cost of implementing  
> security design changes is going to increase over time, to consider  
> making those design changes sooner rather than later.

I hope I outlined these thoughts above. Please keep asking/suggesting.
My mind is always open to change and I like good arguments ;)

Rainer
> 
> In closing, I am learning a lot and enjoying the thoughtful and  
> patient responses to my original email on this topic, kudos to the  
> rsyslog community, it is a welcome change from some other mailing  
> lists :-)
> 
> Best regards,
> 
> Don
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From pete.philips at secerno.com  Mon Nov 24 13:36:35 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 12:36:35 +0000
Subject: [rsyslog] Logging lost TCP connections
Message-ID: <492A9FD3.2040700@secerno.com>

Hi.

I've googled this extensively but can can't find a solution. Please could anyone tell
me if there is an option to enable logging of lost TCP connections? I'd like the
event recorded locally if contact is lost with a remote TCP syslog server.

Many thanks.


Pete Philips.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 14:36:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 14:36:44 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492A9FD3.2040700@secerno.com>
References: <492A9FD3.2040700@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>

Hi Pete,

I think there is no such option, but I will check with the code.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 1:37 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Logging lost TCP connections
> 
> Hi.
> 
> I've googled this extensively but can can't find a solution. Please
> could anyone tell
> me if there is an option to enable logging of lost TCP connections?
I'd
> like the
> event recorded locally if contact is lost with a remote TCP syslog
> server.
> 
> Many thanks.
> 
> 
> Pete Philips.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 15:17:38 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:17:38 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>

Hi Pete,

I checked, but this information is only available in the debug log.
Having said that, it is probably not hard to add it. What exactly would
you like to see (e.g. complete closes or just aborted ones)?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, November 24, 2008 2:37 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Hi Pete,
> 
> I think there is no such option, but I will check with the code.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Pete Philips
> > Sent: Monday, November 24, 2008 1:37 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Logging lost TCP connections
> >
> > Hi.
> >
> > I've googled this extensively but can can't find a solution. Please
> > could anyone tell
> > me if there is an option to enable logging of lost TCP connections?
> I'd
> > like the
> > event recorded locally if contact is lost with a remote TCP syslog
> > server.
> >
> > Many thanks.
> >
> >
> > Pete Philips.
> > --
> > Pete Philips
> > Secerno Ltd
> > Email: pete.philips at secerno.com
> > PGP key: http://www.secerno.com/pgp/pete.gpg
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:27:02 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:27:02 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
Message-ID: <492AB9B6.4040805@secerno.com>

Rainer,

Thanks for your help on this.

Rainer Gerhards wrote:
> I checked, but this information is only available in the debug log.
> Having said that, it is probably not hard to add it. What exactly would
> you like to see (e.g. complete closes or just aborted ones)?

I'm not sure of the exact terminology but I'd like it to log a record locally everytime
it becomes not possible to send a TCP syslog message to a remote server. Perhaps if
this would be useful to others then it could be added to a future release?

Thanks.


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 15:29:45 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:29:45 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492AB9B6.4040805@secerno.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
	<492AB9B6.4040805@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 3:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Rainer,
> 
> Thanks for your help on this.
> 
> Rainer Gerhards wrote:
> > I checked, but this information is only available in the debug log.
> > Having said that, it is probably not hard to add it. What exactly
> would
> > you like to see (e.g. complete closes or just aborted ones)?
> 
> I'm not sure of the exact terminology but I'd like it to log a record
> locally everytime
> it becomes not possible to send a TCP syslog message to a remote
> server. Perhaps if
> this would be useful to others then it could be added to a future
> release?

Ummm... I was on the wrong side, checking the receiver. So you would
like to log if the send does not succeed?

Which brings up the question: do you have reasons for accepting the
message loss? I am asking because you can instruct rsyslog to retry
sending the data when the remote server is ready again. For an example,
see here:

http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html

Rainer
> 
> Thanks.
> 
> 
> Pete.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:44:02 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:44:02 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
Message-ID: <492ABDB2.2040001@secerno.com>

Rainer Gerhards wrote:
> Ummm... I was on the wrong side, checking the receiver. So you would
> like to log if the send does not succeed?

Yes exactly.

> Which brings up the question: do you have reasons for accepting the
> message loss? I am asking because you can instruct rsyslog to retry
> sending the data when the remote server is ready again. For an example,
> see here:
> 
> http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html

I've seen that page and would like to use the disk buffering capability of
rsyslog to make logging failures unnecessary. Unfortunately the spec for the system
I am delivering requires logging of failed delivery attempts and not disk
buffering :-(


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 15:45:32 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:45:32 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492ABDB2.2040001@secerno.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
	<492ABDB2.2040001@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>

OK, I will look into that, but I can not promise how trivial this may be
;) I keep you posted.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 3:44 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Rainer Gerhards wrote:
> > Ummm... I was on the wrong side, checking the receiver. So you would
> > like to log if the send does not succeed?
> 
> Yes exactly.
> 
> > Which brings up the question: do you have reasons for accepting the
> > message loss? I am asking because you can instruct rsyslog to retry
> > sending the data when the remote server is ready again. For an
> example,
> > see here:
> >
> > http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html
> 
> I've seen that page and would like to use the disk buffering
capability
> of
> rsyslog to make logging failures unnecessary. Unfortunately the spec
> for the system
> I am delivering requires logging of failed delivery attempts and not
> disk
> buffering :-(
> 
> 
> Pete.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 15:49:57 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:49:57 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com><492ABDB2.2040001@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6EA@grfint2.intern.adiscon.com>

Well... I was too quick. The issue is that no matter what we do, we
cannot simply log failed *tcp sends*. The reason is that actions fail
some place inside the rule engine. At the point where it fails, it is
unaware if it is a TCP send, file store, sql store, etc, etc, ... Of
course the tcp output plugin can emit the failure message, but the tcp
output plugin does not know if this is an ultimate failure or not - this
depends on the rule engine configuration.

So if anything is set to report errors, you can either instrument the
plugin and live with potentially false positives OR you can instrument
the rule engine and generate messages in any instance (well, we could
create an action-specific failure notice, which probably can be
configured to what you exactly wanted to have - probably the
solution...).

In any case, feedback from your side is appreciated.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, November 24, 2008 3:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> OK, I will look into that, but I can not promise how trivial this may
> be
> ;) I keep you posted.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Pete Philips
> > Sent: Monday, November 24, 2008 3:44 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Logging lost TCP connections
> >
> > Rainer Gerhards wrote:
> > > Ummm... I was on the wrong side, checking the receiver. So you
> would
> > > like to log if the send does not succeed?
> >
> > Yes exactly.
> >
> > > Which brings up the question: do you have reasons for accepting
the
> > > message loss? I am asking because you can instruct rsyslog to
retry
> > > sending the data when the remote server is ready again. For an
> > example,
> > > see here:
> > >
> > > http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html
> >
> > I've seen that page and would like to use the disk buffering
> capability
> > of
> > rsyslog to make logging failures unnecessary. Unfortunately the spec
> > for the system
> > I am delivering requires logging of failed delivery attempts and not
> > disk
> > buffering :-(
> >
> >
> > Pete.
> > --
> > Pete Philips
> > Secerno Ltd
> > Email: pete.philips at secerno.com
> > PGP key: http://www.secerno.com/pgp/pete.gpg
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:51:14 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:51:14 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>	<492ABDB2.2040001@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
Message-ID: <492ABF62.2060809@secerno.com>

Rainer Gerhards wrote:
> OK, I will look into that, but I can not promise how trivial this may be
> ;) I keep you posted.

Thanks very much for all your help.

Regards,


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Wed Nov 26 12:20:48 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 12:20:48 +0100
Subject: [rsyslog] rsyslog documentation
Message-ID: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>

Hi all,

as you probably know, I keep rsyslog's documentation in html files, with
the man pages containing the bare minimum to make sense of the system.
This need arises from the sheer volume of the information that must be
provided.

However, this makes the documentation pretty closed (everything needs to
go through me) and consequently I have seen very few doc contributions.
It probably also makes it more complicated than needed to translate the
documentation.

Initially, I considered a purely web-based doc vs. a html-file based
doc. I went for the file-based doc as this can easily be distributed
together with every rsyslog version.

However, looking at other projects, many have adapted a web-based
approach where version differences are flagged within the documentation.
The advantage of a web-based doc is that we can use thinks like a wiki
to generate it. That, I think, would make it much easier for users to
contribute doc or at least samples. Also, it looks like a web-based doc
is also more convenient for most users. Everyone has a browser open and
checks the web, but who installs doc packages? ;)

So I am now consider changing to a web-based system, too. I'd probably
consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
point. Since I created it, it receives a slow but steady traffic
increase and has now around the same number of hits than the rsyslog
main site. I would move over the current file-based doc into that system
and at the same time see that I can improve the structure and usability
of the doc.

With the many new and powerful features that appeared in rsyslog over
the past couple of month, I think it is very important to make them
accessible by a sufficiently good doc. The current one is, to phrase it
politely, not good. It probably even hinders adoption of rsyslog in some
cases.

With a web-based system open to user contributions I hope to solve this
issues.

Please let me know your thoughts. All feedback is deeply appreciated.

Many thanks,
Rainer Gerhards



From mbiebl at gmail.com  Wed Nov 26 13:10:38 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 26 Nov 2008 13:10:38 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
Message-ID: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>

2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>
> However, looking at other projects, many have adapted a web-based
> approach where version differences are flagged within the documentation.
> The advantage of a web-based doc is that we can use thinks like a wiki
> to generate it. That, I think, would make it much easier for users to
> contribute doc or at least samples. Also, it looks like a web-based doc
> is also more convenient for most users. Everyone has a browser open and
> checks the web, but who installs doc packages? ;)
>
> So I am now consider changing to a web-based system, too. I'd probably
> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
> point. Since I created it, it receives a slow but steady traffic
> increase and has now around the same number of hits than the rsyslog
> main site. I would move over the current file-based doc into that system
> and at the same time see that I can improve the structure and usability
> of the doc.
>
> With the many new and powerful features that appeared in rsyslog over
> the past couple of month, I think it is very important to make them
> accessible by a sufficiently good doc. The current one is, to phrase it
> politely, not good. It probably even hinders adoption of rsyslog in some
> cases.
>
> With a web-based system open to user contributions I hope to solve this
> issues.
>
> Please let me know your thoughts. All feedback is deeply appreciated.

Imho it would be very unfortunate, if the documentation would only be
available online.

A wiki can be a very good *additional* source of documentation (for
stuff like best practices, tips and tricks), but it doesn't substitute
a well written manual.

What I would rather like to see (and I know, I'm repeating myself
here), is to have the existing documentation a bit more structured and
easier accessible. I posted examples like the exim [1] or postgresql
[2] documentation, which I think are excellent.

The postgresql documentation is interesting. If you follow [2] the
link, you can see, that users are able to add comments, which could be
helpful to improve the overall documentation.
There is still a "static" version though, also available as pdf, which
you could print and even use as a book.

Cheers,
Michael


[1] http://www.exim.org/exim-html-current/doc/html/spec_html/index.html
http://www.exim.org/exim-pdf-current/doc/spec.pdf
[2] http://www.postgresql.org/docs/8.3/interactive/index.html
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Wed Nov 26 13:16:02 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 26 Nov 2008 13:16:02 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>

2008/11/26 Michael Biebl <mbiebl at gmail.com>:
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.

And distribute easily with the rsyslog release tarball

Cheers,
Michael

P.S: Maybe a book could be an additional source of income.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 26 13:28:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 13:28:25 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>

Hi Michael,

thanks for the feedback.

On Wed, 2008-11-26 at 13:10 +0100, Michael Biebl wrote:
> Imho it would be very unfortunate, if the documentation would only be
> available online.
> 
> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.
> 
> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.

I think the PHP manual is also a good sample.

But actually this doesn't solve my main point: It is quite
time-consuming to create a good doc. I am ready to put a bit more time
into that (moving that away from development), but I would like to
encourage user contributions. Without bashing, it is one thing to show a
good example, but it is a totally different one to provide good
content ;)

The bottom line is that I know I could do better, but I have not
sufficient time (and probably not the best talent in that area) to
actually do it.

> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.

This is the main point. Maybe I can find a way to easily make user
comments available to the current doc set. That would help drive in user
contributions (at least I hope so). The problem again is that I simply
can not invest a few weeks of my time "just" to get the right doc
display system done.

It's a trade-off between time available, hoping to get contributions and
quality of doc.

Again, I value your thoughts very much, they go in the same direction of
what I hope to have, but it looks like I currently can not achieve this
and need to lower my demands in order to get working, solution - but one
that is better than the current state...

Rainer



From rgerhards at hq.adiscon.com  Wed Nov 26 13:43:12 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 13:43:12 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
Message-ID: <1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>

Hi all,

some times it seems to help ask around ;) Andre who cares for the web
sites (also the phpLogCon lead devel) has pointed out to me that we may
try the CMS' comment function. We have now enabled that. It seems to
solve the immediate need. So this looks like a way we can at least try
out.

Anyhow, if you have any additional thoughts please let me know them.
They probably help shape a medium-term doc strategy.

Thanks,
Rainer

On Wed, 2008-11-26 at 13:28 +0100, Rainer Gerhards wrote:
> Hi Michael,
> 
> thanks for the feedback.
> 
> On Wed, 2008-11-26 at 13:10 +0100, Michael Biebl wrote:
> > Imho it would be very unfortunate, if the documentation would only be
> > available online.
> > 
> > A wiki can be a very good *additional* source of documentation (for
> > stuff like best practices, tips and tricks), but it doesn't substitute
> > a well written manual.
> > 
> > What I would rather like to see (and I know, I'm repeating myself
> > here), is to have the existing documentation a bit more structured and
> > easier accessible. I posted examples like the exim [1] or postgresql
> > [2] documentation, which I think are excellent.
> 
> I think the PHP manual is also a good sample.
> 
> But actually this doesn't solve my main point: It is quite
> time-consuming to create a good doc. I am ready to put a bit more time
> into that (moving that away from development), but I would like to
> encourage user contributions. Without bashing, it is one thing to show a
> good example, but it is a totally different one to provide good
> content ;)
> 
> The bottom line is that I know I could do better, but I have not
> sufficient time (and probably not the best talent in that area) to
> actually do it.
> 
> > The postgresql documentation is interesting. If you follow [2] the
> > link, you can see, that users are able to add comments, which could be
> > helpful to improve the overall documentation.
> 
> This is the main point. Maybe I can find a way to easily make user
> comments available to the current doc set. That would help drive in user
> contributions (at least I hope so). The problem again is that I simply
> can not invest a few weeks of my time "just" to get the right doc
> display system done.
> 
> It's a trade-off between time available, hoping to get contributions and
> quality of doc.
> 
> Again, I value your thoughts very much, they go in the same direction of
> what I hope to have, but it looks like I currently can not achieve this
> and need to lower my demands in order to get working, solution - but one
> that is better than the current state...
> 
> Rainer
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From rfujita at redhat.com  Wed Nov 26 13:19:03 2008
From: rfujita at redhat.com (=?ISO-2022-JP?B?GyRCTkcbKEIgGyRCRiNFRBsoQg==?=)
Date: Wed, 26 Nov 2008 21:19:03 +0900
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>
Message-ID: <E5D6C192-A983-4807-8B63-686A2D61A018@redhat.com>

Hi,

> P.S: Maybe a book could be an additional source of income.

+1

Best Rio.

On 2008/11/26, at 21:16, Michael Biebl wrote:

> 2008/11/26 Michael Biebl <mbiebl at gmail.com>:
>>
>> The postgresql documentation is interesting. If you follow [2] the
>> link, you can see, that users are able to add comments, which could  
>> be
>> helpful to improve the overall documentation.
>> There is still a "static" version though, also available as pdf,  
>> which
>> you could print and even use as a book.
>
> And distribute easily with the rsyslog release tarball
>
> Cheers,
> Michael
>
> P.S: Maybe a book could be an additional source of income.
>
>
> -- 
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################



From rfujita at redhat.com  Wed Nov 26 13:18:34 2008
From: rfujita at redhat.com (=?ISO-2022-JP?B?GyRCTkcbKEIgGyRCRiNFRBsoQg==?=)
Date: Wed, 26 Nov 2008 21:18:34 +0900
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <0C7538F5-2324-4EB7-9A55-02E28965ABDA@redhat.com>

Hi all,

> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/ 
> index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html

Both these links are made with DocBook.

Best Rio.

On 2008/11/26, at 21:10, Michael Biebl wrote:

> 2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> However, looking at other projects, many have adapted a web-based
>> approach where version differences are flagged within the  
>> documentation.
>> The advantage of a web-based doc is that we can use thinks like a  
>> wiki
>> to generate it. That, I think, would make it much easier for users to
>> contribute doc or at least samples. Also, it looks like a web-based  
>> doc
>> is also more convenient for most users. Everyone has a browser open  
>> and
>> checks the web, but who installs doc packages? ;)
>>
>> So I am now consider changing to a web-based system, too. I'd  
>> probably
>> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
>> point. Since I created it, it receives a slow but steady traffic
>> increase and has now around the same number of hits than the rsyslog
>> main site. I would move over the current file-based doc into that  
>> system
>> and at the same time see that I can improve the structure and  
>> usability
>> of the doc.
>>
>> With the many new and powerful features that appeared in rsyslog over
>> the past couple of month, I think it is very important to make them
>> accessible by a sufficiently good doc. The current one is, to  
>> phrase it
>> politely, not good. It probably even hinders adoption of rsyslog in  
>> some
>> cases.
>>
>> With a web-based system open to user contributions I hope to solve  
>> this
>> issues.
>>
>> Please let me know your thoughts. All feedback is deeply appreciated.
>
> Imho it would be very unfortunate, if the documentation would only be
> available online.
>
> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.
>
> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.
>
> Cheers,
> Michael
>
>
> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/ 
> index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html
> -- 
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################



From mrdemeanour at jackpot.uk.net  Wed Nov 26 13:24:42 2008
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Wed, 26 Nov 2008 12:24:42 +0000
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <492D400A.2010105@jackpot.uk.net>

Michael Biebl wrote:
>> 
>> With a web-based system open to user contributions I hope to solve
>> this issues.
>> 
>> Please let me know your thoughts. All feedback is deeply
>> appreciated.
> 
> Imho it would be very unfortunate, if the documentation would only be
>  available online.

+1.

Many projects now use Docbook, to generate variously HTML, PDF and manpages.

On most of my servers, I have no X GUI set up; HTML docs are therefore a
pain in the neck, unless the HTML is extremely clean (and even then it's
a hassle).

The documentation should be a part of the project, and it should be
possible to check it out, modify it (e.g. translate it) and 'build' it.

I think wiki docs are great, but I also think the foundation for a
documentation wiki should be the authorised version of the docs. Users
can then comment on that authorised version, which should be the docs
that ship with the product. It might be possible to rig some mechanism
for automagically pulling content from the wiki back into the
repository; but I'm not sure how to go about that.

Thanks for this excellent project, and for being so committed to it!
-- 
Jack.


From mikel at irontec.com  Wed Nov 26 16:24:32 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 16:24:32 +0100
Subject: [rsyslog] rsyslog freeze dude
Message-ID: <492D6A30.9060105@irontec.com>

Hello
If I configure rsyslog client like this:

WorkDirectory /root/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName myfile
$ActionResumeRetryCount -1

and

*.* @@somehost.domain.com

Is possible that lost of conectivity (internet down) of this server 
during days, produce that server freeze? or not be possible to login to 
the machine?

If I do /etc/init.d/rsyslog restart it takes long long time to restart.

Please Rainer I hope you help me. thanks


-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 16:39:36 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:39:36 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D6A30.9060105@irontec.com>
References: <492D6A30.9060105@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>

Hi Mikel,

which version are you using? The "ssh hang" can probably be related to
the initial thought that imuxsock messages can be delayed. This has been
changed in recent releases.

Other than that, please note that actions are retried in intervals and
these intervals get longer each time the action is retried without
success. This is done in order to save unnecessary work. I think there
currently is no config setting to change the default intervals.

It may be useful to get a shutdown debug log from such an occurrence.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Wednesday, November 26, 2008 4:25 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog freeze dude
> 
> Hello
> If I configure rsyslog client like this:
> 
> WorkDirectory /root/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueFileName myfile
> $ActionResumeRetryCount -1
> 
> and
> 
> *.* @@somehost.domain.com
> 
> Is possible that lost of conectivity (internet down) of this server
> during days, produce that server freeze? or not be possible to login
to
> the machine?
> 
> If I do /etc/init.d/rsyslog restart it takes long long time to
restart.
> 
> Please Rainer I hope you help me. thanks
> 
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 26 16:54:42 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:54:42 +0100
Subject: [rsyslog] rsyslog 4.1.1 (devel) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F714@grfint2.intern.adiscon.com>

Hi all,

rsyslog 4.1.1, a member of the development branch, has been released
today. Most importantly, it now contains the ability to drop privileges,
thus enabling it to be executed without root permissions even if
listening to privileged network ports. The release also contains a fix
for a memory leak in the postgres output module and a fix for the klog
input module, which did not compile under BSD.

Download:
http://www.rsyslog.com/Article317.phtml

Change Log:
http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-140.phtml

The "privilege drop" functionality was discussed on this list and, while
not perfect, seems to provide some useful service. A number of
restrictions is described here:

http://wiki.rsyslog.com/index.php/Security

I would like to express my gratitude for all opinions raised. If you
have any more comments, please let them flow. Also feel free to add
anything of interest to the Wiki. Obviously, I would be very interested
in feedback from folks who really try the new functionality out.

I hope this release is useful.

Rainer



From mikel at irontec.com  Wed Nov 26 16:54:51 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 16:54:51 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
Message-ID: <492D714B.4050107@irontec.com>

Rainer Gerhards escribi?:
> Hi Mikel,
>
> which version are you using? The "ssh hang" can probably be related to
> the initial thought that imuxsock messages can be delayed. This has been
> changed in recent releases.
>
> Other than that, please note that actions are retried in intervals and
> these intervals get longer each time the action is retried without
> success. This is done in order to save unnecessary work. I think there
> currently is no config setting to change the default intervals.
>
> It may be useful to get a shutdown debug log from such an occurrence.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Wednesday, November 26, 2008 4:25 PM
>> To: rsyslog-users
>> Subject: [rsyslog] rsyslog freeze dude
>>
>> Hello
>> If I configure rsyslog client like this:
>>
>> WorkDirectory /root/rsyslog
>> $ActionQueueType LinkedList
>> $ActionQueueFileName myfile
>> $ActionResumeRetryCount -1
>>
>> and
>>
>> *.* @@somehost.domain.com
>>
>> Is possible that lost of conectivity (internet down) of this server
>> during days, produce that server freeze? or not be possible to login
>>     
> to
>   
>> the machine?
>>
>> If I do /etc/init.d/rsyslog restart it takes long long time to
>>     
> restart.
>   
>> Please Rainer I hope you help me. thanks
>>
>>
>> --
>> Mikel Jimenez Fernandez
>> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
>> +34 94.404.81.82
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Hello Rainer


root at ironpbx:~# rsyslogd -v
rsyslogd 3.16.1, compiled with:
      FEATURE_REGEXP:           Yes
      FEATURE_LARGEFILE:        Yes
      FEATURE_NETZIP (message compression):     Yes
      GSSAPI Kerberos 5 support:                No
      FEATURE_DEBUG (debug build, slow code):   No
      Runtime Instrumentation (slow code):      No

See http://www.rsyslog.com for more information.



Say me if you need more information.

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 16:56:52 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:56:52 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D714B.4050107@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
	<492D714B.4050107@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>

Hi Mikel,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Wednesday, November 26, 2008 4:55 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog freeze dude
 
> root at ironpbx:~# rsyslogd -v
> rsyslogd 3.16.1, compiled with:
>       FEATURE_REGEXP:           Yes
>       FEATURE_LARGEFILE:        Yes
>       FEATURE_NETZIP (message compression):     Yes
>       GSSAPI Kerberos 5 support:                No
>       FEATURE_DEBUG (debug build, slow code):   No
>       Runtime Instrumentation (slow code):      No
> 
> See http://www.rsyslog.com for more information.

Please upgrade to the recent v3-stable. That will probably solve the big
issue, if not all.

Rainer


From mikel at irontec.com  Wed Nov 26 17:03:15 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 17:03:15 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
Message-ID: <492D7343.1040101@irontec.com>

Rainer Gerhards escribi?:
> Hi Mikel,
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Wednesday, November 26, 2008 4:55 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog freeze dude
>>     
>  
>   
>> root at ironpbx:~# rsyslogd -v
>> rsyslogd 3.16.1, compiled with:
>>       FEATURE_REGEXP:           Yes
>>       FEATURE_LARGEFILE:        Yes
>>       FEATURE_NETZIP (message compression):     Yes
>>       GSSAPI Kerberos 5 support:                No
>>       FEATURE_DEBUG (debug build, slow code):   No
>>       Runtime Instrumentation (slow code):      No
>>
>> See http://www.rsyslog.com for more information.
>>     
>
> Please upgrade to the recent v3-stable. That will probably solve the big
> issue, if not all.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
OK

What is the last stable version?

There is a debian package?
Other thing:
The workDirectory is necesary that exist?
For example: WorkDirectory /rsyslog/work is a directory in / called 
rsyslog and another one inside this called work?

If they aren?t can cause a bug?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 17:13:10 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 17:13:10 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D7343.1040101@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
	<492D7343.1040101@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>

> OK
> 
> What is the last stable version?

I checked, it is 3.20.0 (and while checking I noticed that the project status was once again outdated ;))

> 
> There is a debian package?

This I don't know.

> Other thing:
> The workDirectory is necesary that exist?
Yes, absolutely

> For example: WorkDirectory /rsyslog/work is a directory in / called
> rsyslog and another one inside this called work?
> 
> If they aren?t can cause a bug?

I'd say yes.

Rainer


From mikel at irontec.com  Wed Nov 26 17:22:37 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 17:22:37 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
Message-ID: <492D77CD.3000204@irontec.com>

Rainer Gerhards escribi?:
>> OK
>>
>> What is the last stable version?
>>     
>
> I checked, it is 3.20.0 (and while checking I noticed that the project status was once again outdated ;))
>
>   
>> There is a debian package?
>>     
>
> This I don't know.
>
>   
>> Other thing:
>> The workDirectory is necesary that exist?
>>     
> Yes, absolutely
>
>   
>> For example: WorkDirectory /rsyslog/work is a directory in / called
>> rsyslog and another one inside this called work?
>>
>> If they aren?t can cause a bug?
>>     
>
> I'd say yes.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   

Before installing this version, would be possible to do more debugging?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 17:35:13 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 17:35:13 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D77CD.3000204@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
	<492D77CD.3000204@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>

> Before installing this version, would be possible to do more
debugging?

Sorry, I don't want to upset you, but I have no time to look at things
already fixed. Quite seriously, we have paid support options for this. I
see that it can make sense for you to stay with the older version, but
there is absolutely no benefit to the community. Sorry...

Rainer


From david at lang.hm  Wed Nov 26 18:57:05 2008
From: david at lang.hm (david at lang.hm)
Date: Wed, 26 Nov 2008 09:57:05 -0800 (PST)
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <alpine.DEB.1.10.0811260954070.22089@asgard.lang.hm>

On Wed, 26 Nov 2008, Michael Biebl wrote:

> 2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> However, looking at other projects, many have adapted a web-based
>> approach where version differences are flagged within the documentation.
>> The advantage of a web-based doc is that we can use thinks like a wiki
>> to generate it. That, I think, would make it much easier for users to
>> contribute doc or at least samples. Also, it looks like a web-based doc
>> is also more convenient for most users. Everyone has a browser open and
>> checks the web, but who installs doc packages? ;)
>>
>> So I am now consider changing to a web-based system, too. I'd probably
>> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
>> point. Since I created it, it receives a slow but steady traffic
>> increase and has now around the same number of hits than the rsyslog
>> main site. I would move over the current file-based doc into that system
>> and at the same time see that I can improve the structure and usability
>> of the doc.
>>
>> With the many new and powerful features that appeared in rsyslog over
>> the past couple of month, I think it is very important to make them
>> accessible by a sufficiently good doc. The current one is, to phrase it
>> politely, not good. It probably even hinders adoption of rsyslog in some
>> cases.
>>
>> With a web-based system open to user contributions I hope to solve this
>> issues.
>>
>> Please let me know your thoughts. All feedback is deeply appreciated.
>
> Imho it would be very unfortunate, if the documentation would only be
> available online.

I definantly agree. whatever is done online needs to have some way of 
having a snapshot of it done to be distributed for offline access.

> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.

the other problem with a wiki is that unless you have the time to review 
the changes there is a real risk that the documentation will be wrong. 
this is true of any user-contributed documentation, but with a wiki there 
is no way for other users to know what has been checked and is correct vs 
what is wrong.

the suggestion elsewhere in this thread to have an authoritative source 
with the ability for people to make comments does a good job of 
differentiating between the two.

David Lang

> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.
>
> Cheers,
> Michael
>
>
> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html
>


From hks.private at gmail.com  Wed Nov 26 21:20:51 2008
From: hks.private at gmail.com ((private) HKS)
Date: Wed, 26 Nov 2008 15:20:51 -0500
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
	<1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>
Message-ID: <ac9ded160811261220i7fb06ff7r7cfe3a69f010db91@mail.gmail.com>

On Wed, Nov 26, 2008 at 7:43 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Hi all,
>
> some times it seems to help ask around ;) Andre who cares for the web
> sites (also the phpLogCon lead devel) has pointed out to me that we may
> try the CMS' comment function. We have now enabled that. It seems to
> solve the immediate need. So this looks like a way we can at least try
> out.
>
> Anyhow, if you have any additional thoughts please let me know them.
> They probably help shape a medium-term doc strategy.
>
> Thanks,
> Rainer


I agree with most contributors so far, that a typical user-contributed
wiki alone is not sufficient. For a perfect example of a great project
crippled by abysmal wiki documentation, see OpenNMS.

Wikis /can/ work as official documentation, provided that they are
carefully thought through and official sources are the main
contributors. A general structure also needs to be in place so that
things don't spread out into a web of maze-like links with multiple
pages describing the same subject, albeit with small and crucial
differences. Their main benefits are, obviously, some measure of
self-correction and allowing users to share their solutions to
problems the developers may not have encountered.


Don't know if that helps, but I hope so.

-HKS


From mikel at irontec.com  Thu Nov 27 10:07:39 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 27 Nov 2008 10:07:39 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
Message-ID: <492E635B.80607@irontec.com>

Rainer Gerhards escribi?:
>> Before installing this version, would be possible to do more
>>     
> debugging?
>
> Sorry, I don't want to upset you, but I have no time to look at things
> already fixed. Quite seriously, we have paid support options for this. I
> see that it can make sense for you to stay with the older version, but
> there is absolutely no benefit to the community. Sorry...
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Ok Rainer
Thanks, really. I?m going to use the stable version.

Last question:
Why could be the reason, if I block internet in this host, and when pass 
10-12 hours, obviously not logging to remote (because firewall), but why 
not locally to?

I do:
logger "message"

And I have not notice in /var/log/syslog or /var/log/messages, and in 
normal situations yes...

Thanks

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From mikel at irontec.com  Thu Nov 27 10:19:25 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 27 Nov 2008 10:19:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>	<577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
Message-ID: <492E661D.8020501@irontec.com>

Andre Lorbach escribi?:
> By not adding it into the standard schema, you mean the default table
> layout for the database? 
> We may could reuse an existing unused field instead?
>
> --
> Andre 
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Mittwoch, 19. November 2008 10:39
>> To: rsyslog-users
>> Subject: Re: [rsyslog] milliseconds timestamp
>>
>> Andre,
>>
>> So do you think we should add this to the standard schema? I am a bit
>> hesitant, as not all folks will probably want that and it keeps
>> backward
>> compatibility. Maybe it is better to just us a standard name but not
>> include it in the standard schema.
>>
>> What do you (and others) think?
>>
>> Rainer
>>
>>     
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>>> Sent: Wednesday, November 19, 2008 10:37 AM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>
>>> I think this would be possible yes, we need to define a common name
>>>       
>> for
>>     
>>> the database and property field in rsyslog, then we can start adding
>>> support for it in phplogcon.
>>>
>>> regards,
>>> Andre
>>>
>>>       
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of mikel
>>>> Sent: Dienstag, 18. November 2008 21:06
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>
>>>>
>>>> Dear Andre
>>>>
>>>> So, in near future will be possible?
>>>>
>>>> Or in far future?
>>>>
>>>> Thanx
>>>>
>>>> On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
>>>> <rgerhards at hq.adiscon.com> wrote:
>>>>         
>>>>> Andre and all,
>>>>>
>>>>> the rsyslog engine has the necessary plumbing and I know of at
>>>>>           
>>> least
>>>       
>>>> one
>>>>         
>>>>> user who stores subsecond timestamps in this way. As far as
>>>>>           
>> rsyslog
>>     
>>>> is
>>>>         
>>>>> concerned, this just requires a special timestamp. So if you
>>>>>           
>> would
>>     
>>>>> support that in phpLogCon (what I would find a good idea ;)), I
>>>>>           
>>> would
>>>       
>>>>> include a standard template into rsyslog (hardcoded). That would
>>>>>           
>>> make
>>>       
>>>> it
>>>>         
>>>>> easier for people to work with it. But maybe we get along by
>>>>>           
> just
>   
>>>>> properly documenting it, because the schema must be modified in
>>>>>           
>> any
>>     
>>>>> case.
>>>>>
>>>>> Another alternative would be to update the default schema, but I
>>>>>           
>> am
>>     
>>>> not
>>>>         
>>>>> sure it that is actually a good idea. On the other hand, high-
>>>>>           
>>>> precision
>>>>         
>>>>> timestamps hopefully get into more widespread use...
>>>>>
>>>>> Thoughts from the community are appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rainer
>>>>>
>>>>>           
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>>>>>> Sent: Tuesday, November 18, 2008 5:56 PM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>>>
>>>>>> I think it wouldn't be so difficult to implement, if the
>>>>>>             
>>>> milliseconds
>>>>         
>>>>>> were simply stored in a second field.
>>>>>> We can add custom new fields into phpLogCon very easily.
>>>>>>
>>>>>> best regards,
>>>>>> Andre Lorbach
>>>>>>
>>>>>>             
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>>>>>> Sent: Dienstag, 18. November 2008 16:39
>>>>>>> To: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>>>>
>>>>>>> Hi Mikel,
>>>>>>>
>>>>>>> I have talked to Andre, phpLogCon's lead developer. He says
>>>>>>>               
>>> MySQL
>>>       
>>>>>> does
>>>>>>             
>>>>>>> not support millisecond resolution in timestamps. So a
>>>>>>>               
>>> work-around
>>>       
>>>>>>> would
>>>>>>> be to store them to another field and add this as a second
>>>>>>>               
>> field
>>     
>>>> to
>>>>         
>>>>>> the
>>>>>>             
>>>>>>> phpLogCon view. I think it would probably smart to enable
>>>>>>>               
>>>> phpLogCon
>>>>         
>>>>>> to
>>>>>>             
>>>>>>> combine two such fields, but I leave this to Andre...
>>>>>>>
>>>>>>> Rainer
>>>>>>>
>>>>>>>               
>>>>>>>> -----Original Message-----
>>>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>>>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>>>>>>>> Sent: Tuesday, November 18, 2008 2:10 PM
>>>>>>>> To: rsyslog-users
>>>>>>>> Subject: [rsyslog] milliseconds timestamp
>>>>>>>>
>>>>>>>> Hello
>>>>>>>> I am using phplogcon viewer and I have loosed milliseconds
>>>>>>>>                 
>>>>>>> timestamps.
>>>>>>>               
>>>>>>>> I watched that "DATE" in mysql databases is date-time type.
>>>>>>>>
>>>>>>>> Does rsyslog gives the date in milliseconds? is possible to
>>>>>>>>                 
>>> see
>>>       
>>>>>> this
>>>>>>             
>>>>>>>> milliseconds din phplogcon?
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mikel Jimenez Fernandez
>>>>>>>> Irontec, Internet y Sistemas sobre GNU/LinuX -
>>>>>>>>                 
>>>>>> http://www.irontec.com
>>>>>>             
>>>>>>>> +34 94.404.81.82
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>                 
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Hello Andre/Rainer

Have do you decide how to progress with that?

Can I help

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Fri Nov 28 09:50:29 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 09:50:29 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492E635B.80607@irontec.com>
References: <492D6A30.9060105@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
	<492D714B.4050107@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
	<492D7343.1040101@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
	<492D77CD.3000204@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
	<492E635B.80607@irontec.com>
Message-ID: <1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>

Mikel,

sorry, I was busy yesterday. See inline...
On Thu, 2008-11-27 at 10:07 +0100, Mikel Jimenez wrote:

> Last question:
> Why could be the reason, if I block internet in this host, and when pass 
> 10-12 hours, obviously not logging to remote (because firewall), but why 
> not locally to?
> 
> I do:
> logger "message"
> 
> And I have not notice in /var/log/syslog or /var/log/messages, and in 
> normal situations yes...

If I am right (the update will prove that) the reason is that the local
log socket is not read (because rsyslog delays imuxsock) and thus fills
up. Then, other processes trying to write to that socket will block
during the write process.

Please note that the rsyslog core will discard messages after some time,
but the rate at which this happens is too slow to get the system back to
normal. As I said, this should disappear with the recent stable version.

HTH
Rainer



From mikel at irontec.com  Fri Nov 28 10:41:37 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Fri, 28 Nov 2008 10:41:37 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com>
	<1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
Message-ID: <492FBCD1.7010509@irontec.com>

Rainer Gerhards escribi?:
> Mikel,
>
> sorry, I was busy yesterday. See inline...
> On Thu, 2008-11-27 at 10:07 +0100, Mikel Jimenez wrote:
>
>   
>> Last question:
>> Why could be the reason, if I block internet in this host, and when pass 
>> 10-12 hours, obviously not logging to remote (because firewall), but why 
>> not locally to?
>>
>> I do:
>> logger "message"
>>
>> And I have not notice in /var/log/syslog or /var/log/messages, and in 
>> normal situations yes...
>>     
>
> If I am right (the update will prove that) the reason is that the local
> log socket is not read (because rsyslog delays imuxsock) and thus fills
> up. Then, other processes trying to write to that socket will block
> during the write process.
>
> Please note that the rsyslog core will discard messages after some time,
> but the rate at which this happens is too slow to get the system back to
> normal. As I said, this should disappear with the recent stable version.
>
> HTH
> Rainer
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Thanks for your explication Rainer!! I will install 3.20 right now!!

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Fri Nov 28 10:43:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 10:43:22 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492FBCD1.7010509@irontec.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com><1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
	<492FBCD1.7010509@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>

> Thanks for your explication Rainer!! I will install 3.20 right now!!

Please let me know if the problem then re-appears. I don't think so, but
if it does this is definitely something we should look into ASAP. I'd
also appreciate if you let me know if the issue is fixed after the
upgrade.

Rainer


From mikel at irontec.com  Fri Nov 28 10:46:05 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Fri, 28 Nov 2008 10:46:05 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com><1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>	<492FBCD1.7010509@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>
Message-ID: <492FBDDD.2070308@irontec.com>

Rainer Gerhards escribi?:
>> Thanks for your explication Rainer!! I will install 3.20 right now!!
>>     
>
> Please let me know if the problem then re-appears. I don't think so, but
> if it does this is definitely something we should look into ASAP. I'd
> also appreciate if you let me know if the issue is fixed after the
> upgrade.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Ok, I will do it

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From peter.matulis at canonical.com  Fri Nov 28 17:59:56 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 11:59:56 -0500
Subject: [rsyslog] property based filtering with regex operator
Message-ID: <4930238C.5040208@canonical.com>

Hi.  I tried the following rule to match any occurrence of 'sda' but it
did not work:

:msg, regex, ".*sda.*"         /var/log/properties/sd_regex

Replacing the regex with simply "sda" matches.  Do I not need to take
the entire message into account?

Peter


From peter.matulis at canonical.com  Fri Nov 28 19:05:54 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 13:05:54 -0500
Subject: [rsyslog] QueueDiscardSeverity defaults
Message-ID: <49303302.3090200@canonical.com>

Hi.  The documentation states that for both the main queue and any
action queue the default value for QueueDiscardSeverity is '4'.  In
another place it is stated to be '8' (to prevent any message loss).

Can anyone also confirm whether both numerical as well as textual
severity can be given as a value?  The docs state that text is ok for
MainMsg but not for Action.

Peter


From rgerhards at hq.adiscon.com  Fri Nov 28 19:54:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 19:54:44 +0100
Subject: [rsyslog] property based filtering with regex operator
In-Reply-To: <4930238C.5040208@canonical.com>
References: <4930238C.5040208@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F754@grfint2.intern.adiscon.com>

Hi Peter,

can you run this through the debug log? That spits out additional
information, maybe that points to the source of the problem.

It's evening over here, I do not have a test system right at hand and
will be off for family things over the weekend. If it's not urgent, I
can run a test on Monday.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 6:00 PM
> To: rsyslog-users
> Subject: [rsyslog] property based filtering with regex operator
> 
> Hi.  I tried the following rule to match any occurrence of 'sda' but
it
> did not work:
> 
> :msg, regex, ".*sda.*"         /var/log/properties/sd_regex
> 
> Replacing the regex with simply "sda" matches.  Do I not need to take
> the entire message into account?
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Nov 28 19:57:08 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 19:57:08 +0100
Subject: [rsyslog] QueueDiscardSeverity defaults
In-Reply-To: <49303302.3090200@canonical.com>
References: <49303302.3090200@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F755@grfint2.intern.adiscon.com>

Peter,

I think that's another doc issue.

The default is 8 now, it was 4 but this has shown to be very
unproductive. I think textual priorities can now be given, but will need
to double-check.

I'll see that I update the doc ASAP.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 7:06 PM
> To: rsyslog-users
> Subject: [rsyslog] QueueDiscardSeverity defaults
> 
> Hi.  The documentation states that for both the main queue and any
> action queue the default value for QueueDiscardSeverity is '4'.  In
> another place it is stated to be '8' (to prevent any message loss).
> 
> Can anyone also confirm whether both numerical as well as textual
> severity can be given as a value?  The docs state that text is ok for
> MainMsg but not for Action.
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Fri Nov 28 20:29:26 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 14:29:26 -0500
Subject: [rsyslog] broken link to 4.1.1 release
Message-ID: <49304696.4000403@canonical.com>

The download link to the latest devel release (4.1.1) is broken:


http://www.rsyslog.com/Downloads-req-getit-lid-140.phtml


Peter


From rgerhards at hq.adiscon.com  Sat Nov 29 16:54:31 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 29 Nov 2008 16:54:31 +0100
Subject: [rsyslog] broken link to 4.1.1 release
In-Reply-To: <49304696.4000403@canonical.com>
References: <49304696.4000403@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F75A@grfint2.intern.adiscon.com>

Hi Peter,

thanks for letting me know. It is fixed now.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 8:29 PM
> To: rsyslog-users
> Subject: [rsyslog] broken link to 4.1.1 release
> 
> The download link to the latest devel release (4.1.1) is broken:
> 
> 
> http://www.rsyslog.com/Downloads-req-getit-lid-140.phtml
> 
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mikel at irontec.com  Mon Nov  3 14:14:50 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Mon, 03 Nov 2008 14:14:50 +0100
Subject: [rsyslog] filter expression dude
Message-ID: <490EF94A.70704@irontec.com>

hello

Is this correct?

if $msg contains 'GUI_set' and not $msg contains 'VALIDATOR' then 
@@192.1.100.1


I want that if message contains GUI_set and no VALIDATOR logs remotelly 
in 192.1.100.1 via TCP


Thanks


From hks.private at gmail.com  Mon Nov  3 16:51:01 2008
From: hks.private at gmail.com ((private) HKS)
Date: Mon, 3 Nov 2008 10:51:01 -0500
Subject: [rsyslog] filter expression dude
In-Reply-To: <490EF94A.70704@irontec.com>
References: <490EF94A.70704@irontec.com>
Message-ID: <ac9ded160811030751v7220f0eas69a25b5faa926da7@mail.gmail.com>

Yes, that should work as expected.

-HKS

On Mon, Nov 3, 2008 at 8:14 AM, Mikel Jimenez <mikel at irontec.com> wrote:
> hello
>
> Is this correct?
>
> if $msg contains 'GUI_set' and not $msg contains 'VALIDATOR' then
> @@192.1.100.1
>
>
> I want that if message contains GUI_set and no VALIDATOR logs remotelly
> in 192.1.100.1 via TCP
>
>
> Thanks
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Wed Nov  5 17:46:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 5 Nov 2008 17:46:22 +0100
Subject: [rsyslog] IPv6 problem (was:  new v3-stable release candidate)
In-Reply-To: <alpine.DEB.1.10.0810281023570.635@asgard.lang.hm>
References: <577465F99B41C842AAFBE9ED71E70ABA44F4BB@grfint2.intern.adiscon.com>
	<alpine.DEB.1.10.0810281023570.635@asgard.lang.hm>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F555@grfint2.intern.adiscon.com>

David,

I had some time to set up a new test environment today and try to
reproduce the problem. Unfortunately, I still did not see it. If you
have any new information, that would be appreciated.

I would also appreciate if someone else could try rsyslog under the
conditions told by David.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, October 28, 2008 6:26 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] new v3-stable release candidate
> 
> On Tue, 28 Oct 2008, Rainer Gerhards wrote:
> 
> > Hi all,
> >
> > I am about to switch the v3-stable release to 3.20, which means it
> will
> > be based on the 3.19.x beta branch. I have created a test tarball,
> > available at:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-3.20.0.tar.gz
> >
> > I would appreciate if some could try it out and report any potential
> > issues they see. I intend to release this tomorrow or Thursday if
> > nothing bad happens.
> 
> one problem I have been seeing in the nextmaster branch is that with
> imudp
> on a box that has IPv6 in the kernel, but not IPv6 IP addresses on the
> box
> rsyslog goes into a loop (eating 100% cpu, but not doing anything)
when
> it
> recieves a IPv4 syslog packet.
> 
> Rainer has not been able to duplicate the problem, could someone else
> test
> this?
> 
> I've been seeing it on Debian amd64 boxes.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From friedl at hq.adiscon.com  Thu Nov  6 16:47:53 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Thu, 6 Nov 2008 16:47:53 +0100
Subject: [rsyslog] rsyslog 3.20.0 (stable) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F566@grfint2.intern.adiscon.com>

Hi all,

we have released rsyslog 3.20.0, the next iteration of the rsyslog
v3-stable branch. Most importantly, TLS support is now available in the
stable branches. This version includes all features of the current beta
plus a bugfix. For users of the v3-stable branch, it brings many feature
enhancements, too many to list all of them here. This is a recommended
update for all v3-stable branch users.

Changelog:

http://www.rsyslog.com/Article302.phtml

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-137.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .



From rgerhards at hq.adiscon.com  Thu Nov  6 17:53:27 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 6 Nov 2008 17:53:27 +0100
Subject: [rsyslog] rsyslog regular expressen checker/generator
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F56C@grfint2.intern.adiscon.com>

Hi all,

regular expressions are quite powerful, but the syntax in rsyslog is,
well, not easy to use. Also, as we have seen, the usual regex check
tools don't work always well with rsyslog's POSIX expressions. I have
created a web-based checker/generator today. It is more or less
finished, but of course needs fine-tuning. I would appreciate if some of
you could have a quick look and comment with whatever suggestion they
have. The URL is:

http://www.rsyslog.com/tool-regex

If this tool turns out to be useful (judging on comments and access
count over the next weeks), I will probably do some other online tools
aiding in config generation.

Thanks,
Rainer


From rgerhards at hq.adiscon.com  Tue Nov 11 17:41:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Nov 2008 17:41:44 +0100
Subject: [rsyslog] rsyslog 3.21.7 released, new beta
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F5E0@grfint2.intern.adiscon.com>

Hi all,

I have just released rsyslog 3.21.7, the new beta branch. This version
brings all features of the current development to the beta branch. It
also contains one new addition, that is the ability to replace a not
found regular expression match with a zero. This can be useful for
storing numerical values into databases. This is a recommended update
for all beta branch users. 

Please note that a new devel branch will follow shortly. Currently,
devel is *older* than the beta.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-138.phtml

Change Log:

http://www.rsyslog.com/Article305.phtml

As always, feedback is appreciated.

Best regards,
Rainer Gerhards


From patrick.shen at net-m.de  Wed Nov 12 11:06:03 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Wed, 12 Nov 2008 11:06:03 +0100
Subject: [rsyslog] FQDN with rsyslogd
Message-ID: <491AAA8B.5010805@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear Rainer,

Have a look at online man doc, maybe it's outdated. I find one thing I
really care about.

######################################################################
If the remote host is located in the same domain  as  the  host,
rsyslogd  is running on, only the simple hostname will be logged instead
of the whole fqdn.
######################################################################

Currently we are using rsyslog as client to send logs to central
loghost, actually in the same domain. But I wanna need FQDN in logs, not
simple hostname.

How need I do?

Many thanks,

- --
Patrick Shen
Operations Engineer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJGqqLkHhYtFevC+MRAoMlAJ9M1CeXQf27KTz0/GznkreUFYcb7QCeKjl6
8R3DzRHtpDZU+a0/B1XQny4=
=RUGG
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Wed Nov 12 11:51:53 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Nov 2008 11:51:53 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491AAA8B.5010805@net-m.de>
References: <491AAA8B.5010805@net-m.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>

Hi Patrick,

this is part of the sysklogd legacy code. While there is no longer much
of sysklogd in rsyslog, this part actually is. I need to figure out if
you can turn it off, I remember to have added such an option.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> Sent: Wednesday, November 12, 2008 11:06 AM
> To: rsyslog-users
> Subject: [rsyslog] FQDN with rsyslogd
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dear Rainer,
> 
> Have a look at online man doc, maybe it's outdated. I find one thing I
> really care about.
> 
> ######################################################################
> If the remote host is located in the same domain  as  the  host,
> rsyslogd  is running on, only the simple hostname will be logged
> instead
> of the whole fqdn.
> ######################################################################
> 
> Currently we are using rsyslog as client to send logs to central
> loghost, actually in the same domain. But I wanna need FQDN in logs,
> not
> simple hostname.
> 
> How need I do?
> 
> Many thanks,
> 
> - --
> Patrick Shen
> Operations Engineer
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJGqqLkHhYtFevC+MRAoMlAJ9M1CeXQf27KTz0/GznkreUFYcb7QCeKjl6
> 8R3DzRHtpDZU+a0/B1XQny4=
> =RUGG
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From patrick.shen at net-m.de  Thu Nov 13 03:11:36 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Thu, 13 Nov 2008 03:11:36 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
References: <491AAA8B.5010805@net-m.de>
	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
Message-ID: <491B8CD8.3020701@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Rainer,

I will appreciate if you could show me the magic code.

Best regards,
Patrick

Rainer Gerhards wrote:
> Hi Patrick,
> 
> this is part of the sysklogd legacy code. While there is no longer much
> of sysklogd in rsyslog, this part actually is. I need to figure out if
> you can turn it off, I remember to have added such an option.
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
>> Sent: Wednesday, November 12, 2008 11:06 AM
>> To: rsyslog-users
>> Subject: [rsyslog] FQDN with rsyslogd
>>
> Dear Rainer,
> 
> Have a look at online man doc, maybe it's outdated. I find one thing I
> really care about.
> 
> ######################################################################
> If the remote host is located in the same domain  as  the  host,
> rsyslogd  is running on, only the simple hostname will be logged
> instead
> of the whole fqdn.
> ######################################################################
> 
> Currently we are using rsyslog as client to send logs to central
> loghost, actually in the same domain. But I wanna need FQDN in logs,
> not
> simple hostname.
> 
> How need I do?
> 
> Many thanks,
> 
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJG4zYkHhYtFevC+MRAhtxAKCFQABa96nbmKFTtrhNN1scqz769gCeLI+J
MHPb4zA20/cupkxuCL6uqQ4=
=TyV0
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Thu Nov 13 09:56:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Nov 2008 09:56:22 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491B8CD8.3020701@net-m.de>
References: <491AAA8B.5010805@net-m.de>
	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
	<491B8CD8.3020701@net-m.de>
Message-ID: <1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>

Hi Patrick,

the code in question is this:

http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008aac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171

doesn't look hard to add a global option to prevent it, I'll see when I
can do it. In the meantime, you can simply comment out the strip of the
local domain (line 1174 I guess, but you need to check if it works).

Rainer

On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Rainer,
> 
> I will appreciate if you could show me the magic code.
> 
> Best regards,
> Patrick
> 
> Rainer Gerhards wrote:
> > Hi Patrick,
> > 
> > this is part of the sysklogd legacy code. While there is no longer much
> > of sysklogd in rsyslog, this part actually is. I need to figure out if
> > you can turn it off, I remember to have added such an option.
> > 
> > Rainer
> > 
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> >> Sent: Wednesday, November 12, 2008 11:06 AM
> >> To: rsyslog-users
> >> Subject: [rsyslog] FQDN with rsyslogd
> >>
> > Dear Rainer,
> > 
> > Have a look at online man doc, maybe it's outdated. I find one thing I
> > really care about.
> > 
> > ######################################################################
> > If the remote host is located in the same domain  as  the  host,
> > rsyslogd  is running on, only the simple hostname will be logged
> > instead
> > of the whole fqdn.
> > ######################################################################
> > 
> > Currently we are using rsyslog as client to send logs to central
> > loghost, actually in the same domain. But I wanna need FQDN in logs,
> > not
> > simple hostname.
> > 
> > How need I do?
> > 
> > Many thanks,
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJG4zYkHhYtFevC+MRAhtxAKCFQABa96nbmKFTtrhNN1scqz769gCeLI+J
> MHPb4zA20/cupkxuCL6uqQ4=
> =TyV0
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From mikel at irontec.com  Thu Nov 13 12:37:55 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 13 Nov 2008 12:37:55 +0100
Subject: [rsyslog] filter expression dude
Message-ID: <491C1193.3060609@irontec.com>

Hello
I usually use startswitch 'xxxxx' but is anything similar to "endswith" ?
Or regex that makes "endwith" function?

Would be appreciated an example please

Thanks!!

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Thu Nov 13 13:55:55 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Nov 2008 13:55:55 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C1193.3060609@irontec.com>
References: <491C1193.3060609@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>

You can use 'xxxxx$' as a regular expression. For the property replacer,
I have recently written a tool where you can try this out:

http://www.rsyslog.com/tool-regex

Note, though, that the syntax for conditions is different. But you get
the idea...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Thursday, November 13, 2008 12:38 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] filter expression dude
> 
> Hello
> I usually use startswitch 'xxxxx' but is anything similar to
"endswith"
> ?
> Or regex that makes "endwith" function?
> 
> Would be appreciated an example please
> 
> Thanks!!
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mikel at irontec.com  Thu Nov 13 14:00:05 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 13 Nov 2008 14:00:05 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
References: <491C1193.3060609@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
Message-ID: <491C24D5.4000607@irontec.com>

Rainer Gerhards escribi?:
> You can use 'xxxxx$' as a regular expression. For the property replacer,
> I have recently written a tool where you can try this out:
>
> http://www.rsyslog.com/tool-regex
>
> Note, though, that the syntax for conditions is different. But you get
> the idea...
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Thursday, November 13, 2008 12:38 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] filter expression dude
>>
>> Hello
>> I usually use startswitch 'xxxxx' but is anything similar to
>>     
> "endswith"
>   
>> ?
>> Or regex that makes "endwith" function?
>>
>> Would be appreciated an example please
>>
>> Thanks!!
>>
>> --
>> Mikel Jimenez Fernandez
>> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
>> +34 94.404.81.82
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
ok!
And in full example?
I have this
if $FROMHOST startswith 'deusto' then 
:ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
& ~

And I want al "fromhost" that ends with 'sto'. I dont understand how to 
do this with regex tool...
For example:
esto
ppesto
rtisto

Can you write me these practical example?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From r.bhatia at ipax.at  Thu Nov 13 13:45:34 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Thu, 13 Nov 2008 13:45:34 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C1193.3060609@irontec.com>
References: <491C1193.3060609@irontec.com>
Message-ID: <491C216E.4020101@ipax.at>

Mikel Jimenez wrote:
> Hello
> I usually use startswitch 'xxxxx' but is anything similar to "endswith" ?
> Or regex that makes "endwith" function?
> 
> Would be appreciated an example please

thou i do not know about the actual implementation in rsyslog,
you can match an end-of-line with: "$".

e.g.
> raoul $ echo "my test"|grep test$
> my test
> raoul $ echo "my test2"|grep test$
> raoul $

maybe [1] might be of use.

cheers,
raoul
[1] http://www.addedbytes.com/cheat-sheets/regular-expressions-cheat-sheet/
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From hks.private at gmail.com  Thu Nov 13 15:17:58 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 13 Nov 2008 09:17:58 -0500
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C24D5.4000607@irontec.com>
References: <491C1193.3060609@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
	<491C24D5.4000607@irontec.com>
Message-ID: <ac9ded160811130617s4926431ewc177f8137b52299@mail.gmail.com>

> ok!
> And in full example?
> I have this
> if $FROMHOST startswith 'deusto' then
> :ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
> & ~
>
> And I want al "fromhost" that ends with 'sto'. I dont understand how to
> do this with regex tool...
> For example:
> esto
> ppesto
> rtisto
>
> Can you write me these practical example?
>


:fromhost, regex, "sto$"
:ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
& ~

-HKS


From patrick.shen at net-m.de  Fri Nov 14 03:48:42 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Fri, 14 Nov 2008 03:48:42 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
References: <491AAA8B.5010805@net-m.de>	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>	<491B8CD8.3020701@net-m.de>
	<1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
Message-ID: <491CE70A.8050108@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Rainer,

It works. Thank you very much. But I also find an issue:

###################################################################
2008-11-14T03:32:47.137221+01:00 hydra-t kernel: imklog 3.18.4, log
source = /proc/kmsg started.
2008-11-14T03:32:47.137273+01:00 hydra-t rsyslogd: [origin
software="rsyslogd" swVersion="3.18.4" x-pid="651"
x-info="http://www.rsyslog.com"] restart
...
2008-11-14T03:34:51.680382+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,679 [INFO] [main] [:] [] [] [] []
[] [] [] [] mbg.servlet.BillingGateway:
/opt/as/APP/mbg/WEB-INF/conf/MBGBillingProviderConfig.xml loaded
2008-11-14T03:34:51.785187+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,784 [INFO] [main] [:] [] [] [] []
[] [] [] [] base.db.MBGCounterManager: Starting MBGCounterManager
[dataSourceName:jdbc/dataSource/factory]
2008-11-14T03:34:51.785724+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,785 [INFO] [main] [:] [] [] [] []
[] [] [] [] mbg.servlet.BillingGateway: Initialization finish. Start
waiting for request ...
...
2008-11-14T03:37:15.555764+01:00 hydra-t snmpd[1699]: Connection from
UDP: [192.168.4.15]:34136
2008-11-14T03:37:15.555829+01:00 hydra-t snmpd[1699]: Received SNMP
packet(s) from UDP: [192.168.4.15]:34136
2008-11-14T03:37:16.357732+01:00 hydra-t snmpd[1699]: Connection from
UDP: [192.168.4.15]:34136
#####################################################################

We use "Log4j" to produce application(MBG) logs to hydra-t.test.xxx.xxx
(localhost) via TCP. Every line of them is tagged with
hydra-t.test.xxx.xxx. That's working after you show me the code.

But some other logs still use shortname of FQDN (hydra-t), like rsyslog
itself and snmpd.

Is it relative to $template in rsyslog config file? We use "$template
RSYSLOG_TraditionalForwardFormat" currently.

Many thanks,
Patrick


Rainer Gerhards wrote:
> Hi Patrick,
> 
> the code in question is this:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008aac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171
> 
> doesn't look hard to add a global option to prevent it, I'll see when I
> can do it. In the meantime, you can simply comment out the strip of the
> local domain (line 1174 I guess, but you need to check if it works).
> 
> Rainer
> 
> On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> Hi Rainer,
> 
> I will appreciate if you could show me the magic code.
> 
> Best regards,
> Patrick
> 
> Rainer Gerhards wrote:
>>>> Hi Patrick,
>>>>
>>>> this is part of the sysklogd legacy code. While there is no longer much
>>>> of sysklogd in rsyslog, this part actually is. I need to figure out if
>>>> you can turn it off, I remember to have added such an option.
>>>>
>>>> Rainer
>>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
>>>>> Sent: Wednesday, November 12, 2008 11:06 AM
>>>>> To: rsyslog-users
>>>>> Subject: [rsyslog] FQDN with rsyslogd
>>>>>
>>>> Dear Rainer,
>>>>
>>>> Have a look at online man doc, maybe it's outdated. I find one thing I
>>>> really care about.
>>>>
>>>> ######################################################################
>>>> If the remote host is located in the same domain  as  the  host,
>>>> rsyslogd  is running on, only the simple hostname will be logged
>>>> instead
>>>> of the whole fqdn.
>>>> ######################################################################
>>>>
>>>> Currently we are using rsyslog as client to send logs to central
>>>> loghost, actually in the same domain. But I wanna need FQDN in logs,
>>>> not
>>>> simple hostname.
>>>>
>>>> How need I do?
>>>>
>>>> Many thanks,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJHOcKkHhYtFevC+MRAjafAJ9aSXu91K9Hj6xj+tGs0SNWjk/vswCgiE9C
c6P4BTbHAnCa8+/hcw/MkGc=
=GeS4
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Fri Nov 14 08:24:21 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Nov 2008 08:24:21 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491CE70A.8050108@net-m.de>
References: <491AAA8B.5010805@net-m.de>	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>	<491B8CD8.3020701@net-m.de><1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
	<491CE70A.8050108@net-m.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F621@grfint2.intern.adiscon.com>

Hi Patrick,

I think it would be good if you file a feature request in bugzilla. I
will try to have a look, but these messages (or more precisely
hostnames) stem back to some places where the hostname is generated. I
need to review the code more in-depth to see if there is one ultimate
place and, if not, if there is a reason for multiple places or if they
could be restructured to a single code module (very likely!).

There are many other things in the queue, but I'll check if that's
something easy to do to shuffle in between...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> Sent: Friday, November 14, 2008 3:49 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] FQDN with rsyslogd
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Rainer,
> 
> It works. Thank you very much. But I also find an issue:
> 
> ###################################################################
> 2008-11-14T03:32:47.137221+01:00 hydra-t kernel: imklog 3.18.4, log
> source = /proc/kmsg started.
> 2008-11-14T03:32:47.137273+01:00 hydra-t rsyslogd: [origin
> software="rsyslogd" swVersion="3.18.4" x-pid="651"
> x-info="http://www.rsyslog.com"] restart
> ...
> 2008-11-14T03:34:51.680382+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,679 [INFO] [main] [:] [] [] [] []
> [] [] [] [] mbg.servlet.BillingGateway:
> /opt/as/APP/mbg/WEB-INF/conf/MBGBillingProviderConfig.xml loaded
> 2008-11-14T03:34:51.785187+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,784 [INFO] [main] [:] [] [] [] []
> [] [] [] [] base.db.MBGCounterManager: Starting MBGCounterManager
> [dataSourceName:jdbc/dataSource/factory]
> 2008-11-14T03:34:51.785724+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,785 [INFO] [main] [:] [] [] [] []
> [] [] [] [] mbg.servlet.BillingGateway: Initialization finish. Start
> waiting for request ...
> ...
> 2008-11-14T03:37:15.555764+01:00 hydra-t snmpd[1699]: Connection from
> UDP: [192.168.4.15]:34136
> 2008-11-14T03:37:15.555829+01:00 hydra-t snmpd[1699]: Received SNMP
> packet(s) from UDP: [192.168.4.15]:34136
> 2008-11-14T03:37:16.357732+01:00 hydra-t snmpd[1699]: Connection from
> UDP: [192.168.4.15]:34136
> #####################################################################
> 
> We use "Log4j" to produce application(MBG) logs to
hydra-t.test.xxx.xxx
> (localhost) via TCP. Every line of them is tagged with
> hydra-t.test.xxx.xxx. That's working after you show me the code.
> 
> But some other logs still use shortname of FQDN (hydra-t), like
rsyslog
> itself and snmpd.
> 
> Is it relative to $template in rsyslog config file? We use "$template
> RSYSLOG_TraditionalForwardFormat" currently.
> 
> Many thanks,
> Patrick
> 
> 
> Rainer Gerhards wrote:
> > Hi Patrick,
> >
> > the code in question is this:
> >
> >
>
http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008a
> ac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171
> >
> > doesn't look hard to add a global option to prevent it, I'll see
when
> I
> > can do it. In the meantime, you can simply comment out the strip of
> the
> > local domain (line 1174 I guess, but you need to check if it works).
> >
> > Rainer
> >
> > On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> > Hi Rainer,
> >
> > I will appreciate if you could show me the magic code.
> >
> > Best regards,
> > Patrick
> >
> > Rainer Gerhards wrote:
> >>>> Hi Patrick,
> >>>>
> >>>> this is part of the sysklogd legacy code. While there is no
longer
> much
> >>>> of sysklogd in rsyslog, this part actually is. I need to figure
> out if
> >>>> you can turn it off, I remember to have added such an option.
> >>>>
> >>>> Rainer
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>>>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> >>>>> Sent: Wednesday, November 12, 2008 11:06 AM
> >>>>> To: rsyslog-users
> >>>>> Subject: [rsyslog] FQDN with rsyslogd
> >>>>>
> >>>> Dear Rainer,
> >>>>
> >>>> Have a look at online man doc, maybe it's outdated. I find one
> thing I
> >>>> really care about.
> >>>>
> >>>>
> ######################################################################
> >>>> If the remote host is located in the same domain  as  the  host,
> >>>> rsyslogd  is running on, only the simple hostname will be logged
> >>>> instead
> >>>> of the whole fqdn.
> >>>>
> ######################################################################
> >>>>
> >>>> Currently we are using rsyslog as client to send logs to central
> >>>> loghost, actually in the same domain. But I wanna need FQDN in
> logs,
> >>>> not
> >>>> simple hostname.
> >>>>
> >>>> How need I do?
> >>>>
> >>>> Many thanks,
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJHOcKkHhYtFevC+MRAjafAJ9aSXu91K9Hj6xj+tGs0SNWjk/vswCgiE9C
> c6P4BTbHAnCa8+/hcw/MkGc=
> =GeS4
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Nov 18 10:41:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 10:41:35 +0100
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>

Hi all,

I have been asked if it is possible that the property replacer provides
an option of emitting a Unix timestamp for timestamp fields. This sounds
reasonable, but there are a number of subtleties. They are outlined in
the forum thread, with this post:

http://kb.monitorware.com/post14667.html#p14667

being probably a good wrap-up of the problems.

I would appreciate if someone from the mailing list could point me to a
resource, or share an idea, or whatever, on how I may tackle the
problem. Note that the primary concern is that I need to generate the
correct timestamp for messages from different time zones (problems
outlined in mentioned post).

Any feedback is highly welcome.

Thanks,
Rainer


From denemici at libero.it  Tue Nov 18 10:57:04 2008
From: denemici at libero.it (denemici at libero.it)
Date: Tue, 18 Nov 2008 10:57:04 +0100 (CET)
Subject: [rsyslog] undefined symbol: mysql_init
Message-ID: <19844959.335851227002224564.JavaMail.defaultUser@defaultHost>

Hi all,
I have upgrade Rsyslog from the 1.13.1 version (perfectly working with 
mysql) to the 3.20.0.
I have installed the new version from the source package, 
i have followed this step,
1) ./configure --enable-mysql
2) make
3) make 
install

Installation went fine but when i try to run rsyslog i have this error


Could not load module '/usr/local/lib/rsyslog/ommysql.so', dlopen: 
/usr/local/lib/rsyslog/ommysql.so: undefined symbol: mysql_init

If useful this 
a piece of configure phase:
.....
checking mysql/mysql.h usability... yes

checking mysql/mysql.h presence... yes
checking for mysql/mysql.h... yes

checking for mysql_config... yes
checking for mysql_init in -lmysqlclient... 
yes
...
Rsyslog is installed on Red Hat Enterprise Linux AS U4

Linux hostname 
2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64 
GNU/Linux

I try to find this error on the forum but i haven't found nothing :(


Thanks for any help.

Bye
denemici


From mikel at irontec.com  Tue Nov 18 14:10:20 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Tue, 18 Nov 2008 14:10:20 +0100
Subject: [rsyslog] milliseconds timestamp
Message-ID: <4922BEBC.3050606@irontec.com>

Hello
I am using phplogcon viewer and I have loosed milliseconds timestamps.

I watched that "DATE" in mysql databases is date-time type.

Does rsyslog gives the date in milliseconds? is possible to see this 
milliseconds din phplogcon?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From hks.private at gmail.com  Tue Nov 18 15:46:32 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 09:46:32 -0500
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>

On Tue, Nov 18, 2008 at 4:41 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Hi all,
>
> I have been asked if it is possible that the property replacer provides
> an option of emitting a Unix timestamp for timestamp fields. This sounds
> reasonable, but there are a number of subtleties. They are outlined in
> the forum thread, with this post:
>
> http://kb.monitorware.com/post14667.html#p14667
>
> being probably a good wrap-up of the problems.
>
> I would appreciate if someone from the mailing list could point me to a
> resource, or share an idea, or whatever, on how I may tackle the
> problem. Note that the primary concern is that I need to generate the
> correct timestamp for messages from different time zones (problems
> outlined in mentioned post).


Perhaps I'm overlooking a subtlety of the calculation, but why not
convert the time with respect to the host's locale, then apply the the
offset in seconds?

On the subject of leap seconds: epoch time has no such concept. This
is one of the (many) reasons it's crap and should be deserted in favor
of proper ISO timestamps or something similar, but it at least makes
your job a bit easier. To epoch time, every day is 86400 seconds, no
matter what.

-HKS


From rgerhards at hq.adiscon.com  Tue Nov 18 15:49:02 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 15:49:02 +0100
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>

> Perhaps I'm overlooking a subtlety of the calculation, but why not
> convert the time with respect to the host's locale, then apply the the
> offset in seconds?

Lol, there is a saying in German "you don't see the forest because of
the trees" which means you don't see the obvious thing, because it is so
obvious you are overlooking it. This sounds like a perfect case. Of
course, this solution is quite simple. Lol, at least I am glad I asked
;)

Thanks for hinting me.

Rainer


From hks.private at gmail.com  Tue Nov 18 15:51:29 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 09:51:29 -0500
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160811180651l72d85273if6ebea5c979e0c7b@mail.gmail.com>

Glad to help.

-HKS


On Tue, Nov 18, 2008 at 9:49 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
>> Perhaps I'm overlooking a subtlety of the calculation, but why not
>> convert the time with respect to the host's locale, then apply the the
>> offset in seconds?
>
> Lol, there is a saying in German "you don't see the forest because of
> the trees" which means you don't see the obvious thing, because it is so
> obvious you are overlooking it. This sounds like a perfect case. Of
> course, this solution is quite simple. Lol, at least I am glad I asked
> ;)
>
> Thanks for hinting me.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Tue Nov 18 16:38:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 16:38:56 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <4922BEBC.3050606@irontec.com>
References: <4922BEBC.3050606@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>

Hi Mikel,

I have talked to Andre, phpLogCon's lead developer. He says MySQL does
not support millisecond resolution in timestamps. So a work-around would
be to store them to another field and add this as a second field to the
phpLogCon view. I think it would probably smart to enable phpLogCon to
combine two such fields, but I leave this to Andre...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Tuesday, November 18, 2008 2:10 PM
> To: rsyslog-users
> Subject: [rsyslog] milliseconds timestamp
> 
> Hello
> I am using phplogcon viewer and I have loosed milliseconds timestamps.
> 
> I watched that "DATE" in mysql databases is date-time type.
> 
> Does rsyslog gives the date in milliseconds? is possible to see this
> milliseconds din phplogcon?
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From alorbach at ro1.adiscon.com  Tue Nov 18 17:56:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 18 Nov 2008 17:56:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>

I think it wouldn't be so difficult to implement, if the milliseconds
were simply stored in a second field.
We can add custom new fields into phpLogCon very easily. 

best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Dienstag, 18. November 2008 16:39
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> Hi Mikel,
> 
> I have talked to Andre, phpLogCon's lead developer. He says MySQL does
> not support millisecond resolution in timestamps. So a work-around
> would
> be to store them to another field and add this as a second field to
the
> phpLogCon view. I think it would probably smart to enable phpLogCon to
> combine two such fields, but I leave this to Andre...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > Sent: Tuesday, November 18, 2008 2:10 PM
> > To: rsyslog-users
> > Subject: [rsyslog] milliseconds timestamp
> >
> > Hello
> > I am using phplogcon viewer and I have loosed milliseconds
> timestamps.
> >
> > I watched that "DATE" in mysql databases is date-time type.
> >
> > Does rsyslog gives the date in milliseconds? is possible to see this
> > milliseconds din phplogcon?
> >
> > --
> > Mikel Jimenez Fernandez
> > Irontec, Internet y Sistemas sobre GNU/LinuX -
http://www.irontec.com
> > +34 94.404.81.82
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Nov 18 18:00:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 18:00:04 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>

Andre and all,

the rsyslog engine has the necessary plumbing and I know of at least one
user who stores subsecond timestamps in this way. As far as rsyslog is
concerned, this just requires a special timestamp. So if you would
support that in phpLogCon (what I would find a good idea ;)), I would
include a standard template into rsyslog (hardcoded). That would make it
easier for people to work with it. But maybe we get along by just
properly documenting it, because the schema must be modified in any
case.

Another alternative would be to update the default schema, but I am not
sure it that is actually a good idea. On the other hand, high-precision
timestamps hopefully get into more widespread use...

Thoughts from the community are appreciated.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Tuesday, November 18, 2008 5:56 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> I think it wouldn't be so difficult to implement, if the milliseconds
> were simply stored in a second field.
> We can add custom new fields into phpLogCon very easily.
> 
> best regards,
> Andre Lorbach
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Dienstag, 18. November 2008 16:39
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > Hi Mikel,
> >
> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
> does
> > not support millisecond resolution in timestamps. So a work-around
> > would
> > be to store them to another field and add this as a second field to
> the
> > phpLogCon view. I think it would probably smart to enable phpLogCon
> to
> > combine two such fields, but I leave this to Andre...
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > To: rsyslog-users
> > > Subject: [rsyslog] milliseconds timestamp
> > >
> > > Hello
> > > I am using phplogcon viewer and I have loosed milliseconds
> > timestamps.
> > >
> > > I watched that "DATE" in mysql databases is date-time type.
> > >
> > > Does rsyslog gives the date in milliseconds? is possible to see
> this
> > > milliseconds din phplogcon?
> > >
> > > --
> > > Mikel Jimenez Fernandez
> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> http://www.irontec.com
> > > +34 94.404.81.82
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From friedl at hq.adiscon.com  Tue Nov 18 18:03:16 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Tue, 18 Nov 2008 18:03:16 +0100
Subject: [rsyslog] rsyslog 4.1.0 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F681@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 4.1.0 today. This is the initial development
branch version of the upcoming v4 series. The main theme is performance
enhancement, 4.1.0 contains many enhancements which make the engine
process many more events per second than the previous versions. This was
made possible thanks to large changes inside the rsyslog core, and
consequently there is ample room for bugs. So this version should be
used with care. Note that it also contains some additional bugfixes and
small feature enhancements which are not found in other versions.

Please report any issue you see when working with that version. We are
especially interested in any anomaly on IPv6 enabled systems without
IPv6 addresses. There is a problem report for such an environment, which
we could not yet reproduce.

In case you wonder: there is no explicit schedule for the first
v4-stable version. But expect it no earlier than February 2009.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-139.phtml

Changelog:

http://www.rsyslog.com/Article308.phtml

Feedback is very appreciated for this new release.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .



From aoz.syn at gmail.com  Tue Nov 18 18:30:11 2008
From: aoz.syn at gmail.com (RB)
Date: Tue, 18 Nov 2008 10:30:11 -0700
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
Message-ID: <4255c2570811180930m77862fddu984f045127d7116f@mail.gmail.com>

> On the subject of leap seconds: epoch time has no such concept. This
> is one of the (many) reasons it's crap and should be deserted in favor
> of proper ISO timestamps or something similar, but it at least makes
> your job a bit easier. To epoch time, every day is 86400 seconds, no
> matter what.

I think your statement is mildly misleading.  As I understand it,
POSIX.1 epoch (as opposed to TAI) is ambiguous when it comes to leap
seconds, since certain timestamps can be interpreted as two separate
ISO timestamps - i.e. some values are repeated.  Therefore, POSIX.1
epochs most certainly account for leap seconds, but make conversions
of certain stamps difficult.

Regardless, you're right - ISO stamps (or TAI with a leap-second
table) is the way to go when your timestamps must be absolutely
precise across leap boundaries.


From mikel at irontec.com  Tue Nov 18 21:06:04 2008
From: mikel at irontec.com (mikel)
Date: Tue, 18 Nov 2008 21:06:04 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
Message-ID: <a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>


Dear Andre

So, in near future will be possible?

Or in far future? 

Thanx

On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
<rgerhards at hq.adiscon.com> wrote:
> Andre and all,
> 
> the rsyslog engine has the necessary plumbing and I know of at least one
> user who stores subsecond timestamps in this way. As far as rsyslog is
> concerned, this just requires a special timestamp. So if you would
> support that in phpLogCon (what I would find a good idea ;)), I would
> include a standard template into rsyslog (hardcoded). That would make it
> easier for people to work with it. But maybe we get along by just
> properly documenting it, because the schema must be modified in any
> case.
> 
> Another alternative would be to update the default schema, but I am not
> sure it that is actually a good idea. On the other hand, high-precision
> timestamps hopefully get into more widespread use...
> 
> Thoughts from the community are appreciated.
> 
> Thanks,
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>> Sent: Tuesday, November 18, 2008 5:56 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] milliseconds timestamp
>>
>> I think it wouldn't be so difficult to implement, if the milliseconds
>> were simply stored in a second field.
>> We can add custom new fields into phpLogCon very easily.
>>
>> best regards,
>> Andre Lorbach
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> > Sent: Dienstag, 18. November 2008 16:39
>> > To: rsyslog-users
>> > Subject: Re: [rsyslog] milliseconds timestamp
>> >
>> > Hi Mikel,
>> >
>> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
>> does
>> > not support millisecond resolution in timestamps. So a work-around
>> > would
>> > be to store them to another field and add this as a second field to
>> the
>> > phpLogCon view. I think it would probably smart to enable phpLogCon
>> to
>> > combine two such fields, but I leave this to Andre...
>> >
>> > Rainer
>> >
>> > > -----Original Message-----
>> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> > > Sent: Tuesday, November 18, 2008 2:10 PM
>> > > To: rsyslog-users
>> > > Subject: [rsyslog] milliseconds timestamp
>> > >
>> > > Hello
>> > > I am using phplogcon viewer and I have loosed milliseconds
>> > timestamps.
>> > >
>> > > I watched that "DATE" in mysql databases is date-time type.
>> > >
>> > > Does rsyslog gives the date in milliseconds? is possible to see
>> this
>> > > milliseconds din phplogcon?
>> > >
>> > > --
>> > > Mikel Jimenez Fernandez
>> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
>> http://www.irontec.com
>> > > +34 94.404.81.82
>> > >
>> > >
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > http://www.rsyslog.com
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From rsyslog at clark-communications.com  Tue Nov 18 21:23:51 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 12:23:51 -0800
Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
Message-ID: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>


Hello,

I have motivated the creation of a port of rsyslog to OpenBSD.
The port is intended to fully support the OpenBSD ports framework, and  
will build an OpenBSD binary package
suitable for installation via pkg_add.  See the message below for a  
copy of the port.

I have done some testing of this port on OpenBSD 4.4 (stable) under  
amd64.

I would very much appreciate any additional testing and/or comments  
from the rsyslog community.

Best regards,

Don


Begin forwarded message:
> From: Don Jackson <don.jackson at gmail.com>
> Date: November 18, 2008 11:07:45 AM PST
> To: ports at openbsd.org
> Subject: NEW: sysutils/rsyslog-3.20.0
>
>
> Tested on 4.4/amd64, definitely interested in more testing.
>
> $ cat ./pkg/DESCR
>
> A syslogd replacement
>
>
>
>
>
>
>


From rsyslog at clark-communications.com  Tue Nov 18 21:57:02 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 12:57:02 -0800
Subject: [rsyslog] rsyslogd security questions/comments
Message-ID: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>


Hello,

I was reading the man page for rsyslogd today, and saw:

      SECURITY THREATS

      There  is the potential for the rsyslogd daemon to be used
      as a conduit for a denial of service attack.  A rogue pro-
      gram(mer) could very easily flood the rsyslogd daemon with
      syslog messages resulting in the log files  consuming  all
      the remaining space on the filesystem.  Activating logging
      over the inet domain sockets will of course expose a  sys-
      tem  to  risks  outside  of programs or individuals on the
      local machine.

      There are a number of methods of protecting a machine:

      1.     Implement kernel firewalling to limit  which  hosts
             or networks have access to the 514/UDP socket.

      2.     Logging  can be directed to an isolated or non-root
             filesystem which, if filled, will  not  impair  the
             machine.

      3.     The  ext2  filesystem can be used which can be con-
             figured to limit a certain percentage of a filesys-
             tem  to  usage  by  root only.  NOTE that this will
             require rsyslogd to be run as a  non-root  process.
             ALSO  NOTE  that  this will prevent usage of remote
             logging on the default port since rsyslogd will  be
             unable to bind to the 514/UDP socket.

I had the following questions:

Would it be possible (optionally) to have rsyslogd chroot to a  
particular directory on startup?

That seems the safest.  One could configure a disk partition for log  
messages, configure rsyslogd to log there,
and also chroot to a directory on that partition, so if the rsyslogd  
process itself is compromised,
it can't do other damage.

There must be a way to have a daemon run as a non-root user, and also  
to open ports < 1024.
This seems to be done all the time on *bsd machines:

# ps -aux
USER       PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED       TIME  
COMMAND
root         1  0.0  0.0   428   356 ??  Is    Thu02PM    0:00.01 / 
sbin/init
_dhcp    22078  0.0  0.0   396   432 ??  Is    Thu03PM    0:00.01  
dhclient: bge0 (dhclient)
_syslogd 27943  0.0  0.0   452   812 ??  S     Thu03PM    0:00.19  
syslogd -a /var/empty/dev/log

I'm not sure how this is done, but it looks like chroot also supports  
changing the userid...

Just some thoughts,

Best regards,

Don









From david at lang.hm  Tue Nov 18 22:10:13 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 13:10:13 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
Message-ID: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, Don Jackson wrote:

> I had the following questions:
>
> Would it be possible (optionally) to have rsyslogd chroot to a
> particular directory on startup?
> That seems the safest.  One could configure a disk partition for log
> messages, configure rsyslogd to log there,

chroot doesn't help. if you have rsyslog set to log to a seperate 
partition it can only fill that partition, but it can fill _all of that 
partition even if you chroot into a subdirectory on that partition.

> and also chroot to a directory on that partition, so if the rsyslogd
> process itself is compromised,
> it can't do other damage.

that gives you protection if you are receiving logs from the network, but 
if you are receiving logs from /dev/log (local logs) you can't go into a 
chroot effectivly

> There must be a way to have a daemon run as a non-root user, and also
> to open ports < 1024.
> This seems to be done all the time on *bsd machines:

the POSIX standard still calls for ports < 1024 to require root to bind to 
them, different systems have different ways to be non-compliant with the 
standard and let you do so anyway. what OS are you using?

David Lang

> # ps -aux
> USER       PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED       TIME
> COMMAND
> root         1  0.0  0.0   428   356 ??  Is    Thu02PM    0:00.01 /
> sbin/init
> _dhcp    22078  0.0  0.0   396   432 ??  Is    Thu03PM    0:00.01
> dhclient: bge0 (dhclient)
> _syslogd 27943  0.0  0.0   452   812 ??  S     Thu03PM    0:00.19
> syslogd -a /var/empty/dev/log
>
> I'm not sure how this is done, but it looks like chroot also supports
> changing the userid...
>
> Just some thoughts,
>
> Best regards,
>
> Don
>
>
>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From hks.private at gmail.com  Tue Nov 18 22:57:58 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 16:57:58 -0500
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
Message-ID: <ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>

On Tue, Nov 18, 2008 at 4:10 PM,  <david at lang.hm> wrote:
> On Tue, 18 Nov 2008, Don Jackson wrote:
>
>> I had the following questions:
>>
>> Would it be possible (optionally) to have rsyslogd chroot to a
>> particular directory on startup?
>> That seems the safest.  One could configure a disk partition for log
>> messages, configure rsyslogd to log there,
>
> chroot doesn't help. if you have rsyslog set to log to a seperate
> partition it can only fill that partition, but it can fill _all of that
> partition even if you chroot into a subdirectory on that partition.

Yes, but chrooting precludes any possibility of a rogue syslog agent
filling up /other/ partitions. In the event that a security compromise
were found in the rsyslogd service itself, it also confines an
attacker's damage to the chrooted environment. Paired with non-root
privileges, that's decent insurance against a full-on machine ownage.


>> and also chroot to a directory on that partition, so if the rsyslogd
>> process itself is compromised,
>> it can't do other damage.
>
> that gives you protection if you are receiving logs from the network, but
> if you are receiving logs from /dev/log (local logs) you can't go into a
> chroot effectivly
>
>> There must be a way to have a daemon run as a non-root user, and also
>> to open ports < 1024.
>> This seems to be done all the time on *bsd machines:
>
> the POSIX standard still calls for ports < 1024 to require root to bind to
> them, different systems have different ways to be non-compliant with the
> standard and let you do so anyway. what OS are you using?

OpenBSD has solved problems like this with their own daemons by
following two (general) stages of startup:

 - Privileged, where the daemon reads its config file and opens the
necessary sockets (but doesn't yet listen on them). It then chroots
itself and drops its privileges:
 - Unprivileged, where the daemon begins communication with the rest
of the environment and performing whatever job is required of it

For a concise discussion of how this applies to Apache (the best
explanation I've found in their docs), see:
http://www.openbsd.org/faq/faq10.html#httpdchroot

-HKS


From rsyslog at clark-communications.com  Tue Nov 18 23:02:46 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 14:02:46 -0800
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
Message-ID: <85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>


On Nov 18, 2008, at 1:10 PM, david at lang.hm wrote:

> On Tue, 18 Nov 2008, Don Jackson wrote:
>
>> I had the following questions:
>>
>> Would it be possible (optionally) to have rsyslogd chroot to a
>> particular directory on startup?
>> That seems the safest.  One could configure a disk partition for log
>> messages, configure rsyslogd to log there,
>
> chroot doesn't help. if you have rsyslog set to log to a seperate
> partition it can only fill that partition, but it can fill _all of  
> that
> partition even if you chroot into a subdirectory on that partition.

Agree that file systems or partitions can get filled up no matter what  
you do.
But it seems to me the safest thing to do is to enable rsyslog to be  
configured to
use a specific file system on a dedicated partition, and then always  
monitor that.
Clearly that is possible with rsyslog now, but chrooting would make  
this even more restrictive/secure.

If the (r)syslog clients are also using rsyslog, then they could be  
configured for reliable delivery,
and queue log msgs in the event the rsyslog "server" gets hosed,so one  
you fixed your rsyslog server,
they would then be able to deliver their logs.

Seems like that would be the best overall approach.

>
>> and also chroot to a directory on that partition, so if the rsyslogd
>> process itself is compromised,
>> it can't do other damage.
>
> that gives you protection if you are receiving logs from the  
> network, but
> if you are receiving logs from /dev/log (local logs) you can't go  
> into a
> chroot effectivly

Protecting the "server" from attacks from networked clients seems like  
a good idea.

To answer your later question, I use OpenBSD a lot.
Here is how the stock syslogd on OpenBSD provides a /dev/log for each  
chrooted app:

NAME
      syslogd - log systems messages

SYNOPSIS
      syslogd [-dnu] [-a path] [-f config_file] [-m mark_interval]
              [-p log_socket] [-s reporting_socket]

DESCRIPTION
      syslogd reads and logs messages to the system console, log  
files, pipes
      to other programs, other machines and/or users as specified by  
its con-
      figuration file.

      The options are as follows:

      -a path
              Specify a location where syslogd should place an  
additional log
              socket.  Up to about 20 additional logging sockets can  
be speci-
              fied.  The primary use for this is to place additional  
log sock-
              ets in /dev/log of various chroot filespaces.

So you can arrange for each chroot'ed application to have it's own  
copy of /dev/log in its chroot.

It looks like syslogd on OpenBSD itself doesn't chroot.  I am still  
thinking it might be possible to do so (see below), but maybe I am  
totally wrong about this.

>
>> There must be a way to have a daemon run as a non-root user, and also
>> to open ports < 1024.
>> This seems to be done all the time on *bsd machines:
>
> the POSIX standard still calls for ports < 1024 to require root to  
> bind to
> them, different systems have different ways to be non-compliant with  
> the
> standard and let you do so anyway. what OS are you using?

Sorry, I was not being clear nor accurate here.

What I meant to say was, OK, start up as root, open the resources you  
need, and then chroot, both to change your userid to something that is  
non-root ( user _rsyslog ?), and also
to put rsyslog into a restricted subset file system.  Could you not  
open up the network ports needed, and also /dev/log (for the benefit  
of apps running on that machine that will
write there?) before the call to chroot?

If not, it looks like what syslogd on OpenBSD does is separate itself  
into to processes, the main process runs as root, and then it spawns a  
child that runs as user _syslogd,
it is this child process that listens to log sockets, while the parent  
process gives the child access to write log files.

I'm just brainstorming here.  Perhaps what I am thinking is not  
possible.

Don




From david at lang.hm  Tue Nov 18 23:06:48 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 14:06:48 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
Message-ID: <alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, (private) HKS wrote:

> On Tue, Nov 18, 2008 at 4:10 PM,  <david at lang.hm> wrote:
>> On Tue, 18 Nov 2008, Don Jackson wrote:
>>
>>> I had the following questions:
>>>
>>> Would it be possible (optionally) to have rsyslogd chroot to a
>>> particular directory on startup?
>>> That seems the safest.  One could configure a disk partition for log
>>> messages, configure rsyslogd to log there,
>>
>> chroot doesn't help. if you have rsyslog set to log to a seperate
>> partition it can only fill that partition, but it can fill _all of that
>> partition even if you chroot into a subdirectory on that partition.
>
> Yes, but chrooting precludes any possibility of a rogue syslog agent
> filling up /other/ partitions. In the event that a security compromise
> were found in the rsyslogd service itself, it also confines an
> attacker's damage to the chrooted environment. Paired with non-root
> privileges, that's decent insurance against a full-on machine ownage.

true, but that's a very different problem than the one that Don quoted in 
his message

>>> and also chroot to a directory on that partition, so if the rsyslogd
>>> process itself is compromised,
>>> it can't do other damage.
>>
>> that gives you protection if you are receiving logs from the network, but
>> if you are receiving logs from /dev/log (local logs) you can't go into a
>> chroot effectivly
>>
>>> There must be a way to have a daemon run as a non-root user, and also
>>> to open ports < 1024.
>>> This seems to be done all the time on *bsd machines:
>>
>> the POSIX standard still calls for ports < 1024 to require root to bind to
>> them, different systems have different ways to be non-compliant with the
>> standard and let you do so anyway. what OS are you using?
>
> OpenBSD has solved problems like this with their own daemons by
> following two (general) stages of startup:
>
> - Privileged, where the daemon reads its config file and opens the
> necessary sockets (but doesn't yet listen on them). It then chroots
> itself and drops its privileges:
> - Unprivileged, where the daemon begins communication with the rest
> of the environment and performing whatever job is required of it
>
> For a concise discussion of how this applies to Apache (the best
> explanation I've found in their docs), see:
> http://www.openbsd.org/faq/faq10.html#httpdchroot

Yes, many applications drop privilages (and optionally chroot) after 
starting up.

I suspect that changing rsyslog to do this would be a fair bit of work 
(among other things it can't do a full restart without full privilages, so 
the historic HUP behavior couldn't work)

David Lang


From david at lang.hm  Tue Nov 18 23:13:43 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 14:13:43 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
Message-ID: <alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, Don Jackson wrote:

> On Nov 18, 2008, at 1:10 PM, david at lang.hm wrote:
>
>> On Tue, 18 Nov 2008, Don Jackson wrote:
>>
>>
>>> and also chroot to a directory on that partition, so if the rsyslogd
>>> process itself is compromised,
>>> it can't do other damage.
>>
>> that gives you protection if you are receiving logs from the
>> network, but
>> if you are receiving logs from /dev/log (local logs) you can't go
>> into a
>> chroot effectivly
>
> Protecting the "server" from attacks from networked clients seems like
> a good idea.
>
> To answer your later question, I use OpenBSD a lot.
> Here is how the stock syslogd on OpenBSD provides a /dev/log for each
> chrooted app:

the issue isn't providing a /dev/log for each chroot app, that can be done 
easily from a syslogd that's not in the chroot by opening additional 
pipes, the problem is that if the syslogd itself is inside a chroot it 
cannot open anything _outside_ of that chroot, so it can't provide the 
/dev/log device for the main system.

and you don't want to open that before you go into the chroot becouse one 
of the ways to break out of a chroot involves having a file handle open to 
something outside of the chroot.

>>> There must be a way to have a daemon run as a non-root user, and also
>>> to open ports < 1024.
>>> This seems to be done all the time on *bsd machines:
>>
>> the POSIX standard still calls for ports < 1024 to require root to
>> bind to
>> them, different systems have different ways to be non-compliant with
>> the
>> standard and let you do so anyway. what OS are you using?
>
> Sorry, I was not being clear nor accurate here.
>
> What I meant to say was, OK, start up as root, open the resources you
> need, and then chroot, both to change your userid to something that is
> non-root ( user _rsyslog ?), and also
> to put rsyslog into a restricted subset file system.  Could you not
> open up the network ports needed, and also /dev/log (for the benefit
> of apps running on that machine that will
> write there?) before the call to chroot?

changing the userid will work, doing a chroot will not.

> If not, it looks like what syslogd on OpenBSD does is separate itself
> into to processes, the main process runs as root, and then it spawns a
> child that runs as user _syslogd,
> it is this child process that listens to log sockets, while the parent
> process gives the child access to write log files.

rsyslog is multi-threaded, not multi-process. all the threads have access 
to everything that any one thread has access to, so it wouldn't help to 
seperate things out without serious surgery to rsyslog.

you could do something like have one copy of rsyslog running inside a 
chroot, and another one running outside the chroot with the one outside 
the chroot not listening to the network but forwarding everything it 
receives to the one inside the chroot.

this may require a little bit of work to make work (depending on how 
outbound network connections are made from rsyslog)

David Lang


From hks.private at gmail.com  Tue Nov 18 23:29:43 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 17:29:43 -0500
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
Message-ID: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>

FWIW, while I believe this discussion is relevant and the security
changes proposed are important, I don't see this as a high priority
feature. The scripting language/syntax, greater performance, continued
stability enhancements, and true RELP support are all far more
important. As far as security goes, I would much rather see two-way
host authentication than a chroot/unprivilieged framework implemented.

Of course, all of the above would be just fine with me.

-HKS


From rgerhards at hq.adiscon.com  Wed Nov 19 09:39:18 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 09:39:18 +0100
Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
In-Reply-To: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>
References: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F683@grfint2.intern.adiscon.com>

Hi Don,

this is very good news. Please let me know if you run into any problems
that require an upstream fix. I have also seen your other message, but
will read through that thread before I reply.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Don Jackson
> Sent: Tuesday, November 18, 2008 9:24 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
> 
> 
> Hello,
> 
> I have motivated the creation of a port of rsyslog to OpenBSD.
> The port is intended to fully support the OpenBSD ports framework, and
> will build an OpenBSD binary package
> suitable for installation via pkg_add.  See the message below for a
> copy of the port.
> 
> I have done some testing of this port on OpenBSD 4.4 (stable) under
> amd64.
> 
> I would very much appreciate any additional testing and/or comments
> from the rsyslog community.
> 
> Best regards,
> 
> Don
> 
> 
> Begin forwarded message:
> > From: Don Jackson <don.jackson at gmail.com>
> > Date: November 18, 2008 11:07:45 AM PST
> > To: ports at openbsd.org
> > Subject: NEW: sysutils/rsyslog-3.20.0
> >
> >
> > Tested on 4.4/amd64, definitely interested in more testing.
> >
> > $ cat ./pkg/DESCR
> >
> > A syslogd replacement
> >
> >
> >
> >
> >
> >
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 09:46:58 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 09:46:58 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
Message-ID: <1227084418.19905.38.camel@rgf9dev.intern.adiscon.com>

Hi HKS,

just one point for now:

On Tue, 2008-11-18 at 17:29 -0500, (private) HKS wrote:
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

Two-way host authentication is available in the stable branch since
3.20.0. This is done by configuring the proper certficate settings in
TLS mode. As syslog-TLS will soon turn into a full-blown RFC, I expect
to see interoperable implementations in the not so distant future.

Rainer



From alorbach at ro1.adiscon.com  Wed Nov 19 10:37:15 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 19 Nov 2008 10:37:15 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
	<a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>

I think this would be possible yes, we need to define a common name for
the database and property field in rsyslog, then we can start adding
support for it in phplogcon. 

regards,
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of mikel
> Sent: Dienstag, 18. November 2008 21:06
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> 
> Dear Andre
> 
> So, in near future will be possible?
> 
> Or in far future?
> 
> Thanx
> 
> On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> <rgerhards at hq.adiscon.com> wrote:
> > Andre and all,
> >
> > the rsyslog engine has the necessary plumbing and I know of at least
> one
> > user who stores subsecond timestamps in this way. As far as rsyslog
> is
> > concerned, this just requires a special timestamp. So if you would
> > support that in phpLogCon (what I would find a good idea ;)), I
would
> > include a standard template into rsyslog (hardcoded). That would
make
> it
> > easier for people to work with it. But maybe we get along by just
> > properly documenting it, because the schema must be modified in any
> > case.
> >
> > Another alternative would be to update the default schema, but I am
> not
> > sure it that is actually a good idea. On the other hand, high-
> precision
> > timestamps hopefully get into more widespread use...
> >
> > Thoughts from the community are appreciated.
> >
> > Thanks,
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> >> Sent: Tuesday, November 18, 2008 5:56 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] milliseconds timestamp
> >>
> >> I think it wouldn't be so difficult to implement, if the
> milliseconds
> >> were simply stored in a second field.
> >> We can add custom new fields into phpLogCon very easily.
> >>
> >> best regards,
> >> Andre Lorbach
> >>
> >> > -----Original Message-----
> >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> >> > Sent: Dienstag, 18. November 2008 16:39
> >> > To: rsyslog-users
> >> > Subject: Re: [rsyslog] milliseconds timestamp
> >> >
> >> > Hi Mikel,
> >> >
> >> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
> >> does
> >> > not support millisecond resolution in timestamps. So a
work-around
> >> > would
> >> > be to store them to another field and add this as a second field
> to
> >> the
> >> > phpLogCon view. I think it would probably smart to enable
> phpLogCon
> >> to
> >> > combine two such fields, but I leave this to Andre...
> >> >
> >> > Rainer
> >> >
> >> > > -----Original Message-----
> >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> >> > > To: rsyslog-users
> >> > > Subject: [rsyslog] milliseconds timestamp
> >> > >
> >> > > Hello
> >> > > I am using phplogcon viewer and I have loosed milliseconds
> >> > timestamps.
> >> > >
> >> > > I watched that "DATE" in mysql databases is date-time type.
> >> > >
> >> > > Does rsyslog gives the date in milliseconds? is possible to see
> >> this
> >> > > milliseconds din phplogcon?
> >> > >
> >> > > --
> >> > > Mikel Jimenez Fernandez
> >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> >> http://www.irontec.com
> >> > > +34 94.404.81.82
> >> > >
> >> > >
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > > http://www.rsyslog.com
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 10:38:45 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 10:38:45 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>

Andre,

So do you think we should add this to the standard schema? I am a bit
hesitant, as not all folks will probably want that and it keeps backward
compatibility. Maybe it is better to just us a standard name but not
include it in the standard schema.

What do you (and others) think?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Wednesday, November 19, 2008 10:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> I think this would be possible yes, we need to define a common name
for
> the database and property field in rsyslog, then we can start adding
> support for it in phplogcon.
> 
> regards,
> Andre
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of mikel
> > Sent: Dienstag, 18. November 2008 21:06
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> >
> > Dear Andre
> >
> > So, in near future will be possible?
> >
> > Or in far future?
> >
> > Thanx
> >
> > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > <rgerhards at hq.adiscon.com> wrote:
> > > Andre and all,
> > >
> > > the rsyslog engine has the necessary plumbing and I know of at
> least
> > one
> > > user who stores subsecond timestamps in this way. As far as
rsyslog
> > is
> > > concerned, this just requires a special timestamp. So if you would
> > > support that in phpLogCon (what I would find a good idea ;)), I
> would
> > > include a standard template into rsyslog (hardcoded). That would
> make
> > it
> > > easier for people to work with it. But maybe we get along by just
> > > properly documenting it, because the schema must be modified in
any
> > > case.
> > >
> > > Another alternative would be to update the default schema, but I
am
> > not
> > > sure it that is actually a good idea. On the other hand, high-
> > precision
> > > timestamps hopefully get into more widespread use...
> > >
> > > Thoughts from the community are appreciated.
> > >
> > > Thanks,
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > >> To: rsyslog-users
> > >> Subject: Re: [rsyslog] milliseconds timestamp
> > >>
> > >> I think it wouldn't be so difficult to implement, if the
> > milliseconds
> > >> were simply stored in a second field.
> > >> We can add custom new fields into phpLogCon very easily.
> > >>
> > >> best regards,
> > >> Andre Lorbach
> > >>
> > >> > -----Original Message-----
> > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > >> > Sent: Dienstag, 18. November 2008 16:39
> > >> > To: rsyslog-users
> > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > >> >
> > >> > Hi Mikel,
> > >> >
> > >> > I have talked to Andre, phpLogCon's lead developer. He says
> MySQL
> > >> does
> > >> > not support millisecond resolution in timestamps. So a
> work-around
> > >> > would
> > >> > be to store them to another field and add this as a second
field
> > to
> > >> the
> > >> > phpLogCon view. I think it would probably smart to enable
> > phpLogCon
> > >> to
> > >> > combine two such fields, but I leave this to Andre...
> > >> >
> > >> > Rainer
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > >> > > To: rsyslog-users
> > >> > > Subject: [rsyslog] milliseconds timestamp
> > >> > >
> > >> > > Hello
> > >> > > I am using phplogcon viewer and I have loosed milliseconds
> > >> > timestamps.
> > >> > >
> > >> > > I watched that "DATE" in mysql databases is date-time type.
> > >> > >
> > >> > > Does rsyslog gives the date in milliseconds? is possible to
> see
> > >> this
> > >> > > milliseconds din phplogcon?
> > >> > >
> > >> > > --
> > >> > > Mikel Jimenez Fernandez
> > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > >> http://www.irontec.com
> > >> > > +34 94.404.81.82
> > >> > >
> > >> > >
> > >> > > _______________________________________________
> > >> > > rsyslog mailing list
> > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > > http://www.rsyslog.com
> > >> > _______________________________________________
> > >> > rsyslog mailing list
> > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > http://www.rsyslog.com
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From alorbach at ro1.adiscon.com  Wed Nov 19 10:48:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 19 Nov 2008 10:48:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>

By not adding it into the standard schema, you mean the default table
layout for the database? 
We may could reuse an existing unused field instead?

--
Andre 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Mittwoch, 19. November 2008 10:39
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> Andre,
> 
> So do you think we should add this to the standard schema? I am a bit
> hesitant, as not all folks will probably want that and it keeps
> backward
> compatibility. Maybe it is better to just us a standard name but not
> include it in the standard schema.
> 
> What do you (and others) think?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > Sent: Wednesday, November 19, 2008 10:37 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > I think this would be possible yes, we need to define a common name
> for
> > the database and property field in rsyslog, then we can start adding
> > support for it in phplogcon.
> >
> > regards,
> > Andre
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of mikel
> > > Sent: Dienstag, 18. November 2008 21:06
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] milliseconds timestamp
> > >
> > >
> > > Dear Andre
> > >
> > > So, in near future will be possible?
> > >
> > > Or in far future?
> > >
> > > Thanx
> > >
> > > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > > <rgerhards at hq.adiscon.com> wrote:
> > > > Andre and all,
> > > >
> > > > the rsyslog engine has the necessary plumbing and I know of at
> > least
> > > one
> > > > user who stores subsecond timestamps in this way. As far as
> rsyslog
> > > is
> > > > concerned, this just requires a special timestamp. So if you
> would
> > > > support that in phpLogCon (what I would find a good idea ;)), I
> > would
> > > > include a standard template into rsyslog (hardcoded). That would
> > make
> > > it
> > > > easier for people to work with it. But maybe we get along by
just
> > > > properly documenting it, because the schema must be modified in
> any
> > > > case.
> > > >
> > > > Another alternative would be to update the default schema, but I
> am
> > > not
> > > > sure it that is actually a good idea. On the other hand, high-
> > > precision
> > > > timestamps hopefully get into more widespread use...
> > > >
> > > > Thoughts from the community are appreciated.
> > > >
> > > > Thanks,
> > > > Rainer
> > > >
> > > >> -----Original Message-----
> > > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > > >> To: rsyslog-users
> > > >> Subject: Re: [rsyslog] milliseconds timestamp
> > > >>
> > > >> I think it wouldn't be so difficult to implement, if the
> > > milliseconds
> > > >> were simply stored in a second field.
> > > >> We can add custom new fields into phpLogCon very easily.
> > > >>
> > > >> best regards,
> > > >> Andre Lorbach
> > > >>
> > > >> > -----Original Message-----
> > > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > >> > Sent: Dienstag, 18. November 2008 16:39
> > > >> > To: rsyslog-users
> > > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > > >> >
> > > >> > Hi Mikel,
> > > >> >
> > > >> > I have talked to Andre, phpLogCon's lead developer. He says
> > MySQL
> > > >> does
> > > >> > not support millisecond resolution in timestamps. So a
> > work-around
> > > >> > would
> > > >> > be to store them to another field and add this as a second
> field
> > > to
> > > >> the
> > > >> > phpLogCon view. I think it would probably smart to enable
> > > phpLogCon
> > > >> to
> > > >> > combine two such fields, but I leave this to Andre...
> > > >> >
> > > >> > Rainer
> > > >> >
> > > >> > > -----Original Message-----
> > > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > >> > > To: rsyslog-users
> > > >> > > Subject: [rsyslog] milliseconds timestamp
> > > >> > >
> > > >> > > Hello
> > > >> > > I am using phplogcon viewer and I have loosed milliseconds
> > > >> > timestamps.
> > > >> > >
> > > >> > > I watched that "DATE" in mysql databases is date-time type.
> > > >> > >
> > > >> > > Does rsyslog gives the date in milliseconds? is possible to
> > see
> > > >> this
> > > >> > > milliseconds din phplogcon?
> > > >> > >
> > > >> > > --
> > > >> > > Mikel Jimenez Fernandez
> > > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > > >> http://www.irontec.com
> > > >> > > +34 94.404.81.82
> > > >> > >
> > > >> > >
> > > >> > > _______________________________________________
> > > >> > > rsyslog mailing list
> > > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > > http://www.rsyslog.com
> > > >> > _______________________________________________
> > > >> > rsyslog mailing list
> > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > http://www.rsyslog.com
> > > >> _______________________________________________
> > > >> rsyslog mailing list
> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 10:49:28 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 10:49:28 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F688@grfint2.intern.adiscon.com>

No, I mean that we define a field name, but leave it to the user if he
or she adds/creates the field to the schema (actually two, one for
received and one for generated).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Wednesday, November 19, 2008 10:48 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> By not adding it into the standard schema, you mean the default table
> layout for the database?
> We may could reuse an existing unused field instead?
> 
> --
> Andre
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Mittwoch, 19. November 2008 10:39
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > Andre,
> >
> > So do you think we should add this to the standard schema? I am a
bit
> > hesitant, as not all folks will probably want that and it keeps
> > backward
> > compatibility. Maybe it is better to just us a standard name but not
> > include it in the standard schema.
> >
> > What do you (and others) think?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > Sent: Wednesday, November 19, 2008 10:37 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] milliseconds timestamp
> > >
> > > I think this would be possible yes, we need to define a common
name
> > for
> > > the database and property field in rsyslog, then we can start
> adding
> > > support for it in phplogcon.
> > >
> > > regards,
> > > Andre
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of mikel
> > > > Sent: Dienstag, 18. November 2008 21:06
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] milliseconds timestamp
> > > >
> > > >
> > > > Dear Andre
> > > >
> > > > So, in near future will be possible?
> > > >
> > > > Or in far future?
> > > >
> > > > Thanx
> > > >
> > > > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > > > <rgerhards at hq.adiscon.com> wrote:
> > > > > Andre and all,
> > > > >
> > > > > the rsyslog engine has the necessary plumbing and I know of at
> > > least
> > > > one
> > > > > user who stores subsecond timestamps in this way. As far as
> > rsyslog
> > > > is
> > > > > concerned, this just requires a special timestamp. So if you
> > would
> > > > > support that in phpLogCon (what I would find a good idea ;)),
I
> > > would
> > > > > include a standard template into rsyslog (hardcoded). That
> would
> > > make
> > > > it
> > > > > easier for people to work with it. But maybe we get along by
> just
> > > > > properly documenting it, because the schema must be modified
in
> > any
> > > > > case.
> > > > >
> > > > > Another alternative would be to update the default schema, but
> I
> > am
> > > > not
> > > > > sure it that is actually a good idea. On the other hand, high-
> > > > precision
> > > > > timestamps hopefully get into more widespread use...
> > > > >
> > > > > Thoughts from the community are appreciated.
> > > > >
> > > > > Thanks,
> > > > > Rainer
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > > > >> To: rsyslog-users
> > > > >> Subject: Re: [rsyslog] milliseconds timestamp
> > > > >>
> > > > >> I think it wouldn't be so difficult to implement, if the
> > > > milliseconds
> > > > >> were simply stored in a second field.
> > > > >> We can add custom new fields into phpLogCon very easily.
> > > > >>
> > > > >> best regards,
> > > > >> Andre Lorbach
> > > > >>
> > > > >> > -----Original Message-----
> > > > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > > >> > Sent: Dienstag, 18. November 2008 16:39
> > > > >> > To: rsyslog-users
> > > > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > > > >> >
> > > > >> > Hi Mikel,
> > > > >> >
> > > > >> > I have talked to Andre, phpLogCon's lead developer. He says
> > > MySQL
> > > > >> does
> > > > >> > not support millisecond resolution in timestamps. So a
> > > work-around
> > > > >> > would
> > > > >> > be to store them to another field and add this as a second
> > field
> > > > to
> > > > >> the
> > > > >> > phpLogCon view. I think it would probably smart to enable
> > > > phpLogCon
> > > > >> to
> > > > >> > combine two such fields, but I leave this to Andre...
> > > > >> >
> > > > >> > Rainer
> > > > >> >
> > > > >> > > -----Original Message-----
> > > > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > > >> > > To: rsyslog-users
> > > > >> > > Subject: [rsyslog] milliseconds timestamp
> > > > >> > >
> > > > >> > > Hello
> > > > >> > > I am using phplogcon viewer and I have loosed
milliseconds
> > > > >> > timestamps.
> > > > >> > >
> > > > >> > > I watched that "DATE" in mysql databases is date-time
> type.
> > > > >> > >
> > > > >> > > Does rsyslog gives the date in milliseconds? is possible
> to
> > > see
> > > > >> this
> > > > >> > > milliseconds din phplogcon?
> > > > >> > >
> > > > >> > > --
> > > > >> > > Mikel Jimenez Fernandez
> > > > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > > > >> http://www.irontec.com
> > > > >> > > +34 94.404.81.82
> > > > >> > >
> > > > >> > >
> > > > >> > > _______________________________________________
> > > > >> > > rsyslog mailing list
> > > > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> > > http://www.rsyslog.com
> > > > >> > _______________________________________________
> > > > >> > rsyslog mailing list
> > > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> > http://www.rsyslog.com
> > > > >> _______________________________________________
> > > > >> rsyslog mailing list
> > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 13:32:08 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:32:08 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
	<alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68A@grfint2.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, November 18, 2008 11:07 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
[snip]

> Yes, many applications drop privilages (and optionally chroot) after
> starting up.
> 
> I suspect that changing rsyslog to do this would be a fair bit of work
> (among other things it can't do a full restart without full
privilages,
> so
> the historic HUP behavior couldn't work)

Yes, the traditional HUP behavior is simply incompatible with dropping
privileges. But I assume that someone who configures rsyslogd in such a
way knows what he does and thus changes config-reload scripts away from
HUP to a real reload.

Rainer


From rgerhards at hq.adiscon.com  Wed Nov 19 13:46:01 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:46:01 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
Message-ID: <1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>

Hi all,

So, let me use this reply as a warp-up of my thoughts. Basically, I
agree to HKS argument. I also agree to David Lang's argument that some
degree of design change would be necessary to get the requested features
done 100% correct.

But, of course, it all depends on the effort required. And more security
is always welcome. So I took time out today to think about the options
and experiment a bit. I came to the conclusion that some improvement can
probably be made in short time. I experimentally implemented a new
$PrivDropToUser config directive, which basically issues a setuid() call
with the user in question. It is far from being bullet-proof and will
never be totally save without architecture changes. But I think it still
can considerably improve security and when we can achieve this with a
few hours of work, it's probably worth the effort (a bit more work would
improve its security, but I don't do this unless I get feedback it makes
sense at all).

HOWEVER, there are also some things that simply don't work out. For
example, imklog reads the proc file system under Linux and, even though
it is opened as root, these reads fail after dropping to another user.
There may be other issues, I have not further investigated that.

I have created a bugzilla enhancement request for it:

http://bugzilla.adiscon.com/show_bug.cgi?id=105

You may want to subscribe to that bug for updates. 

The experimental code is available here:

http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/droppriv

I would appreciate feedback

a) on the general issue
b) on the experimental code itself

Many thanks,
Rainer

On Tue, 2008-11-18 at 17:29 -0500, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.
> 
> Of course, all of the above would be just fine with me.
> 
> -HKS
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From mbiebl at gmail.com  Wed Nov 19 13:53:12 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 13:53:12 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
Message-ID: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>

2008/11/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> So, let me use this reply as a warp-up of my thoughts. Basically, I
> agree to HKS argument. I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested features
> done 100% correct.
>
> But, of course, it all depends on the effort required. And more security
> is always welcome. So I took time out today to think about the options
> and experiment a bit. I came to the conclusion that some improvement can
> probably be made in short time. I experimentally implemented a new
> $PrivDropToUser config directive, which basically issues a setuid() call
> with the user in question. It is far from being bullet-proof and will
> never be totally save without architecture changes. But I think it still
> can considerably improve security and when we can achieve this with a
> few hours of work, it's probably worth the effort (a bit more work would
> improve its security, but I don't do this unless I get feedback it makes
> sense at all).
>
> HOWEVER, there are also some things that simply don't work out. For
> example, imklog reads the proc file system under Linux and, even though
> it is opened as root, these reads fail after dropping to another user.
> There may be other issues, I have not further investigated that.

/dev/klog could be made accessible via an ACL or a separate group.
(e.g. the init script of the distro could chgrp or setfacl /dev/klog).
Distros will also have to make sure, that the rsyslog daemon has write
access to /var/log (or wherever they put their log files by default,
but that is easily solvable too).

More tricky will be the already mentioned opening of privileged ports I think.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Wed Nov 19 13:55:29 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 13:55:29 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>

2008/11/19 Michael Biebl <mbiebl at gmail.com>:

>
> /dev/klog could be made accessible via an ACL or a separate group.
^^^^^^^^^^^^
Argh, /proc/kmsg, I mean.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 19 13:56:26 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:56:26 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68B@grfint2.intern.adiscon.com>

Hi Michael,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, November 19, 2008 1:53 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> 2008/11/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Hi all,
> >
> > So, let me use this reply as a warp-up of my thoughts. Basically, I
> > agree to HKS argument. I also agree to David Lang's argument that
> some
> > degree of design change would be necessary to get the requested
> features
> > done 100% correct.
> >
> > But, of course, it all depends on the effort required. And more
> security
> > is always welcome. So I took time out today to think about the
> options
> > and experiment a bit. I came to the conclusion that some improvement
> can
> > probably be made in short time. I experimentally implemented a new
> > $PrivDropToUser config directive, which basically issues a setuid()
> call
> > with the user in question. It is far from being bullet-proof and
will
> > never be totally save without architecture changes. But I think it
> still
> > can considerably improve security and when we can achieve this with
a
> > few hours of work, it's probably worth the effort (a bit more work
> would
> > improve its security, but I don't do this unless I get feedback it
> makes
> > sense at all).
> >
> > HOWEVER, there are also some things that simply don't work out. For
> > example, imklog reads the proc file system under Linux and, even
> though
> > it is opened as root, these reads fail after dropping to another
> user.
> > There may be other issues, I have not further investigated that.
> 
> /dev/klog could be made accessible via an ACL or a separate group.
> (e.g. the init script of the distro could chgrp or setfacl /dev/klog).
> Distros will also have to make sure, that the rsyslog daemon has write
> access to /var/log (or wherever they put their log files by default,
> but that is easily solvable too).
> 
> More tricky will be the already mentioned opening of privileged ports
I
> think.

The experimental code handles that. It drops privileges after processing
the config file. There is a window of insecurity between when the
listeners start and the privileges are dropped, and that can not be
solved without an architecture change. But in essence, the "port open"
problem is solved (of course, HUP will always be of non-restart type). I
think security is improved even with this Window of insecurity, so if
the rest works OK, I would consider the new code a valuable addition.

Rainer 


From mbiebl at gmail.com  Wed Nov 19 14:07:34 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 14:07:34 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
Message-ID: <aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>

2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
>
>>
>> /dev/klog could be made accessible via an ACL or a separate group.
> ^^^^^^^^^^^^
> Argh, /proc/kmsg, I mean.

Looks like one can't set an ACL on a procfs unfortunately.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 19 14:10:24 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 14:10:24 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
	<aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68D@grfint2.intern.adiscon.com>

An out-of-process implementation for the plugins would solve all such
issues. But that takes time... A work-around may be to run two rsyslogd
processes concurrently. Definitely not a mainstream or default config,
but could be done. Also, I wonder if the kernel log works on BSD (not
tested), in which case the experimental code would work there without
any bad side effects (better phrased: with side-effects yet to be seen,
as I said I did so far not invest any serious time in testing).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, November 19, 2008 2:08 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> > 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> >
> >>
> >> /dev/klog could be made accessible via an ACL or a separate group.
> > ^^^^^^^^^^^^
> > Argh, /proc/kmsg, I mean.
> 
> Looks like one can't set an ACL on a procfs unfortunately.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Thu Nov 20 16:14:50 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Thu, 20 Nov 2008 10:14:50 -0500
Subject: [rsyslog] documentation: <object>blah, what is object?
Message-ID: <49257EEA.2000706@canonical.com>

I've been looking over the rsyslog documentation and I see a lot of
directives that begin with the nomenclature "<object>".  I see examples
of this tag being assigned to "Action" and "MainMsg".

Is there an explanation of what <object> is all about?

Thanks,

Peter


From rgerhards at hq.adiscon.com  Thu Nov 20 17:10:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:10:04 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <49257EEA.2000706@canonical.com>
References: <49257EEA.2000706@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>

I guess you mean the queue documentation. It looks like I have forgotten
that piece of information ;) <object> is "Action" for action queues and
"MainMsg" for the main message queue. Will add that soon, thanks for
pointing out.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Thursday, November 20, 2008 4:15 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] documentation: <object>blah, what is object?
> 
> I've been looking over the rsyslog documentation and I see a lot of
> directives that begin with the nomenclature "<object>".  I see
examples
> of this tag being assigned to "Action" and "MainMsg".
> 
> Is there an explanation of what <object> is all about?
> 
> Thanks,
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Nov 20 17:11:39 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:11:39 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>

Lol, actually it is described at the *bottom* of the document. I will
add a refrecence to that at the front ;)

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, November 20, 2008 5:10 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> 
> I guess you mean the queue documentation. It looks like I have
> forgotten
> that piece of information ;) <object> is "Action" for action queues
and
> "MainMsg" for the main message queue. Will add that soon, thanks for
> pointing out.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> > Sent: Thursday, November 20, 2008 4:15 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] documentation: <object>blah, what is object?
> >
> > I've been looking over the rsyslog documentation and I see a lot of
> > directives that begin with the nomenclature "<object>".  I see
> examples
> > of this tag being assigned to "Action" and "MainMsg".
> >
> > Is there an explanation of what <object> is all about?
> >
> > Thanks,
> >
> > Peter
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Nov 20 17:16:49 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:16:49 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com><577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>

I "fixed" this. The bottom part could simply be moved up, I think it now
makes more sense. You may want to have a look at the online version:

http://www.rsyslog.com/doc-queues.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, November 20, 2008 5:12 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> 
> Lol, actually it is described at the *bottom* of the document. I will
> add a refrecence to that at the front ;)
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, November 20, 2008 5:10 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> >
> > I guess you mean the queue documentation. It looks like I have
> > forgotten
> > that piece of information ;) <object> is "Action" for action queues
> and
> > "MainMsg" for the main message queue. Will add that soon, thanks for
> > pointing out.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> > > Sent: Thursday, November 20, 2008 4:15 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] documentation: <object>blah, what is object?
> > >
> > > I've been looking over the rsyslog documentation and I see a lot
of
> > > directives that begin with the nomenclature "<object>".  I see
> > examples
> > > of this tag being assigned to "Action" and "MainMsg".
> > >
> > > Is there an explanation of what <object> is all about?
> > >
> > > Thanks,
> > >
> > > Peter
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Thu Nov 20 17:32:40 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Thu, 20 Nov 2008 11:32:40 -0500
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com><577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>
Message-ID: <49259128.2040506@canonical.com>

Thanks for your prompt response.

Peter

Rainer Gerhards wrote:
> I "fixed" this. The bottom part could simply be moved up, I think it now
> makes more sense. You may want to have a look at the online version:
> 
> http://www.rsyslog.com/doc-queues.html
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Thursday, November 20, 2008 5:12 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
>>
>> Lol, actually it is described at the *bottom* of the document. I will
>> add a refrecence to that at the front ;)
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>> Sent: Thursday, November 20, 2008 5:10 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
>>>
>>> I guess you mean the queue documentation. It looks like I have
>>> forgotten
>>> that piece of information ;) <object> is "Action" for action queues
>> and
>>> "MainMsg" for the main message queue. Will add that soon, thanks for
>>> pointing out.
>>>
>>> Rainer
>>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
>>>> Sent: Thursday, November 20, 2008 4:15 PM
>>>> To: rsyslog at lists.adiscon.com
>>>> Subject: [rsyslog] documentation: <object>blah, what is object?
>>>>
>>>> I've been looking over the rsyslog documentation and I see a lot
> of
>>>> directives that begin with the nomenclature "<object>".  I see
>>> examples
>>>> of this tag being assigned to "Action" and "MainMsg".
>>>>
>>>> Is there an explanation of what <object> is all about?
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From janfrode at tanso.net  Fri Nov 21 13:11:02 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 13:11:02 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>

For my usage, I need two modes of operation for syslog daemons. 

    1 - local syslog. Requires privileges to on local devices (/dev/log,
        /dev/klogd or similar), write to local log-files, and send to
        remote log server.

    2 - central log server. Only listen on the needed network ports
        (514/udp/tcp), and write to local log files (possibly also send
        to other remote log servers).

For #1 I think it's OK not being able to chroot, keep more privileges,
etc.,  as the attacks against it will mostly be from local processes.

#2 needs to be *very* openly available on the network ports, since all 
my servers needs to send logs to it. #2 will also be holding a lot more
sensitive data than #1, so I think this server needs to be protected as
much as possible. If chroot'ing, or dropping privileges prevents it from
reading from local /proc og /dev, I think that wouldn't matter much. One
could always run two instances on these few central servers, i.e. #1
sending to #2.


  -jf



From rgerhards at hq.adiscon.com  Fri Nov 21 14:50:28 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 14:50:28 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>

So it looks like the new idea, though not perfect, seems to provide some
value, at least under some circumstances. It also looks trivially to
implement. Most effort is probably to tell people precisely why it is
not a fully security guard. I'll see if I get this fully implemented
next week if nobody objects. I guess it's not more than a day's worth.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Friday, November 21, 2008 1:11 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> For my usage, I need two modes of operation for syslog daemons.
> 
>     1 - local syslog. Requires privileges to on local devices
> (/dev/log,
>         /dev/klogd or similar), write to local log-files, and send to
>         remote log server.
> 
>     2 - central log server. Only listen on the needed network ports
>         (514/udp/tcp), and write to local log files (possibly also
send
>         to other remote log servers).
> 
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
> 
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot
more
> sensitive data than #1, so I think this server needs to be protected
as
> much as possible. If chroot'ing, or dropping privileges prevents it
> from
> reading from local /proc og /dev, I think that wouldn't matter much.
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.
> 
> 
>   -jf
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Fri Nov 21 15:31:24 2008
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Nov 2008 06:31:24 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>

On Fri, 21 Nov 2008, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
>    1 - local syslog. Requires privileges to on local devices (/dev/log,
>        /dev/klogd or similar), write to local log-files, and send to
>        remote log server.
>
>    2 - central log server. Only listen on the needed network ports
>        (514/udp/tcp), and write to local log files (possibly also send
>        to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot more
> sensitive data than #1, so I think this server needs to be protected as
> much as possible.

just pointing out that on this central server the data will be in the same 
place as the daemon listening to the network, and accessable to that 
daemon, so chrooting, dropping privilages, etc won't protect the data that 
gets logged (it will allow you to protect other data that lives on the 
server, just not log data)

well, you could do something where rsyslog writes the file in the chroot, 
and when it rolls the file something outside the chroot picks it up and 
moves it elsewhere, but that's starting to get a little extreme.

> If chroot'ing, or dropping privileges prevents it from
> reading from local /proc og /dev, I think that wouldn't matter much. One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

correct.

David Lang


From david at lang.hm  Fri Nov 21 15:31:49 2008
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Nov 2008 06:31:49 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>
Message-ID: <alpine.DEB.1.10.0811210631270.28119@asgard.lang.hm>

On Fri, 21 Nov 2008, Rainer Gerhards wrote:

> So it looks like the new idea, though not perfect, seems to provide some
> value, at least under some circumstances. It also looks trivially to
> implement. Most effort is probably to tell people precisely why it is
> not a fully security guard. I'll see if I get this fully implemented
> next week if nobody objects. I guess it's not more than a day's worth.

I agree that even the limited version has benifits.

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
>> Sent: Friday, November 21, 2008 1:11 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] rsyslogd security questions/comments
>>
>> For my usage, I need two modes of operation for syslog daemons.
>>
>>     1 - local syslog. Requires privileges to on local devices
>> (/dev/log,
>>         /dev/klogd or similar), write to local log-files, and send to
>>         remote log server.
>>
>>     2 - central log server. Only listen on the needed network ports
>>         (514/udp/tcp), and write to local log files (possibly also
> send
>>         to other remote log servers).
>>
>> For #1 I think it's OK not being able to chroot, keep more privileges,
>> etc.,  as the attacks against it will mostly be from local processes.
>>
>> #2 needs to be *very* openly available on the network ports, since all
>> my servers needs to send logs to it. #2 will also be holding a lot
> more
>> sensitive data than #1, so I think this server needs to be protected
> as
>> much as possible. If chroot'ing, or dropping privileges prevents it
>> from
>> reading from local /proc og /dev, I think that wouldn't matter much.
>> One
>> could always run two instances on these few central servers, i.e. #1
>> sending to #2.
>>
>>
>>   -jf
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From peter.matulis at canonical.com  Fri Nov 21 15:33:07 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 21 Nov 2008 09:33:07 -0500
Subject: [rsyslog] available compare operations in properties-base filters
Message-ID: <4926C6A3.8030706@canonical.com>

For properties-based filtering, the documentation states that currently
there is only a single compare operation ('contains') available.  It
then goes on to describe additional ones.  So what operations are
implemented at this point?

Peter



From rgerhards at hq.adiscon.com  Fri Nov 21 15:38:16 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 15:38:16 +0100
Subject: [rsyslog] available compare operations in properties-base
	filters
In-Reply-To: <4926C6A3.8030706@canonical.com>
References: <4926C6A3.8030706@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D6@grfint2.intern.adiscon.com>

Peter,

all modes in the table are implemented. The "single compare available"
sentence is wrong, I think I simply forgot to update it as I added
modes. Will do that now :)

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 21, 2008 3:33 PM
> To: rsyslog-users
> Subject: [rsyslog] available compare operations in properties-base
> filters
> 
> For properties-based filtering, the documentation states that
currently
> there is only a single compare operation ('contains') available.  It
> then goes on to describe additional ones.  So what operations are
> implemented at this point?
> 
> Peter
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From aoz.syn at gmail.com  Fri Nov 21 16:26:14 2008
From: aoz.syn at gmail.com (RB)
Date: Fri, 21 Nov 2008 08:26:14 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
Message-ID: <4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>

On Fri, Nov 21, 2008 at 07:31,  <david at lang.hm> wrote:
> just pointing out that on this central server the data will be in the same
> place as the daemon listening to the network, and accessable to that
> daemon, so chrooting, dropping privilages, etc won't protect the data that
> gets logged (it will allow you to protect other data that lives on the
> server, just not log data)

As is the same with all chrooted processes - the chroot model prevents
untrusted processes from messing with each other's candy, not their
own.  If anyone that runs a chroot thinks otherwise, they're sorely
mistaken.  It's why web servers are often chrooted with no write
access: the only writing they can do is back through specific ports
(database, syslog, memcache, etc.)

> well, you could do something where rsyslog writes the file in the chroot,
> and when it rolls the file something outside the chroot picks it up and
> moves it elsewhere, but that's starting to get a little extreme.

I absolutely agree that approach is foolish - better to give the
process no write permissions at all and have it either log to a
database (INSERT-only user) or run a two-layered approach wherein the
chrooted log server passes the [scrubbed] logs it receives back over a
pipe or to a secondary syslog service.  Crash/subvert the front layer,
you have little place to go.

RB


From janfrode at tanso.net  Fri Nov 21 18:34:56 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 18:34:56 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
Message-ID: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, RB <aoz.syn at gmail.com> wrote:
> On Fri, Nov 21, 2008 at 07:31,  <david at lang.hm> wrote:
>> just pointing out that on this central server the data will be in the same
>> place as the daemon listening to the network, and accessable to that
>> daemon, so chrooting, dropping privilages, etc won't protect the data that
>> gets logged (it will allow you to protect other data that lives on the
>> server, just not log data)
>
> As is the same with all chrooted processes - the chroot model prevents
> untrusted processes from messing with each other's candy, not their
> own.  If anyone that runs a chroot thinks otherwise, they're sorely
> mistaken.

I think the chroot() limits the possibility of:

    - executing anything outside the chroot (f.ex. /bin/sh)
    - write a new file, and execute it (noexec on the chroot mountpoint)
    - overwrite something outside the chroot. I'm an itsy bit worried that 
      something like this:
    
        $template PerAppLogs,"/var/log/apps/%programname%.log"
        *.* -?PerAppLogs

      could be tricked to write to unexpected places if you mess with
      %programname%, or other similar variables that the sender has complete
      controle over.

so I would argue that my sensitive syslogged data is more secure in a
chroot environment where there are no other executables, than on a non-
chroot environment.



  -jf

-- 
To know recursion, you must first know recursion.



From rgerhards at hq.adiscon.com  Fri Nov 21 18:43:37 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 18:43:37 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com><alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm><4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D7@grfint2.intern.adiscon.com>

Just a quick note:

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Friday, November 21, 2008 6:35 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> I'm an itsy bit worried
> that
>       something like this:
> 
>         $template PerAppLogs,"/var/log/apps/%programname%.log"
>         *.* -?PerAppLogs
> 
>       could be tricked to write to unexpected places if you mess with
>       %programname%, or other similar variables that the sender has
> complete
>       controle over.

Definitely. Though not a complete cure, the secpath-* property replacer
options prevent at least slashes inside programname. I know this is not
100% bullet proof, but it should be applied. Otherwise, you don't need a
malicious user, a simple program error is sufficient to screw up things.

So the template is suggested to be

           $template
PerAppLogs,"/var/log/apps/%programname:::secpath-replace%.log"

>From the doc:
---
secpath-replace 	Replace slashes inside the field by an
underscore. (e.g. "a/b" becomes "a_b"). Useful for secure pathname
generation (with dynafiles).
---

Full source at http://www.rsyslog.com/doc-property_replacer.html

I just thought I mention it.

Rainer


From janfrode at tanso.net  Fri Nov 21 19:23:54 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 19:23:54 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6D7@grfint2.intern.adiscon.com>
Message-ID: <slrngidv5q.jbu.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>
> Definitely. Though not a complete cure, the secpath-* property replacer
> options prevent at least slashes inside programname. I know this is not
> 100% bullet proof, but it should be applied. Otherwise, you don't need a
> malicious user, a simple program error is sufficient to screw up things.


Oh, I didn't know about this one. Thanks!


  -jf



From aoz.syn at gmail.com  Fri Nov 21 20:08:51 2008
From: aoz.syn at gmail.com (RB)
Date: Fri, 21 Nov 2008 12:08:51 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>

On Fri, Nov 21, 2008 at 10:34, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> I think the chroot() limits the possibility of:
>
>    - executing anything outside the chroot (f.ex. /bin/sh)
>    - write a new file, and execute it (noexec on the chroot mountpoint)
>    - overwrite something outside the chroot. I'm an itsy bit worried that
>      something like this:

You are absolutely correct - an unbroken chroot() provides a security
boundary that prevents the process[es] therein from accessing
filesystem resources outside that boundary.  Permissions on the
directory structure at & below that point can provide additional
restrictions (like noexec).

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment where there are no other executables, than on a non-
> chroot environment.

Argue what you like, but your basis is flawed.  You seem to forget
that most exploits bring their own executable [shell]code with them
and often operate purely in-memory.  Therefore they don't strictly
need external executables; the lack thereof is more of a nuisance than
a roadblock.

System designers _must_ assume that any successful remote attacker may
access anything the application can, with the same permissions and
executing arbitrary code - chroot or not.  If the data you are logging
is sufficiently sensitive that no client should be able to read the
data and any possibility of their doing so would be catastrophic, you
are going to have to either establish a one-way transport out of that
syslog chroot() or limit your scope to the point that remote attackers
are no longer a possibility.


RB


From janfrode at tanso.net  Sat Nov 22 12:17:24 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Sat, 22 Nov 2008 12:17:24 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>
Message-ID: <slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, RB <aoz.syn at gmail.com> wrote:
>
>> so I would argue that my sensitive syslogged data is more secure in a
>> chroot environment where there are no other executables, than on a non-
>> chroot environment.
>
> Argue what you like, but your basis is flawed.  You seem to forget
> that most exploits bring their own executable [shell]code with them
> and often operate purely in-memory.  Therefore they don't strictly
> need external executables; the lack thereof is more of a nuisance than
> a roadblock.

Did you see the comment from Rainer about the secpath-replace property? 
I think that proves my basis is not flawed. The chroot here protects
against code flaws, or configuration flaws that could otherwise give
attacker the possibility of overwriting system files.

Also you pointing at "most exploits bring their own executable .. operate
purely in-memory" argument is flawed when I'm arguing for chroot being 
*more* secure. Not 100% secure against all flaws, but *more* secure than
a non-chrooted process.

.. but dropping privileges is higher up on my wishlist, than chroot()..


  -jf

-- 
http://xkcd.com/386/ ;-)



From rsyslog at clark-communications.com  Sat Nov 22 20:18:22 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Sat, 22 Nov 2008 11:18:22 -0800
Subject: [rsyslog] rsyslog support for other chrooted apps
Message-ID: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>


NB, in this email, I am *not* referring to any potential chroot  
support within rsyslog itself!

In an OS like OpenBSD, some/many daemons run in a chroot jail, two  
common examples include postfix and named.

The stock syslogd on OpenBSD supports this by a command line option "- 
a" that adds new /dev/log devices.

Thus, on one of my machines, syslogd is run like this:

	syslogd -a /var/spool/postfix/dev/log -a /var/named/dev/log -a /var/ 
empty/dev/log

Which adds the three additional /dev/logs to the standard /dev/log,  
and syslogd reads them all.

In further testing of the OpenBSD port of rsyslog that I posted  
earlier this week,

	http://lists.adiscon.net/pipermail/rsyslog/2008-November/001395.html

I finally realized that it is going to difficult to replace the stock  
OpenBSD syslogd with rsyslogd unless it supports something like this.

Questions,

Does rsyslog support anything like this already?

If not, how difficult would it be to add this, and do people here  
think this is valuable addition, or not?

Don



From rsyslog at clark-communications.com  Sat Nov 22 21:34:07 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Sat, 22 Nov 2008 12:34:07 -0800
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>


On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
>    1 - local syslog. Requires privileges to on local devices (/dev/ 
> log,
>        /dev/klogd or similar), write to local log-files, and send to
>        remote log server.
>
>    2 - central log server. Only listen on the needed network ports
>        (514/udp/tcp), and write to local log files (possibly also send
>        to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot  
> more
> sensitive data than #1, so I think this server needs to be protected  
> as
> much as possible. If chroot'ing, or dropping privileges prevents it  
> from
> reading from local /proc og /dev, I think that wouldn't matter much.  
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

Yes, your two modes as defined above are exactly how I run syslog, and  
I think the distinction between the two use cases
local-syslog and central-log-server is very useful and important in  
this discussion.

And I agree with your prioritization of security between the two  
modes, the local-syslog does not need to open a network port
to listen for log messages, and so it is only at risk from other  
processes on the same box.

The central-log-server does need to open and read a network port, and  
needs to be open to lots of devices on the network, and thus it is at  
the most risk.
That is the version that most needs privilege separation and chroot.   
And I agree that the server that runs the central-log-server could  
also run an instance of the local-syslog, the local-syslog would  
handle /dev/log on that machine, and forward msgs to be centrally  
logged onto the central-log-server running on the same machine.

 From another sub-thread:

> On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:

> Yes, the traditional HUP behavior is simply incompatible with dropping
> privileges. But I assume that someone who configures rsyslogd in  
> such a
> way knows what he does and thus changes config-reload scripts away  
> from
> HUP to a real reload.

I am not sure about the details of the implementation, but named on  
OpenBSD supports privilege separation, and it provides
some of your HUP functionality, although possibly in different ways:

Here is how named looks from ps:

	root     	23404  0.0  0.0  2148   880 ??  Is     2:19PM    0:00.00  
named: [priv] (named)
	named    21080  0.0  0.2  8744  9452 ??  I      2:19PM    0:00.24  
named -4

Here is what the named man page says about HUP:

   SIGNALS
        In routine operation, signals should not be used to con-
        trol the nameserver; rndc should be used instead.

        SIGHUP Force a reload of the server.

        SIGINT, SIGTERM
               Shut down the server.

        The result of sending any other signals to the server is
        undefined.

And, as stated above, rndc is the preferred way of controlling named  
(asking it to reload zone files, etc.)

The config file for rsyslogd could/should reside within the chroot, so  
it should always be accessible.


On yet another sub-thread:

On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
>  I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested  
> features
> done 100% correct.

It is generally difficult to add security ex post facto, and the more  
code that is written prior to the implementation of those design  
changes will tend to increase the difficultly/cost of that  
implementation.

On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

I don't know what the author's goals for rsyslog are, so it is  
difficult to prioritize.

What *I* would like rsyslog to be is the clear choice to replace  
syslog on all the machines in my network, on all the OSes I care about  
(OpenBSD, Solaris, FreeBSD, etc),
basically "total world domination of the syslog space" :-)

I would ask Ranier and others here to reflect on the need/importance  
of security vis-a-vis their ultimate goals for rsyslog, and if  
security is going to be important (I think it is crucial, but you need  
to come to that conclusion yourself), and the cost of implementing  
security design changes is going to increase over time, to consider  
making those design changes sooner rather than later.

In closing, I am learning a lot and enjoying the thoughtful and  
patient responses to my original email on this topic, kudos to the  
rsyslog community, it is a welcome change from some other mailing  
lists :-)

Best regards,

Don















From aoz.syn at gmail.com  Sat Nov 22 23:49:08 2008
From: aoz.syn at gmail.com (RB)
Date: Sat, 22 Nov 2008 15:49:08 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>
	<slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>

On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> Did you see the comment from Rainer about the secpath-replace property?
> I think that proves my basis is not flawed. The chroot here protects
> against code flaws, or configuration flaws that could otherwise give
> attacker the possibility of overwriting system files.

I most certainly did, but all the secpath-replace property protects
against is an administrator foolishly trusting unvalidated data from a
remote host to generate file paths.  Chroot does not protect against
code flaws, full stop.  That is the continuing flaw in your argument.
Chroot only prevents code flaws from affecting anything _outside_ of
the chroot environment.

> Also you pointing at "most exploits bring their own executable .. operate
> purely in-memory" argument is flawed when I'm arguing for chroot being
> *more* secure. Not 100% secure against all flaws, but *more* secure than
> a non-chrooted process.

You mistake what I am arguing - there is no question that chroot()
helps protect the host system's files.  What it does _not_ protect
against is abuse of anything within the chroot().  Your statement was:

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment

If your log data is being stored inside that chroot, you are utterly
wrong.  Please actually understand what chroot() does before you
continue making wild, unfounded statements.


From rgerhards at hq.adiscon.com  Mon Nov 24 11:08:16 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:08:16 +0100
Subject: [rsyslog] rsyslog support for other chrooted apps
In-Reply-To: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>
References: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DA@grfint2.intern.adiscon.com>

Hi Don,

I guess you were lost in the docs (it needs to be restrucuted some
day... I begun with that, but that didn't yet result in improved ease of
use, to phrase it politely ;)).

You use case is described in the relevant module's doc:

http://www.rsyslog.com/doc-imuxsock.html

Please let me know if that solves the issue.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Don Jackson
> Sent: Saturday, November 22, 2008 8:18 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog support for other chrooted apps
> 
> 
> NB, in this email, I am *not* referring to any potential chroot
> support within rsyslog itself!
> 
> In an OS like OpenBSD, some/many daemons run in a chroot jail, two
> common examples include postfix and named.
> 
> The stock syslogd on OpenBSD supports this by a command line option "-
> a" that adds new /dev/log devices.
> 
> Thus, on one of my machines, syslogd is run like this:
> 
> 	syslogd -a /var/spool/postfix/dev/log -a /var/named/dev/log -a
> /var/
> empty/dev/log
> 
> Which adds the three additional /dev/logs to the standard /dev/log,
> and syslogd reads them all.
> 
> In further testing of the OpenBSD port of rsyslog that I posted
> earlier this week,
> 
> 	http://lists.adiscon.net/pipermail/rsyslog/2008-
> November/001395.html
> 
> I finally realized that it is going to difficult to replace the stock
> OpenBSD syslogd with rsyslogd unless it supports something like this.
> 
> Questions,
> 
> Does rsyslog support anything like this already?
> 
> If not, how difficult would it be to add this, and do people here
> think this is valuable addition, or not?
> 
> Don
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dhaivat.desai at yahoo.com  Mon Nov 24 11:23:57 2008
From: dhaivat.desai at yahoo.com (Dhaivat Desai)
Date: Mon, 24 Nov 2008 15:53:57 +0530 (IST)
Subject: [rsyslog]  defining own input module in rsyslog
Message-ID: <10144.52888.qm@web94909.mail.in2.yahoo.com>

Hi,

I am using imuxsock module of rsyslog.
I want to make a new module with some changes in the existing one.
in imuxsock.c the macros and callbacks are very confusing.
are there any steps i need to follow for developing my own module working with RSyslog?

i want to change imuxsock module work with stream protocol. as current implementation doesnt support binary message logging, i want to add binary message logging also in the same.

moreover, in the rsyslog.config i want to add one more directive $BINARYFILEPATH which specifies the pathname for binary file which all the binary messages will be logged into.

what changes will i require to get my code working?

Thanks,
Dhaivat....









      Add more friends to your messenger and enjoy! Go to http://messenger.yahoo.com/invite/

From rgerhards at hq.adiscon.com  Mon Nov 24 11:41:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:41:25 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com><alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm><4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com><slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com><4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com><slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DC@grfint2.intern.adiscon.com>

Hi all,

please let me jump into the middle of that discussion.

Maybe it helps if someone else comments. From my (external) point of
view, it looks like your arguments hit more or less the same targets,
but it also looks like you seem to apply different co-notations to
things. So let me try to formalize things a bit.

Let me think of security as a probability of security breach. S_curr is
the security of the reference system without a root jail. S_total is the
security of a hypothetical system that is "totally secure" (knowing well
that no such system exists). In other words, the probability S_total
equals 0.

I think the common ground is that a root jail does not worsen security.
Note that I do not say it improves security, only that it does not
reduce a system's security. S_jail is the security of a system that is
otherwise identical to the reference system, but with a root jail. Than
S_jail <= s_curr, because we assume that the security of the system is
not reduced.

I think it is also common ground that the probability of a security
breach is reduced if the number of attack vectors is reduced, without
any new attack vectors being added. [There is one generic "attack
vector", the "thought of being secure and thus becoming careless" which
always increases as risk is reduced - I will not include that vector in
my thoughts]

We seem to be in agreement that a root jail is able to prevent some
attacks from being successful. I can't enumerate them and it is probably
useless to try to do so (because attackers invent new attacks each day),
but there exist some attacks which can be prevented by a root jail. I do
not try to weigh them by their importance.

For obvious reasons, there exist other attacks which are not affected by
the root jail. Some of them have been mentions, like the class of
in-memory based attacks, code injection and many more. 

I tend to think that the set of attack vectors that can be prevented by
a root jail is much smaller than the set of those which can not. I also
tend to think that the later class contains the more serious attack
vectors.

But even then, a root jail seems to remove a subset of the attack
vectors that otherwise exists and so it reduces the probably of security
breach. So it benefits security. We can only argue that it does not
benefit security if we can show that in all cases we can think of (and
those we can not), security is not improved. However, some cases have
been show, where it improves, so it can not be that security is not
improved in all cases. As such, a root jail improves security, or more
precisely the probability of a security breach is

0 < S_jail < S_curr

We can identify the benefit we gain is the difference between the
reference system's probability of security breach and the system with
the jail. Be S_impr this improvement, than

S_impr = S_curr - S_jail

Now the root jail is just one potential security measure. We could now
try to calculate S_impr for all kinds of security measures, for example
a privilege drop. I find it hard to do the actual probability
calculations, but I would guess that S_impr_privdop > S_impr_jail.

Based on the improvements, one may finally decide what to implement
first (either at the code or admin level), all of this of course
weighted with the importance of the numbers.

In any case, I think I have show that both is correct:

- the root jail is a security improvement
- there exist numerous other improvements, many of them probably more
efficient than the jail

And if I got you right, I think your arguments meant exactly this ;)

For rsyslog as a project, this also means we need to weigh security
measures against functionality. Based on this discussion, I wonder if it
would be a useful undertaking to try document at least the security
options at hand and try to provide some helpful notes for those that are
not so deeply knowledgeable about these issues (including me, someone
who is always surprised by the numerous ways people find to circumvent
what one has considered to be secure ;)).

Feedback is appreciated.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of RB
> Sent: Saturday, November 22, 2008 11:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust
<janfrode at tanso.net>
> wrote:
> > Did you see the comment from Rainer about the secpath-replace
> property?
> > I think that proves my basis is not flawed. The chroot here protects
> > against code flaws, or configuration flaws that could otherwise give
> > attacker the possibility of overwriting system files.
> 
> I most certainly did, but all the secpath-replace property protects
> against is an administrator foolishly trusting unvalidated data from a
> remote host to generate file paths.  Chroot does not protect against
> code flaws, full stop.  That is the continuing flaw in your argument.
> Chroot only prevents code flaws from affecting anything _outside_ of
> the chroot environment.
> 
> > Also you pointing at "most exploits bring their own executable ..
> operate
> > purely in-memory" argument is flawed when I'm arguing for chroot
> being
> > *more* secure. Not 100% secure against all flaws, but *more* secure
> than
> > a non-chrooted process.
> 
> You mistake what I am arguing - there is no question that chroot()
> helps protect the host system's files.  What it does _not_ protect
> against is abuse of anything within the chroot().  Your statement was:
> 
> > so I would argue that my sensitive syslogged data is more secure in
a
> > chroot environment
> 
> If your log data is being stored inside that chroot, you are utterly
> wrong.  Please actually understand what chroot() does before you
> continue making wild, unfounded statements.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 11:46:30 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:46:30 +0100
Subject: [rsyslog] defining own input module in rsyslog
In-Reply-To: <10144.52888.qm@web94909.mail.in2.yahoo.com>
References: <10144.52888.qm@web94909.mail.in2.yahoo.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DD@grfint2.intern.adiscon.com>

Hi,

I suggest that you have a look at the imtemplate modul, which was
specifically written to help understand the interface. Please come back
once you have reviewed it. I would also appreciate if you could do that
as a web-based forum thread, because that would make it easier for me to
refer other people with the same intention to that thread. The forum
link is

http://www.rsyslog.com/forum

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dhaivat Desai
> Sent: Monday, November 24, 2008 11:24 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] defining own input module in rsyslog
> 
> Hi,
> 
> I am using imuxsock module of rsyslog.
> I want to make a new module with some changes in the existing one.
> in imuxsock.c the macros and callbacks are very confusing.
> are there any steps i need to follow for developing my own module
> working with RSyslog?
> 
> i want to change imuxsock module work with stream protocol. as current
> implementation doesnt support binary message logging, i want to add
> binary message logging also in the same.
> 
> moreover, in the rsyslog.config i want to add one more directive
> $BINARYFILEPATH which specifies the pathname for binary file which all
> the binary messages will be logged into.
> 
> what changes will i require to get my code working?
> 
> Thanks,
> Dhaivat....
> 
> 
> 
> 
> 
> 
> 
> 
> 
>       Add more friends to your messenger and enjoy! Go to
> http://messenger.yahoo.com/invite/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 12:55:00 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 12:55:00 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>
Message-ID: <1227527700.5282.21.camel@rgf9dev.intern.adiscon.com>

Hi Don,

thanks for your careful thoughts. I would also appreciate if you could
try out the experimental branch, but on the other hand I think I will be
able to create a regular devel branch based on some of the discussion
this week.

On Sat, 2008-11-22 at 12:34 -0800, Don Jackson wrote:
> On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:
> 
> > For my usage, I need two modes of operation for syslog daemons.
> >
> >    1 - local syslog. Requires privileges to on local devices (/dev/ 
> > log,
> >        /dev/klogd or similar), write to local log-files, and send to
> >        remote log server.
> >
> >    2 - central log server. Only listen on the needed network ports
> >        (514/udp/tcp), and write to local log files (possibly also send
> >        to other remote log servers).
> >
> > For #1 I think it's OK not being able to chroot, keep more privileges,
> > etc.,  as the attacks against it will mostly be from local processes.
> >
> > #2 needs to be *very* openly available on the network ports, since all
> > my servers needs to send logs to it. #2 will also be holding a lot  
> > more
> > sensitive data than #1, so I think this server needs to be protected  
> > as
> > much as possible. If chroot'ing, or dropping privileges prevents it  
> > from
> > reading from local /proc og /dev, I think that wouldn't matter much.  
> > One
> > could always run two instances on these few central servers, i.e. #1
> > sending to #2.
> 
> Yes, your two modes as defined above are exactly how I run syslog, and  
> I think the distinction between the two use cases
> local-syslog and central-log-server is very useful and important in  
> this discussion.

Even further, I think this could be different (but compatible) use cases
in a to-be-written rsyslog security doc.

> 
> And I agree with your prioritization of security between the two  
> modes, the local-syslog does not need to open a network port
> to listen for log messages, and so it is only at risk from other  
> processes on the same box.
> 
> The central-log-server does need to open and read a network port, and  
> needs to be open to lots of devices on the network, and thus it is at  
> the most risk.
> That is the version that most needs privilege separation and chroot.   
> And I agree that the server that runs the central-log-server could  
> also run an instance of the local-syslog, the local-syslog would  
> handle /dev/log on that machine, and forward msgs to be centrally  
> logged onto the central-log-server running on the same machine.
> 
>  From another sub-thread:
> 
> > On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:
> 
> > Yes, the traditional HUP behavior is simply incompatible with dropping
> > privileges. But I assume that someone who configures rsyslogd in  
> > such a
> > way knows what he does and thus changes config-reload scripts away  
> > from
> > HUP to a real reload.
> 
> I am not sure about the details of the implementation, but named on  
> OpenBSD supports privilege separation, and it provides
> some of your HUP functionality, although possibly in different ways:
[snip]

It would be too much info to go in full detail, but HUP is ugly and it
is a full restart. I'd like to phase out that mode, but will probably
need to keep it for "admin compatibility". Anyhow, I do not see any
issue if, in a secure system, HUP does not work as a full restart. (One
way to make it work would be to have a master running all the time and
the actual rsyslogd being a child where the parent handles HUP and
restarts the "real" rsyslogd - but I don't like to make it so compliated
"just" to satisfy old habits... [who likes this can always contribute
the code ;)]).

> On yet another sub-thread:
> 
> On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
> >  I also agree to David Lang's argument that some
> > degree of design change would be necessary to get the requested  
> > features
> > done 100% correct.
> 
> It is generally difficult to add security ex post facto, and the more  
> code that is written prior to the implementation of those design  
> changes will tend to increase the difficultly/cost of that  
> implementation.

That's totally right, and rsyslog is a good example. First of all it is
important to remember that it was not designed from scratch. We forked
it off from sysklogd. And while almost no sysklogd code remains today,
we had a long chain of design decisions which were based on the need to
keep compatible with sysklogd. If you traverse that chain, you'll see
why some things are as they are. 

The security approach was not to be worse than sysklogd and better where
possible, but there always was a strong focus on deliverving something.
You may argue that it is foolish not doing the "right thing" right from
the beginning. My view is that we live in an imperfect (and risky) world
and it is important to get some things done. So I don't want to limit
myself in using a better solution because that better solution is not
better in some aspect than the previous solution, which I would be using
otherwise. 

In somewhat more formal words. If sysklogd's number if features is
F_sysklogd and it's probably of security breach is S_sysklog and the
exact same concepts for rsyslog are F_rsyslog and S_rsyslog, then I
don't see any reason not to use rsyslog over sysklogd if F_rsyslog >
F_sysklogd and R_rsyslog <= R_sysklogd. In even other words, if I would
not use rsyslog because it is not as secure as sysklogd, I would ditch
it without getting any benefit and I would also not get the benefits
from the new features it offers.

One may of course argue that rsyslog's addition feature inherently
increase the risk of using it, in which case S_rsyslog < S_sysklogd. The
only argument I have against this is that you can disable most of these
new features so that you can limit the potential risk exposure.

In any case, you are right that the ultimate goal should be to keep as
secure *as possible* and this is why I immediately followed up on your
questions. 

But I think it is still very important to not let security be a show
stopper in a case if it is not worsened. I think if I had started the
rsyslog project with a totally secure design, rsyslog would be obsolete
by now because anybody would already have moved over to syslog-ng and no
other solution would have a chance to rise.

I am not saying that syslog-ng is a bad project, nor am I saying it is
less secure than rsyslog (right now, it probably is more secure). But a
world with just syslog-ng (or one with just rsyslog!) is a monoculture a
monoculture has inherent and very large security risk.

I sincerely believe (bash me) that Windows is *not* bad software (or at
least not worse than Linux). It "just" is (well, "was" ;)) a monoculture
and that lead to much more interest in attacking it. The financial
benefit of attacking it was much greater than the benefit of attacking
Linux.

If you look from that angle (and I do), you'll find that a world with a
not-so-secure rsyslog AND syslog-ng is more secure than a world with a
totally-secure rsyslog (that nobody uses) and and always-present
syslog-ng (being a monoculture on the vast majority of systems). So less
security in a "subsystem" brought more security to the overall picture.

Still, I strongly favor an as-secure-as possible rsyslog, but I need to
abide to the constraints given (and with the financial crises and loss
of sponsorship these constraints for the rsyslog project unfortunately
have not become more permitting...).

> 
> On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> > FWIW, while I believe this discussion is relevant and the security
> > changes proposed are important, I don't see this as a high priority
> > feature. The scripting language/syntax, greater performance, continued
> > stability enhancements, and true RELP support are all far more
> > important. As far as security goes, I would much rather see two-way
> > host authentication than a chroot/unprivilieged framework implemented.
> 
> I don't know what the author's goals for rsyslog are, so it is  
> difficult to prioritize.
> 
> What *I* would like rsyslog to be is the clear choice to replace  
> syslog on all the machines in my network, on all the OSes I care about  
> (OpenBSD, Solaris, FreeBSD, etc),
> basically "total world domination of the syslog space" :-)

as said above, "total world dominition" is sweet fruit, but it has a big
risk associated with it...

> 
> I would ask Ranier and others here to reflect on the need/importance  
> of security vis-a-vis their ultimate goals for rsyslog, and if  
> security is going to be important (I think it is crucial, but you need  
> to come to that conclusion yourself), and the cost of implementing  
> security design changes is going to increase over time, to consider  
> making those design changes sooner rather than later.

I hope I outlined these thoughts above. Please keep asking/suggesting.
My mind is always open to change and I like good arguments ;)

Rainer
> 
> In closing, I am learning a lot and enjoying the thoughtful and  
> patient responses to my original email on this topic, kudos to the  
> rsyslog community, it is a welcome change from some other mailing  
> lists :-)
> 
> Best regards,
> 
> Don
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From pete.philips at secerno.com  Mon Nov 24 13:36:35 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 12:36:35 +0000
Subject: [rsyslog] Logging lost TCP connections
Message-ID: <492A9FD3.2040700@secerno.com>

Hi.

I've googled this extensively but can can't find a solution. Please could anyone tell
me if there is an option to enable logging of lost TCP connections? I'd like the
event recorded locally if contact is lost with a remote TCP syslog server.

Many thanks.


Pete Philips.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 14:36:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 14:36:44 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492A9FD3.2040700@secerno.com>
References: <492A9FD3.2040700@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>

Hi Pete,

I think there is no such option, but I will check with the code.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 1:37 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Logging lost TCP connections
> 
> Hi.
> 
> I've googled this extensively but can can't find a solution. Please
> could anyone tell
> me if there is an option to enable logging of lost TCP connections?
I'd
> like the
> event recorded locally if contact is lost with a remote TCP syslog
> server.
> 
> Many thanks.
> 
> 
> Pete Philips.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 15:17:38 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:17:38 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>

Hi Pete,

I checked, but this information is only available in the debug log.
Having said that, it is probably not hard to add it. What exactly would
you like to see (e.g. complete closes or just aborted ones)?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, November 24, 2008 2:37 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Hi Pete,
> 
> I think there is no such option, but I will check with the code.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Pete Philips
> > Sent: Monday, November 24, 2008 1:37 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Logging lost TCP connections
> >
> > Hi.
> >
> > I've googled this extensively but can can't find a solution. Please
> > could anyone tell
> > me if there is an option to enable logging of lost TCP connections?
> I'd
> > like the
> > event recorded locally if contact is lost with a remote TCP syslog
> > server.
> >
> > Many thanks.
> >
> >
> > Pete Philips.
> > --
> > Pete Philips
> > Secerno Ltd
> > Email: pete.philips at secerno.com
> > PGP key: http://www.secerno.com/pgp/pete.gpg
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:27:02 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:27:02 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
Message-ID: <492AB9B6.4040805@secerno.com>

Rainer,

Thanks for your help on this.

Rainer Gerhards wrote:
> I checked, but this information is only available in the debug log.
> Having said that, it is probably not hard to add it. What exactly would
> you like to see (e.g. complete closes or just aborted ones)?

I'm not sure of the exact terminology but I'd like it to log a record locally everytime
it becomes not possible to send a TCP syslog message to a remote server. Perhaps if
this would be useful to others then it could be added to a future release?

Thanks.


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 15:29:45 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:29:45 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492AB9B6.4040805@secerno.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
	<492AB9B6.4040805@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 3:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Rainer,
> 
> Thanks for your help on this.
> 
> Rainer Gerhards wrote:
> > I checked, but this information is only available in the debug log.
> > Having said that, it is probably not hard to add it. What exactly
> would
> > you like to see (e.g. complete closes or just aborted ones)?
> 
> I'm not sure of the exact terminology but I'd like it to log a record
> locally everytime
> it becomes not possible to send a TCP syslog message to a remote
> server. Perhaps if
> this would be useful to others then it could be added to a future
> release?

Ummm... I was on the wrong side, checking the receiver. So you would
like to log if the send does not succeed?

Which brings up the question: do you have reasons for accepting the
message loss? I am asking because you can instruct rsyslog to retry
sending the data when the remote server is ready again. For an example,
see here:

http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html

Rainer
> 
> Thanks.
> 
> 
> Pete.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:44:02 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:44:02 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
Message-ID: <492ABDB2.2040001@secerno.com>

Rainer Gerhards wrote:
> Ummm... I was on the wrong side, checking the receiver. So you would
> like to log if the send does not succeed?

Yes exactly.

> Which brings up the question: do you have reasons for accepting the
> message loss? I am asking because you can instruct rsyslog to retry
> sending the data when the remote server is ready again. For an example,
> see here:
> 
> http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html

I've seen that page and would like to use the disk buffering capability of
rsyslog to make logging failures unnecessary. Unfortunately the spec for the system
I am delivering requires logging of failed delivery attempts and not disk
buffering :-(


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 15:45:32 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:45:32 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492ABDB2.2040001@secerno.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
	<492ABDB2.2040001@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>

OK, I will look into that, but I can not promise how trivial this may be
;) I keep you posted.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 3:44 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Rainer Gerhards wrote:
> > Ummm... I was on the wrong side, checking the receiver. So you would
> > like to log if the send does not succeed?
> 
> Yes exactly.
> 
> > Which brings up the question: do you have reasons for accepting the
> > message loss? I am asking because you can instruct rsyslog to retry
> > sending the data when the remote server is ready again. For an
> example,
> > see here:
> >
> > http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html
> 
> I've seen that page and would like to use the disk buffering
capability
> of
> rsyslog to make logging failures unnecessary. Unfortunately the spec
> for the system
> I am delivering requires logging of failed delivery attempts and not
> disk
> buffering :-(
> 
> 
> Pete.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 15:49:57 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:49:57 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com><492ABDB2.2040001@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6EA@grfint2.intern.adiscon.com>

Well... I was too quick. The issue is that no matter what we do, we
cannot simply log failed *tcp sends*. The reason is that actions fail
some place inside the rule engine. At the point where it fails, it is
unaware if it is a TCP send, file store, sql store, etc, etc, ... Of
course the tcp output plugin can emit the failure message, but the tcp
output plugin does not know if this is an ultimate failure or not - this
depends on the rule engine configuration.

So if anything is set to report errors, you can either instrument the
plugin and live with potentially false positives OR you can instrument
the rule engine and generate messages in any instance (well, we could
create an action-specific failure notice, which probably can be
configured to what you exactly wanted to have - probably the
solution...).

In any case, feedback from your side is appreciated.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, November 24, 2008 3:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> OK, I will look into that, but I can not promise how trivial this may
> be
> ;) I keep you posted.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Pete Philips
> > Sent: Monday, November 24, 2008 3:44 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Logging lost TCP connections
> >
> > Rainer Gerhards wrote:
> > > Ummm... I was on the wrong side, checking the receiver. So you
> would
> > > like to log if the send does not succeed?
> >
> > Yes exactly.
> >
> > > Which brings up the question: do you have reasons for accepting
the
> > > message loss? I am asking because you can instruct rsyslog to
retry
> > > sending the data when the remote server is ready again. For an
> > example,
> > > see here:
> > >
> > > http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html
> >
> > I've seen that page and would like to use the disk buffering
> capability
> > of
> > rsyslog to make logging failures unnecessary. Unfortunately the spec
> > for the system
> > I am delivering requires logging of failed delivery attempts and not
> > disk
> > buffering :-(
> >
> >
> > Pete.
> > --
> > Pete Philips
> > Secerno Ltd
> > Email: pete.philips at secerno.com
> > PGP key: http://www.secerno.com/pgp/pete.gpg
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:51:14 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:51:14 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>	<492ABDB2.2040001@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
Message-ID: <492ABF62.2060809@secerno.com>

Rainer Gerhards wrote:
> OK, I will look into that, but I can not promise how trivial this may be
> ;) I keep you posted.

Thanks very much for all your help.

Regards,


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Wed Nov 26 12:20:48 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 12:20:48 +0100
Subject: [rsyslog] rsyslog documentation
Message-ID: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>

Hi all,

as you probably know, I keep rsyslog's documentation in html files, with
the man pages containing the bare minimum to make sense of the system.
This need arises from the sheer volume of the information that must be
provided.

However, this makes the documentation pretty closed (everything needs to
go through me) and consequently I have seen very few doc contributions.
It probably also makes it more complicated than needed to translate the
documentation.

Initially, I considered a purely web-based doc vs. a html-file based
doc. I went for the file-based doc as this can easily be distributed
together with every rsyslog version.

However, looking at other projects, many have adapted a web-based
approach where version differences are flagged within the documentation.
The advantage of a web-based doc is that we can use thinks like a wiki
to generate it. That, I think, would make it much easier for users to
contribute doc or at least samples. Also, it looks like a web-based doc
is also more convenient for most users. Everyone has a browser open and
checks the web, but who installs doc packages? ;)

So I am now consider changing to a web-based system, too. I'd probably
consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
point. Since I created it, it receives a slow but steady traffic
increase and has now around the same number of hits than the rsyslog
main site. I would move over the current file-based doc into that system
and at the same time see that I can improve the structure and usability
of the doc.

With the many new and powerful features that appeared in rsyslog over
the past couple of month, I think it is very important to make them
accessible by a sufficiently good doc. The current one is, to phrase it
politely, not good. It probably even hinders adoption of rsyslog in some
cases.

With a web-based system open to user contributions I hope to solve this
issues.

Please let me know your thoughts. All feedback is deeply appreciated.

Many thanks,
Rainer Gerhards



From mbiebl at gmail.com  Wed Nov 26 13:10:38 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 26 Nov 2008 13:10:38 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
Message-ID: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>

2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>
> However, looking at other projects, many have adapted a web-based
> approach where version differences are flagged within the documentation.
> The advantage of a web-based doc is that we can use thinks like a wiki
> to generate it. That, I think, would make it much easier for users to
> contribute doc or at least samples. Also, it looks like a web-based doc
> is also more convenient for most users. Everyone has a browser open and
> checks the web, but who installs doc packages? ;)
>
> So I am now consider changing to a web-based system, too. I'd probably
> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
> point. Since I created it, it receives a slow but steady traffic
> increase and has now around the same number of hits than the rsyslog
> main site. I would move over the current file-based doc into that system
> and at the same time see that I can improve the structure and usability
> of the doc.
>
> With the many new and powerful features that appeared in rsyslog over
> the past couple of month, I think it is very important to make them
> accessible by a sufficiently good doc. The current one is, to phrase it
> politely, not good. It probably even hinders adoption of rsyslog in some
> cases.
>
> With a web-based system open to user contributions I hope to solve this
> issues.
>
> Please let me know your thoughts. All feedback is deeply appreciated.

Imho it would be very unfortunate, if the documentation would only be
available online.

A wiki can be a very good *additional* source of documentation (for
stuff like best practices, tips and tricks), but it doesn't substitute
a well written manual.

What I would rather like to see (and I know, I'm repeating myself
here), is to have the existing documentation a bit more structured and
easier accessible. I posted examples like the exim [1] or postgresql
[2] documentation, which I think are excellent.

The postgresql documentation is interesting. If you follow [2] the
link, you can see, that users are able to add comments, which could be
helpful to improve the overall documentation.
There is still a "static" version though, also available as pdf, which
you could print and even use as a book.

Cheers,
Michael


[1] http://www.exim.org/exim-html-current/doc/html/spec_html/index.html
http://www.exim.org/exim-pdf-current/doc/spec.pdf
[2] http://www.postgresql.org/docs/8.3/interactive/index.html
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Wed Nov 26 13:16:02 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 26 Nov 2008 13:16:02 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>

2008/11/26 Michael Biebl <mbiebl at gmail.com>:
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.

And distribute easily with the rsyslog release tarball

Cheers,
Michael

P.S: Maybe a book could be an additional source of income.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 26 13:28:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 13:28:25 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>

Hi Michael,

thanks for the feedback.

On Wed, 2008-11-26 at 13:10 +0100, Michael Biebl wrote:
> Imho it would be very unfortunate, if the documentation would only be
> available online.
> 
> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.
> 
> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.

I think the PHP manual is also a good sample.

But actually this doesn't solve my main point: It is quite
time-consuming to create a good doc. I am ready to put a bit more time
into that (moving that away from development), but I would like to
encourage user contributions. Without bashing, it is one thing to show a
good example, but it is a totally different one to provide good
content ;)

The bottom line is that I know I could do better, but I have not
sufficient time (and probably not the best talent in that area) to
actually do it.

> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.

This is the main point. Maybe I can find a way to easily make user
comments available to the current doc set. That would help drive in user
contributions (at least I hope so). The problem again is that I simply
can not invest a few weeks of my time "just" to get the right doc
display system done.

It's a trade-off between time available, hoping to get contributions and
quality of doc.

Again, I value your thoughts very much, they go in the same direction of
what I hope to have, but it looks like I currently can not achieve this
and need to lower my demands in order to get working, solution - but one
that is better than the current state...

Rainer



From rgerhards at hq.adiscon.com  Wed Nov 26 13:43:12 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 13:43:12 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
Message-ID: <1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>

Hi all,

some times it seems to help ask around ;) Andre who cares for the web
sites (also the phpLogCon lead devel) has pointed out to me that we may
try the CMS' comment function. We have now enabled that. It seems to
solve the immediate need. So this looks like a way we can at least try
out.

Anyhow, if you have any additional thoughts please let me know them.
They probably help shape a medium-term doc strategy.

Thanks,
Rainer

On Wed, 2008-11-26 at 13:28 +0100, Rainer Gerhards wrote:
> Hi Michael,
> 
> thanks for the feedback.
> 
> On Wed, 2008-11-26 at 13:10 +0100, Michael Biebl wrote:
> > Imho it would be very unfortunate, if the documentation would only be
> > available online.
> > 
> > A wiki can be a very good *additional* source of documentation (for
> > stuff like best practices, tips and tricks), but it doesn't substitute
> > a well written manual.
> > 
> > What I would rather like to see (and I know, I'm repeating myself
> > here), is to have the existing documentation a bit more structured and
> > easier accessible. I posted examples like the exim [1] or postgresql
> > [2] documentation, which I think are excellent.
> 
> I think the PHP manual is also a good sample.
> 
> But actually this doesn't solve my main point: It is quite
> time-consuming to create a good doc. I am ready to put a bit more time
> into that (moving that away from development), but I would like to
> encourage user contributions. Without bashing, it is one thing to show a
> good example, but it is a totally different one to provide good
> content ;)
> 
> The bottom line is that I know I could do better, but I have not
> sufficient time (and probably not the best talent in that area) to
> actually do it.
> 
> > The postgresql documentation is interesting. If you follow [2] the
> > link, you can see, that users are able to add comments, which could be
> > helpful to improve the overall documentation.
> 
> This is the main point. Maybe I can find a way to easily make user
> comments available to the current doc set. That would help drive in user
> contributions (at least I hope so). The problem again is that I simply
> can not invest a few weeks of my time "just" to get the right doc
> display system done.
> 
> It's a trade-off between time available, hoping to get contributions and
> quality of doc.
> 
> Again, I value your thoughts very much, they go in the same direction of
> what I hope to have, but it looks like I currently can not achieve this
> and need to lower my demands in order to get working, solution - but one
> that is better than the current state...
> 
> Rainer
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From rfujita at redhat.com  Wed Nov 26 13:19:03 2008
From: rfujita at redhat.com (=?ISO-2022-JP?B?GyRCTkcbKEIgGyRCRiNFRBsoQg==?=)
Date: Wed, 26 Nov 2008 21:19:03 +0900
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>
Message-ID: <E5D6C192-A983-4807-8B63-686A2D61A018@redhat.com>

Hi,

> P.S: Maybe a book could be an additional source of income.

+1

Best Rio.

On 2008/11/26, at 21:16, Michael Biebl wrote:

> 2008/11/26 Michael Biebl <mbiebl at gmail.com>:
>>
>> The postgresql documentation is interesting. If you follow [2] the
>> link, you can see, that users are able to add comments, which could  
>> be
>> helpful to improve the overall documentation.
>> There is still a "static" version though, also available as pdf,  
>> which
>> you could print and even use as a book.
>
> And distribute easily with the rsyslog release tarball
>
> Cheers,
> Michael
>
> P.S: Maybe a book could be an additional source of income.
>
>
> -- 
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################



From rfujita at redhat.com  Wed Nov 26 13:18:34 2008
From: rfujita at redhat.com (=?ISO-2022-JP?B?GyRCTkcbKEIgGyRCRiNFRBsoQg==?=)
Date: Wed, 26 Nov 2008 21:18:34 +0900
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <0C7538F5-2324-4EB7-9A55-02E28965ABDA@redhat.com>

Hi all,

> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/ 
> index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html

Both these links are made with DocBook.

Best Rio.

On 2008/11/26, at 21:10, Michael Biebl wrote:

> 2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> However, looking at other projects, many have adapted a web-based
>> approach where version differences are flagged within the  
>> documentation.
>> The advantage of a web-based doc is that we can use thinks like a  
>> wiki
>> to generate it. That, I think, would make it much easier for users to
>> contribute doc or at least samples. Also, it looks like a web-based  
>> doc
>> is also more convenient for most users. Everyone has a browser open  
>> and
>> checks the web, but who installs doc packages? ;)
>>
>> So I am now consider changing to a web-based system, too. I'd  
>> probably
>> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
>> point. Since I created it, it receives a slow but steady traffic
>> increase and has now around the same number of hits than the rsyslog
>> main site. I would move over the current file-based doc into that  
>> system
>> and at the same time see that I can improve the structure and  
>> usability
>> of the doc.
>>
>> With the many new and powerful features that appeared in rsyslog over
>> the past couple of month, I think it is very important to make them
>> accessible by a sufficiently good doc. The current one is, to  
>> phrase it
>> politely, not good. It probably even hinders adoption of rsyslog in  
>> some
>> cases.
>>
>> With a web-based system open to user contributions I hope to solve  
>> this
>> issues.
>>
>> Please let me know your thoughts. All feedback is deeply appreciated.
>
> Imho it would be very unfortunate, if the documentation would only be
> available online.
>
> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.
>
> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.
>
> Cheers,
> Michael
>
>
> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/ 
> index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html
> -- 
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################



From mrdemeanour at jackpot.uk.net  Wed Nov 26 13:24:42 2008
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Wed, 26 Nov 2008 12:24:42 +0000
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <492D400A.2010105@jackpot.uk.net>

Michael Biebl wrote:
>> 
>> With a web-based system open to user contributions I hope to solve
>> this issues.
>> 
>> Please let me know your thoughts. All feedback is deeply
>> appreciated.
> 
> Imho it would be very unfortunate, if the documentation would only be
>  available online.

+1.

Many projects now use Docbook, to generate variously HTML, PDF and manpages.

On most of my servers, I have no X GUI set up; HTML docs are therefore a
pain in the neck, unless the HTML is extremely clean (and even then it's
a hassle).

The documentation should be a part of the project, and it should be
possible to check it out, modify it (e.g. translate it) and 'build' it.

I think wiki docs are great, but I also think the foundation for a
documentation wiki should be the authorised version of the docs. Users
can then comment on that authorised version, which should be the docs
that ship with the product. It might be possible to rig some mechanism
for automagically pulling content from the wiki back into the
repository; but I'm not sure how to go about that.

Thanks for this excellent project, and for being so committed to it!
-- 
Jack.


From mikel at irontec.com  Wed Nov 26 16:24:32 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 16:24:32 +0100
Subject: [rsyslog] rsyslog freeze dude
Message-ID: <492D6A30.9060105@irontec.com>

Hello
If I configure rsyslog client like this:

WorkDirectory /root/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName myfile
$ActionResumeRetryCount -1

and

*.* @@somehost.domain.com

Is possible that lost of conectivity (internet down) of this server 
during days, produce that server freeze? or not be possible to login to 
the machine?

If I do /etc/init.d/rsyslog restart it takes long long time to restart.

Please Rainer I hope you help me. thanks


-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 16:39:36 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:39:36 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D6A30.9060105@irontec.com>
References: <492D6A30.9060105@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>

Hi Mikel,

which version are you using? The "ssh hang" can probably be related to
the initial thought that imuxsock messages can be delayed. This has been
changed in recent releases.

Other than that, please note that actions are retried in intervals and
these intervals get longer each time the action is retried without
success. This is done in order to save unnecessary work. I think there
currently is no config setting to change the default intervals.

It may be useful to get a shutdown debug log from such an occurrence.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Wednesday, November 26, 2008 4:25 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog freeze dude
> 
> Hello
> If I configure rsyslog client like this:
> 
> WorkDirectory /root/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueFileName myfile
> $ActionResumeRetryCount -1
> 
> and
> 
> *.* @@somehost.domain.com
> 
> Is possible that lost of conectivity (internet down) of this server
> during days, produce that server freeze? or not be possible to login
to
> the machine?
> 
> If I do /etc/init.d/rsyslog restart it takes long long time to
restart.
> 
> Please Rainer I hope you help me. thanks
> 
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 26 16:54:42 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:54:42 +0100
Subject: [rsyslog] rsyslog 4.1.1 (devel) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F714@grfint2.intern.adiscon.com>

Hi all,

rsyslog 4.1.1, a member of the development branch, has been released
today. Most importantly, it now contains the ability to drop privileges,
thus enabling it to be executed without root permissions even if
listening to privileged network ports. The release also contains a fix
for a memory leak in the postgres output module and a fix for the klog
input module, which did not compile under BSD.

Download:
http://www.rsyslog.com/Article317.phtml

Change Log:
http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-140.phtml

The "privilege drop" functionality was discussed on this list and, while
not perfect, seems to provide some useful service. A number of
restrictions is described here:

http://wiki.rsyslog.com/index.php/Security

I would like to express my gratitude for all opinions raised. If you
have any more comments, please let them flow. Also feel free to add
anything of interest to the Wiki. Obviously, I would be very interested
in feedback from folks who really try the new functionality out.

I hope this release is useful.

Rainer



From mikel at irontec.com  Wed Nov 26 16:54:51 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 16:54:51 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
Message-ID: <492D714B.4050107@irontec.com>

Rainer Gerhards escribi?:
> Hi Mikel,
>
> which version are you using? The "ssh hang" can probably be related to
> the initial thought that imuxsock messages can be delayed. This has been
> changed in recent releases.
>
> Other than that, please note that actions are retried in intervals and
> these intervals get longer each time the action is retried without
> success. This is done in order to save unnecessary work. I think there
> currently is no config setting to change the default intervals.
>
> It may be useful to get a shutdown debug log from such an occurrence.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Wednesday, November 26, 2008 4:25 PM
>> To: rsyslog-users
>> Subject: [rsyslog] rsyslog freeze dude
>>
>> Hello
>> If I configure rsyslog client like this:
>>
>> WorkDirectory /root/rsyslog
>> $ActionQueueType LinkedList
>> $ActionQueueFileName myfile
>> $ActionResumeRetryCount -1
>>
>> and
>>
>> *.* @@somehost.domain.com
>>
>> Is possible that lost of conectivity (internet down) of this server
>> during days, produce that server freeze? or not be possible to login
>>     
> to
>   
>> the machine?
>>
>> If I do /etc/init.d/rsyslog restart it takes long long time to
>>     
> restart.
>   
>> Please Rainer I hope you help me. thanks
>>
>>
>> --
>> Mikel Jimenez Fernandez
>> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
>> +34 94.404.81.82
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Hello Rainer


root at ironpbx:~# rsyslogd -v
rsyslogd 3.16.1, compiled with:
      FEATURE_REGEXP:           Yes
      FEATURE_LARGEFILE:        Yes
      FEATURE_NETZIP (message compression):     Yes
      GSSAPI Kerberos 5 support:                No
      FEATURE_DEBUG (debug build, slow code):   No
      Runtime Instrumentation (slow code):      No

See http://www.rsyslog.com for more information.



Say me if you need more information.

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 16:56:52 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:56:52 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D714B.4050107@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
	<492D714B.4050107@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>

Hi Mikel,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Wednesday, November 26, 2008 4:55 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog freeze dude
 
> root at ironpbx:~# rsyslogd -v
> rsyslogd 3.16.1, compiled with:
>       FEATURE_REGEXP:           Yes
>       FEATURE_LARGEFILE:        Yes
>       FEATURE_NETZIP (message compression):     Yes
>       GSSAPI Kerberos 5 support:                No
>       FEATURE_DEBUG (debug build, slow code):   No
>       Runtime Instrumentation (slow code):      No
> 
> See http://www.rsyslog.com for more information.

Please upgrade to the recent v3-stable. That will probably solve the big
issue, if not all.

Rainer


From mikel at irontec.com  Wed Nov 26 17:03:15 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 17:03:15 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
Message-ID: <492D7343.1040101@irontec.com>

Rainer Gerhards escribi?:
> Hi Mikel,
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Wednesday, November 26, 2008 4:55 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog freeze dude
>>     
>  
>   
>> root at ironpbx:~# rsyslogd -v
>> rsyslogd 3.16.1, compiled with:
>>       FEATURE_REGEXP:           Yes
>>       FEATURE_LARGEFILE:        Yes
>>       FEATURE_NETZIP (message compression):     Yes
>>       GSSAPI Kerberos 5 support:                No
>>       FEATURE_DEBUG (debug build, slow code):   No
>>       Runtime Instrumentation (slow code):      No
>>
>> See http://www.rsyslog.com for more information.
>>     
>
> Please upgrade to the recent v3-stable. That will probably solve the big
> issue, if not all.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
OK

What is the last stable version?

There is a debian package?
Other thing:
The workDirectory is necesary that exist?
For example: WorkDirectory /rsyslog/work is a directory in / called 
rsyslog and another one inside this called work?

If they aren?t can cause a bug?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 17:13:10 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 17:13:10 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D7343.1040101@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
	<492D7343.1040101@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>

> OK
> 
> What is the last stable version?

I checked, it is 3.20.0 (and while checking I noticed that the project status was once again outdated ;))

> 
> There is a debian package?

This I don't know.

> Other thing:
> The workDirectory is necesary that exist?
Yes, absolutely

> For example: WorkDirectory /rsyslog/work is a directory in / called
> rsyslog and another one inside this called work?
> 
> If they aren?t can cause a bug?

I'd say yes.

Rainer


From mikel at irontec.com  Wed Nov 26 17:22:37 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 17:22:37 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
Message-ID: <492D77CD.3000204@irontec.com>

Rainer Gerhards escribi?:
>> OK
>>
>> What is the last stable version?
>>     
>
> I checked, it is 3.20.0 (and while checking I noticed that the project status was once again outdated ;))
>
>   
>> There is a debian package?
>>     
>
> This I don't know.
>
>   
>> Other thing:
>> The workDirectory is necesary that exist?
>>     
> Yes, absolutely
>
>   
>> For example: WorkDirectory /rsyslog/work is a directory in / called
>> rsyslog and another one inside this called work?
>>
>> If they aren?t can cause a bug?
>>     
>
> I'd say yes.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   

Before installing this version, would be possible to do more debugging?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 17:35:13 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 17:35:13 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D77CD.3000204@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
	<492D77CD.3000204@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>

> Before installing this version, would be possible to do more
debugging?

Sorry, I don't want to upset you, but I have no time to look at things
already fixed. Quite seriously, we have paid support options for this. I
see that it can make sense for you to stay with the older version, but
there is absolutely no benefit to the community. Sorry...

Rainer


From david at lang.hm  Wed Nov 26 18:57:05 2008
From: david at lang.hm (david at lang.hm)
Date: Wed, 26 Nov 2008 09:57:05 -0800 (PST)
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <alpine.DEB.1.10.0811260954070.22089@asgard.lang.hm>

On Wed, 26 Nov 2008, Michael Biebl wrote:

> 2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> However, looking at other projects, many have adapted a web-based
>> approach where version differences are flagged within the documentation.
>> The advantage of a web-based doc is that we can use thinks like a wiki
>> to generate it. That, I think, would make it much easier for users to
>> contribute doc or at least samples. Also, it looks like a web-based doc
>> is also more convenient for most users. Everyone has a browser open and
>> checks the web, but who installs doc packages? ;)
>>
>> So I am now consider changing to a web-based system, too. I'd probably
>> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
>> point. Since I created it, it receives a slow but steady traffic
>> increase and has now around the same number of hits than the rsyslog
>> main site. I would move over the current file-based doc into that system
>> and at the same time see that I can improve the structure and usability
>> of the doc.
>>
>> With the many new and powerful features that appeared in rsyslog over
>> the past couple of month, I think it is very important to make them
>> accessible by a sufficiently good doc. The current one is, to phrase it
>> politely, not good. It probably even hinders adoption of rsyslog in some
>> cases.
>>
>> With a web-based system open to user contributions I hope to solve this
>> issues.
>>
>> Please let me know your thoughts. All feedback is deeply appreciated.
>
> Imho it would be very unfortunate, if the documentation would only be
> available online.

I definantly agree. whatever is done online needs to have some way of 
having a snapshot of it done to be distributed for offline access.

> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.

the other problem with a wiki is that unless you have the time to review 
the changes there is a real risk that the documentation will be wrong. 
this is true of any user-contributed documentation, but with a wiki there 
is no way for other users to know what has been checked and is correct vs 
what is wrong.

the suggestion elsewhere in this thread to have an authoritative source 
with the ability for people to make comments does a good job of 
differentiating between the two.

David Lang

> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.
>
> Cheers,
> Michael
>
>
> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html
>


From hks.private at gmail.com  Wed Nov 26 21:20:51 2008
From: hks.private at gmail.com ((private) HKS)
Date: Wed, 26 Nov 2008 15:20:51 -0500
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
	<1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>
Message-ID: <ac9ded160811261220i7fb06ff7r7cfe3a69f010db91@mail.gmail.com>

On Wed, Nov 26, 2008 at 7:43 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Hi all,
>
> some times it seems to help ask around ;) Andre who cares for the web
> sites (also the phpLogCon lead devel) has pointed out to me that we may
> try the CMS' comment function. We have now enabled that. It seems to
> solve the immediate need. So this looks like a way we can at least try
> out.
>
> Anyhow, if you have any additional thoughts please let me know them.
> They probably help shape a medium-term doc strategy.
>
> Thanks,
> Rainer


I agree with most contributors so far, that a typical user-contributed
wiki alone is not sufficient. For a perfect example of a great project
crippled by abysmal wiki documentation, see OpenNMS.

Wikis /can/ work as official documentation, provided that they are
carefully thought through and official sources are the main
contributors. A general structure also needs to be in place so that
things don't spread out into a web of maze-like links with multiple
pages describing the same subject, albeit with small and crucial
differences. Their main benefits are, obviously, some measure of
self-correction and allowing users to share their solutions to
problems the developers may not have encountered.


Don't know if that helps, but I hope so.

-HKS


From mikel at irontec.com  Thu Nov 27 10:07:39 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 27 Nov 2008 10:07:39 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
Message-ID: <492E635B.80607@irontec.com>

Rainer Gerhards escribi?:
>> Before installing this version, would be possible to do more
>>     
> debugging?
>
> Sorry, I don't want to upset you, but I have no time to look at things
> already fixed. Quite seriously, we have paid support options for this. I
> see that it can make sense for you to stay with the older version, but
> there is absolutely no benefit to the community. Sorry...
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Ok Rainer
Thanks, really. I?m going to use the stable version.

Last question:
Why could be the reason, if I block internet in this host, and when pass 
10-12 hours, obviously not logging to remote (because firewall), but why 
not locally to?

I do:
logger "message"

And I have not notice in /var/log/syslog or /var/log/messages, and in 
normal situations yes...

Thanks

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From mikel at irontec.com  Thu Nov 27 10:19:25 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 27 Nov 2008 10:19:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>	<577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
Message-ID: <492E661D.8020501@irontec.com>

Andre Lorbach escribi?:
> By not adding it into the standard schema, you mean the default table
> layout for the database? 
> We may could reuse an existing unused field instead?
>
> --
> Andre 
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Mittwoch, 19. November 2008 10:39
>> To: rsyslog-users
>> Subject: Re: [rsyslog] milliseconds timestamp
>>
>> Andre,
>>
>> So do you think we should add this to the standard schema? I am a bit
>> hesitant, as not all folks will probably want that and it keeps
>> backward
>> compatibility. Maybe it is better to just us a standard name but not
>> include it in the standard schema.
>>
>> What do you (and others) think?
>>
>> Rainer
>>
>>     
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>>> Sent: Wednesday, November 19, 2008 10:37 AM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>
>>> I think this would be possible yes, we need to define a common name
>>>       
>> for
>>     
>>> the database and property field in rsyslog, then we can start adding
>>> support for it in phplogcon.
>>>
>>> regards,
>>> Andre
>>>
>>>       
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of mikel
>>>> Sent: Dienstag, 18. November 2008 21:06
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>
>>>>
>>>> Dear Andre
>>>>
>>>> So, in near future will be possible?
>>>>
>>>> Or in far future?
>>>>
>>>> Thanx
>>>>
>>>> On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
>>>> <rgerhards at hq.adiscon.com> wrote:
>>>>         
>>>>> Andre and all,
>>>>>
>>>>> the rsyslog engine has the necessary plumbing and I know of at
>>>>>           
>>> least
>>>       
>>>> one
>>>>         
>>>>> user who stores subsecond timestamps in this way. As far as
>>>>>           
>> rsyslog
>>     
>>>> is
>>>>         
>>>>> concerned, this just requires a special timestamp. So if you
>>>>>           
>> would
>>     
>>>>> support that in phpLogCon (what I would find a good idea ;)), I
>>>>>           
>>> would
>>>       
>>>>> include a standard template into rsyslog (hardcoded). That would
>>>>>           
>>> make
>>>       
>>>> it
>>>>         
>>>>> easier for people to work with it. But maybe we get along by
>>>>>           
> just
>   
>>>>> properly documenting it, because the schema must be modified in
>>>>>           
>> any
>>     
>>>>> case.
>>>>>
>>>>> Another alternative would be to update the default schema, but I
>>>>>           
>> am
>>     
>>>> not
>>>>         
>>>>> sure it that is actually a good idea. On the other hand, high-
>>>>>           
>>>> precision
>>>>         
>>>>> timestamps hopefully get into more widespread use...
>>>>>
>>>>> Thoughts from the community are appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rainer
>>>>>
>>>>>           
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>>>>>> Sent: Tuesday, November 18, 2008 5:56 PM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>>>
>>>>>> I think it wouldn't be so difficult to implement, if the
>>>>>>             
>>>> milliseconds
>>>>         
>>>>>> were simply stored in a second field.
>>>>>> We can add custom new fields into phpLogCon very easily.
>>>>>>
>>>>>> best regards,
>>>>>> Andre Lorbach
>>>>>>
>>>>>>             
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>>>>>> Sent: Dienstag, 18. November 2008 16:39
>>>>>>> To: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>>>>
>>>>>>> Hi Mikel,
>>>>>>>
>>>>>>> I have talked to Andre, phpLogCon's lead developer. He says
>>>>>>>               
>>> MySQL
>>>       
>>>>>> does
>>>>>>             
>>>>>>> not support millisecond resolution in timestamps. So a
>>>>>>>               
>>> work-around
>>>       
>>>>>>> would
>>>>>>> be to store them to another field and add this as a second
>>>>>>>               
>> field
>>     
>>>> to
>>>>         
>>>>>> the
>>>>>>             
>>>>>>> phpLogCon view. I think it would probably smart to enable
>>>>>>>               
>>>> phpLogCon
>>>>         
>>>>>> to
>>>>>>             
>>>>>>> combine two such fields, but I leave this to Andre...
>>>>>>>
>>>>>>> Rainer
>>>>>>>
>>>>>>>               
>>>>>>>> -----Original Message-----
>>>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>>>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>>>>>>>> Sent: Tuesday, November 18, 2008 2:10 PM
>>>>>>>> To: rsyslog-users
>>>>>>>> Subject: [rsyslog] milliseconds timestamp
>>>>>>>>
>>>>>>>> Hello
>>>>>>>> I am using phplogcon viewer and I have loosed milliseconds
>>>>>>>>                 
>>>>>>> timestamps.
>>>>>>>               
>>>>>>>> I watched that "DATE" in mysql databases is date-time type.
>>>>>>>>
>>>>>>>> Does rsyslog gives the date in milliseconds? is possible to
>>>>>>>>                 
>>> see
>>>       
>>>>>> this
>>>>>>             
>>>>>>>> milliseconds din phplogcon?
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mikel Jimenez Fernandez
>>>>>>>> Irontec, Internet y Sistemas sobre GNU/LinuX -
>>>>>>>>                 
>>>>>> http://www.irontec.com
>>>>>>             
>>>>>>>> +34 94.404.81.82
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>                 
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Hello Andre/Rainer

Have do you decide how to progress with that?

Can I help

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Fri Nov 28 09:50:29 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 09:50:29 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492E635B.80607@irontec.com>
References: <492D6A30.9060105@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
	<492D714B.4050107@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
	<492D7343.1040101@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
	<492D77CD.3000204@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
	<492E635B.80607@irontec.com>
Message-ID: <1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>

Mikel,

sorry, I was busy yesterday. See inline...
On Thu, 2008-11-27 at 10:07 +0100, Mikel Jimenez wrote:

> Last question:
> Why could be the reason, if I block internet in this host, and when pass 
> 10-12 hours, obviously not logging to remote (because firewall), but why 
> not locally to?
> 
> I do:
> logger "message"
> 
> And I have not notice in /var/log/syslog or /var/log/messages, and in 
> normal situations yes...

If I am right (the update will prove that) the reason is that the local
log socket is not read (because rsyslog delays imuxsock) and thus fills
up. Then, other processes trying to write to that socket will block
during the write process.

Please note that the rsyslog core will discard messages after some time,
but the rate at which this happens is too slow to get the system back to
normal. As I said, this should disappear with the recent stable version.

HTH
Rainer



From mikel at irontec.com  Fri Nov 28 10:41:37 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Fri, 28 Nov 2008 10:41:37 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com>
	<1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
Message-ID: <492FBCD1.7010509@irontec.com>

Rainer Gerhards escribi?:
> Mikel,
>
> sorry, I was busy yesterday. See inline...
> On Thu, 2008-11-27 at 10:07 +0100, Mikel Jimenez wrote:
>
>   
>> Last question:
>> Why could be the reason, if I block internet in this host, and when pass 
>> 10-12 hours, obviously not logging to remote (because firewall), but why 
>> not locally to?
>>
>> I do:
>> logger "message"
>>
>> And I have not notice in /var/log/syslog or /var/log/messages, and in 
>> normal situations yes...
>>     
>
> If I am right (the update will prove that) the reason is that the local
> log socket is not read (because rsyslog delays imuxsock) and thus fills
> up. Then, other processes trying to write to that socket will block
> during the write process.
>
> Please note that the rsyslog core will discard messages after some time,
> but the rate at which this happens is too slow to get the system back to
> normal. As I said, this should disappear with the recent stable version.
>
> HTH
> Rainer
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Thanks for your explication Rainer!! I will install 3.20 right now!!

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Fri Nov 28 10:43:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 10:43:22 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492FBCD1.7010509@irontec.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com><1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
	<492FBCD1.7010509@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>

> Thanks for your explication Rainer!! I will install 3.20 right now!!

Please let me know if the problem then re-appears. I don't think so, but
if it does this is definitely something we should look into ASAP. I'd
also appreciate if you let me know if the issue is fixed after the
upgrade.

Rainer


From mikel at irontec.com  Fri Nov 28 10:46:05 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Fri, 28 Nov 2008 10:46:05 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com><1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>	<492FBCD1.7010509@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>
Message-ID: <492FBDDD.2070308@irontec.com>

Rainer Gerhards escribi?:
>> Thanks for your explication Rainer!! I will install 3.20 right now!!
>>     
>
> Please let me know if the problem then re-appears. I don't think so, but
> if it does this is definitely something we should look into ASAP. I'd
> also appreciate if you let me know if the issue is fixed after the
> upgrade.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Ok, I will do it

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From peter.matulis at canonical.com  Fri Nov 28 17:59:56 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 11:59:56 -0500
Subject: [rsyslog] property based filtering with regex operator
Message-ID: <4930238C.5040208@canonical.com>

Hi.  I tried the following rule to match any occurrence of 'sda' but it
did not work:

:msg, regex, ".*sda.*"         /var/log/properties/sd_regex

Replacing the regex with simply "sda" matches.  Do I not need to take
the entire message into account?

Peter


From peter.matulis at canonical.com  Fri Nov 28 19:05:54 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 13:05:54 -0500
Subject: [rsyslog] QueueDiscardSeverity defaults
Message-ID: <49303302.3090200@canonical.com>

Hi.  The documentation states that for both the main queue and any
action queue the default value for QueueDiscardSeverity is '4'.  In
another place it is stated to be '8' (to prevent any message loss).

Can anyone also confirm whether both numerical as well as textual
severity can be given as a value?  The docs state that text is ok for
MainMsg but not for Action.

Peter


From rgerhards at hq.adiscon.com  Fri Nov 28 19:54:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 19:54:44 +0100
Subject: [rsyslog] property based filtering with regex operator
In-Reply-To: <4930238C.5040208@canonical.com>
References: <4930238C.5040208@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F754@grfint2.intern.adiscon.com>

Hi Peter,

can you run this through the debug log? That spits out additional
information, maybe that points to the source of the problem.

It's evening over here, I do not have a test system right at hand and
will be off for family things over the weekend. If it's not urgent, I
can run a test on Monday.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 6:00 PM
> To: rsyslog-users
> Subject: [rsyslog] property based filtering with regex operator
> 
> Hi.  I tried the following rule to match any occurrence of 'sda' but
it
> did not work:
> 
> :msg, regex, ".*sda.*"         /var/log/properties/sd_regex
> 
> Replacing the regex with simply "sda" matches.  Do I not need to take
> the entire message into account?
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Nov 28 19:57:08 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 19:57:08 +0100
Subject: [rsyslog] QueueDiscardSeverity defaults
In-Reply-To: <49303302.3090200@canonical.com>
References: <49303302.3090200@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F755@grfint2.intern.adiscon.com>

Peter,

I think that's another doc issue.

The default is 8 now, it was 4 but this has shown to be very
unproductive. I think textual priorities can now be given, but will need
to double-check.

I'll see that I update the doc ASAP.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 7:06 PM
> To: rsyslog-users
> Subject: [rsyslog] QueueDiscardSeverity defaults
> 
> Hi.  The documentation states that for both the main queue and any
> action queue the default value for QueueDiscardSeverity is '4'.  In
> another place it is stated to be '8' (to prevent any message loss).
> 
> Can anyone also confirm whether both numerical as well as textual
> severity can be given as a value?  The docs state that text is ok for
> MainMsg but not for Action.
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Fri Nov 28 20:29:26 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 14:29:26 -0500
Subject: [rsyslog] broken link to 4.1.1 release
Message-ID: <49304696.4000403@canonical.com>

The download link to the latest devel release (4.1.1) is broken:


http://www.rsyslog.com/Downloads-req-getit-lid-140.phtml


Peter


From rgerhards at hq.adiscon.com  Sat Nov 29 16:54:31 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 29 Nov 2008 16:54:31 +0100
Subject: [rsyslog] broken link to 4.1.1 release
In-Reply-To: <49304696.4000403@canonical.com>
References: <49304696.4000403@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F75A@grfint2.intern.adiscon.com>

Hi Peter,

thanks for letting me know. It is fixed now.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 8:29 PM
> To: rsyslog-users
> Subject: [rsyslog] broken link to 4.1.1 release
> 
> The download link to the latest devel release (4.1.1) is broken:
> 
> 
> http://www.rsyslog.com/Downloads-req-getit-lid-140.phtml
> 
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mikel at irontec.com  Mon Nov  3 14:14:50 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Mon, 03 Nov 2008 14:14:50 +0100
Subject: [rsyslog] filter expression dude
Message-ID: <490EF94A.70704@irontec.com>

hello

Is this correct?

if $msg contains 'GUI_set' and not $msg contains 'VALIDATOR' then 
@@192.1.100.1


I want that if message contains GUI_set and no VALIDATOR logs remotelly 
in 192.1.100.1 via TCP


Thanks


From hks.private at gmail.com  Mon Nov  3 16:51:01 2008
From: hks.private at gmail.com ((private) HKS)
Date: Mon, 3 Nov 2008 10:51:01 -0500
Subject: [rsyslog] filter expression dude
In-Reply-To: <490EF94A.70704@irontec.com>
References: <490EF94A.70704@irontec.com>
Message-ID: <ac9ded160811030751v7220f0eas69a25b5faa926da7@mail.gmail.com>

Yes, that should work as expected.

-HKS

On Mon, Nov 3, 2008 at 8:14 AM, Mikel Jimenez <mikel at irontec.com> wrote:
> hello
>
> Is this correct?
>
> if $msg contains 'GUI_set' and not $msg contains 'VALIDATOR' then
> @@192.1.100.1
>
>
> I want that if message contains GUI_set and no VALIDATOR logs remotelly
> in 192.1.100.1 via TCP
>
>
> Thanks
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Wed Nov  5 17:46:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 5 Nov 2008 17:46:22 +0100
Subject: [rsyslog] IPv6 problem (was:  new v3-stable release candidate)
In-Reply-To: <alpine.DEB.1.10.0810281023570.635@asgard.lang.hm>
References: <577465F99B41C842AAFBE9ED71E70ABA44F4BB@grfint2.intern.adiscon.com>
	<alpine.DEB.1.10.0810281023570.635@asgard.lang.hm>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F555@grfint2.intern.adiscon.com>

David,

I had some time to set up a new test environment today and try to
reproduce the problem. Unfortunately, I still did not see it. If you
have any new information, that would be appreciated.

I would also appreciate if someone else could try rsyslog under the
conditions told by David.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, October 28, 2008 6:26 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] new v3-stable release candidate
> 
> On Tue, 28 Oct 2008, Rainer Gerhards wrote:
> 
> > Hi all,
> >
> > I am about to switch the v3-stable release to 3.20, which means it
> will
> > be based on the 3.19.x beta branch. I have created a test tarball,
> > available at:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-3.20.0.tar.gz
> >
> > I would appreciate if some could try it out and report any potential
> > issues they see. I intend to release this tomorrow or Thursday if
> > nothing bad happens.
> 
> one problem I have been seeing in the nextmaster branch is that with
> imudp
> on a box that has IPv6 in the kernel, but not IPv6 IP addresses on the
> box
> rsyslog goes into a loop (eating 100% cpu, but not doing anything)
when
> it
> recieves a IPv4 syslog packet.
> 
> Rainer has not been able to duplicate the problem, could someone else
> test
> this?
> 
> I've been seeing it on Debian amd64 boxes.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From friedl at hq.adiscon.com  Thu Nov  6 16:47:53 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Thu, 6 Nov 2008 16:47:53 +0100
Subject: [rsyslog] rsyslog 3.20.0 (stable) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F566@grfint2.intern.adiscon.com>

Hi all,

we have released rsyslog 3.20.0, the next iteration of the rsyslog
v3-stable branch. Most importantly, TLS support is now available in the
stable branches. This version includes all features of the current beta
plus a bugfix. For users of the v3-stable branch, it brings many feature
enhancements, too many to list all of them here. This is a recommended
update for all v3-stable branch users.

Changelog:

http://www.rsyslog.com/Article302.phtml

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-137.phtml

As always, feedback is appreciated.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .



From rgerhards at hq.adiscon.com  Thu Nov  6 17:53:27 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 6 Nov 2008 17:53:27 +0100
Subject: [rsyslog] rsyslog regular expressen checker/generator
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F56C@grfint2.intern.adiscon.com>

Hi all,

regular expressions are quite powerful, but the syntax in rsyslog is,
well, not easy to use. Also, as we have seen, the usual regex check
tools don't work always well with rsyslog's POSIX expressions. I have
created a web-based checker/generator today. It is more or less
finished, but of course needs fine-tuning. I would appreciate if some of
you could have a quick look and comment with whatever suggestion they
have. The URL is:

http://www.rsyslog.com/tool-regex

If this tool turns out to be useful (judging on comments and access
count over the next weeks), I will probably do some other online tools
aiding in config generation.

Thanks,
Rainer


From rgerhards at hq.adiscon.com  Tue Nov 11 17:41:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Nov 2008 17:41:44 +0100
Subject: [rsyslog] rsyslog 3.21.7 released, new beta
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F5E0@grfint2.intern.adiscon.com>

Hi all,

I have just released rsyslog 3.21.7, the new beta branch. This version
brings all features of the current development to the beta branch. It
also contains one new addition, that is the ability to replace a not
found regular expression match with a zero. This can be useful for
storing numerical values into databases. This is a recommended update
for all beta branch users. 

Please note that a new devel branch will follow shortly. Currently,
devel is *older* than the beta.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-138.phtml

Change Log:

http://www.rsyslog.com/Article305.phtml

As always, feedback is appreciated.

Best regards,
Rainer Gerhards


From patrick.shen at net-m.de  Wed Nov 12 11:06:03 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Wed, 12 Nov 2008 11:06:03 +0100
Subject: [rsyslog] FQDN with rsyslogd
Message-ID: <491AAA8B.5010805@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear Rainer,

Have a look at online man doc, maybe it's outdated. I find one thing I
really care about.

######################################################################
If the remote host is located in the same domain  as  the  host,
rsyslogd  is running on, only the simple hostname will be logged instead
of the whole fqdn.
######################################################################

Currently we are using rsyslog as client to send logs to central
loghost, actually in the same domain. But I wanna need FQDN in logs, not
simple hostname.

How need I do?

Many thanks,

- --
Patrick Shen
Operations Engineer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJGqqLkHhYtFevC+MRAoMlAJ9M1CeXQf27KTz0/GznkreUFYcb7QCeKjl6
8R3DzRHtpDZU+a0/B1XQny4=
=RUGG
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Wed Nov 12 11:51:53 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Nov 2008 11:51:53 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491AAA8B.5010805@net-m.de>
References: <491AAA8B.5010805@net-m.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>

Hi Patrick,

this is part of the sysklogd legacy code. While there is no longer much
of sysklogd in rsyslog, this part actually is. I need to figure out if
you can turn it off, I remember to have added such an option.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> Sent: Wednesday, November 12, 2008 11:06 AM
> To: rsyslog-users
> Subject: [rsyslog] FQDN with rsyslogd
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dear Rainer,
> 
> Have a look at online man doc, maybe it's outdated. I find one thing I
> really care about.
> 
> ######################################################################
> If the remote host is located in the same domain  as  the  host,
> rsyslogd  is running on, only the simple hostname will be logged
> instead
> of the whole fqdn.
> ######################################################################
> 
> Currently we are using rsyslog as client to send logs to central
> loghost, actually in the same domain. But I wanna need FQDN in logs,
> not
> simple hostname.
> 
> How need I do?
> 
> Many thanks,
> 
> - --
> Patrick Shen
> Operations Engineer
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJGqqLkHhYtFevC+MRAoMlAJ9M1CeXQf27KTz0/GznkreUFYcb7QCeKjl6
> 8R3DzRHtpDZU+a0/B1XQny4=
> =RUGG
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From patrick.shen at net-m.de  Thu Nov 13 03:11:36 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Thu, 13 Nov 2008 03:11:36 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
References: <491AAA8B.5010805@net-m.de>
	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
Message-ID: <491B8CD8.3020701@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Rainer,

I will appreciate if you could show me the magic code.

Best regards,
Patrick

Rainer Gerhards wrote:
> Hi Patrick,
> 
> this is part of the sysklogd legacy code. While there is no longer much
> of sysklogd in rsyslog, this part actually is. I need to figure out if
> you can turn it off, I remember to have added such an option.
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
>> Sent: Wednesday, November 12, 2008 11:06 AM
>> To: rsyslog-users
>> Subject: [rsyslog] FQDN with rsyslogd
>>
> Dear Rainer,
> 
> Have a look at online man doc, maybe it's outdated. I find one thing I
> really care about.
> 
> ######################################################################
> If the remote host is located in the same domain  as  the  host,
> rsyslogd  is running on, only the simple hostname will be logged
> instead
> of the whole fqdn.
> ######################################################################
> 
> Currently we are using rsyslog as client to send logs to central
> loghost, actually in the same domain. But I wanna need FQDN in logs,
> not
> simple hostname.
> 
> How need I do?
> 
> Many thanks,
> 
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJG4zYkHhYtFevC+MRAhtxAKCFQABa96nbmKFTtrhNN1scqz769gCeLI+J
MHPb4zA20/cupkxuCL6uqQ4=
=TyV0
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Thu Nov 13 09:56:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Nov 2008 09:56:22 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491B8CD8.3020701@net-m.de>
References: <491AAA8B.5010805@net-m.de>
	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>
	<491B8CD8.3020701@net-m.de>
Message-ID: <1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>

Hi Patrick,

the code in question is this:

http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008aac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171

doesn't look hard to add a global option to prevent it, I'll see when I
can do it. In the meantime, you can simply comment out the strip of the
local domain (line 1174 I guess, but you need to check if it works).

Rainer

On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Rainer,
> 
> I will appreciate if you could show me the magic code.
> 
> Best regards,
> Patrick
> 
> Rainer Gerhards wrote:
> > Hi Patrick,
> > 
> > this is part of the sysklogd legacy code. While there is no longer much
> > of sysklogd in rsyslog, this part actually is. I need to figure out if
> > you can turn it off, I remember to have added such an option.
> > 
> > Rainer
> > 
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> >> Sent: Wednesday, November 12, 2008 11:06 AM
> >> To: rsyslog-users
> >> Subject: [rsyslog] FQDN with rsyslogd
> >>
> > Dear Rainer,
> > 
> > Have a look at online man doc, maybe it's outdated. I find one thing I
> > really care about.
> > 
> > ######################################################################
> > If the remote host is located in the same domain  as  the  host,
> > rsyslogd  is running on, only the simple hostname will be logged
> > instead
> > of the whole fqdn.
> > ######################################################################
> > 
> > Currently we are using rsyslog as client to send logs to central
> > loghost, actually in the same domain. But I wanna need FQDN in logs,
> > not
> > simple hostname.
> > 
> > How need I do?
> > 
> > Many thanks,
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJG4zYkHhYtFevC+MRAhtxAKCFQABa96nbmKFTtrhNN1scqz769gCeLI+J
> MHPb4zA20/cupkxuCL6uqQ4=
> =TyV0
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From mikel at irontec.com  Thu Nov 13 12:37:55 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 13 Nov 2008 12:37:55 +0100
Subject: [rsyslog] filter expression dude
Message-ID: <491C1193.3060609@irontec.com>

Hello
I usually use startswitch 'xxxxx' but is anything similar to "endswith" ?
Or regex that makes "endwith" function?

Would be appreciated an example please

Thanks!!

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Thu Nov 13 13:55:55 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Nov 2008 13:55:55 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C1193.3060609@irontec.com>
References: <491C1193.3060609@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>

You can use 'xxxxx$' as a regular expression. For the property replacer,
I have recently written a tool where you can try this out:

http://www.rsyslog.com/tool-regex

Note, though, that the syntax for conditions is different. But you get
the idea...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Thursday, November 13, 2008 12:38 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] filter expression dude
> 
> Hello
> I usually use startswitch 'xxxxx' but is anything similar to
"endswith"
> ?
> Or regex that makes "endwith" function?
> 
> Would be appreciated an example please
> 
> Thanks!!
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mikel at irontec.com  Thu Nov 13 14:00:05 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 13 Nov 2008 14:00:05 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
References: <491C1193.3060609@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
Message-ID: <491C24D5.4000607@irontec.com>

Rainer Gerhards escribi?:
> You can use 'xxxxx$' as a regular expression. For the property replacer,
> I have recently written a tool where you can try this out:
>
> http://www.rsyslog.com/tool-regex
>
> Note, though, that the syntax for conditions is different. But you get
> the idea...
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Thursday, November 13, 2008 12:38 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: [rsyslog] filter expression dude
>>
>> Hello
>> I usually use startswitch 'xxxxx' but is anything similar to
>>     
> "endswith"
>   
>> ?
>> Or regex that makes "endwith" function?
>>
>> Would be appreciated an example please
>>
>> Thanks!!
>>
>> --
>> Mikel Jimenez Fernandez
>> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
>> +34 94.404.81.82
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
ok!
And in full example?
I have this
if $FROMHOST startswith 'deusto' then 
:ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
& ~

And I want al "fromhost" that ends with 'sto'. I dont understand how to 
do this with regex tool...
For example:
esto
ppesto
rtisto

Can you write me these practical example?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From r.bhatia at ipax.at  Thu Nov 13 13:45:34 2008
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Thu, 13 Nov 2008 13:45:34 +0100
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C1193.3060609@irontec.com>
References: <491C1193.3060609@irontec.com>
Message-ID: <491C216E.4020101@ipax.at>

Mikel Jimenez wrote:
> Hello
> I usually use startswitch 'xxxxx' but is anything similar to "endswith" ?
> Or regex that makes "endwith" function?
> 
> Would be appreciated an example please

thou i do not know about the actual implementation in rsyslog,
you can match an end-of-line with: "$".

e.g.
> raoul $ echo "my test"|grep test$
> my test
> raoul $ echo "my test2"|grep test$
> raoul $

maybe [1] might be of use.

cheers,
raoul
[1] http://www.addedbytes.com/cheat-sheets/regular-expressions-cheat-sheet/
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From hks.private at gmail.com  Thu Nov 13 15:17:58 2008
From: hks.private at gmail.com ((private) HKS)
Date: Thu, 13 Nov 2008 09:17:58 -0500
Subject: [rsyslog] filter expression dude
In-Reply-To: <491C24D5.4000607@irontec.com>
References: <491C1193.3060609@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F605@grfint2.intern.adiscon.com>
	<491C24D5.4000607@irontec.com>
Message-ID: <ac9ded160811130617s4926431ewc177f8137b52299@mail.gmail.com>

> ok!
> And in full example?
> I have this
> if $FROMHOST startswith 'deusto' then
> :ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
> & ~
>
> And I want al "fromhost" that ends with 'sto'. I dont understand how to
> do this with regex tool...
> For example:
> esto
> ppesto
> rtisto
>
> Can you write me these practical example?
>


:fromhost, regex, "sto$"
:ommysql:localhost,deustolog,Ursyslog,superf4rs1t4009
& ~

-HKS


From patrick.shen at net-m.de  Fri Nov 14 03:48:42 2008
From: patrick.shen at net-m.de (Patrick Shen)
Date: Fri, 14 Nov 2008 03:48:42 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
References: <491AAA8B.5010805@net-m.de>	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>	<491B8CD8.3020701@net-m.de>
	<1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
Message-ID: <491CE70A.8050108@net-m.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Rainer,

It works. Thank you very much. But I also find an issue:

###################################################################
2008-11-14T03:32:47.137221+01:00 hydra-t kernel: imklog 3.18.4, log
source = /proc/kmsg started.
2008-11-14T03:32:47.137273+01:00 hydra-t rsyslogd: [origin
software="rsyslogd" swVersion="3.18.4" x-pid="651"
x-info="http://www.rsyslog.com"] restart
...
2008-11-14T03:34:51.680382+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,679 [INFO] [main] [:] [] [] [] []
[] [] [] [] mbg.servlet.BillingGateway:
/opt/as/APP/mbg/WEB-INF/conf/MBGBillingProviderConfig.xml loaded
2008-11-14T03:34:51.785187+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,784 [INFO] [main] [:] [] [] [] []
[] [] [] [] base.db.MBGCounterManager: Starting MBGCounterManager
[dataSourceName:jdbc/dataSource/factory]
2008-11-14T03:34:51.785724+01:00 hydra-t.test.xxx.xxx MBG:
iris.tomcat20010 2008-11-14 03:34:51,785 [INFO] [main] [:] [] [] [] []
[] [] [] [] mbg.servlet.BillingGateway: Initialization finish. Start
waiting for request ...
...
2008-11-14T03:37:15.555764+01:00 hydra-t snmpd[1699]: Connection from
UDP: [192.168.4.15]:34136
2008-11-14T03:37:15.555829+01:00 hydra-t snmpd[1699]: Received SNMP
packet(s) from UDP: [192.168.4.15]:34136
2008-11-14T03:37:16.357732+01:00 hydra-t snmpd[1699]: Connection from
UDP: [192.168.4.15]:34136
#####################################################################

We use "Log4j" to produce application(MBG) logs to hydra-t.test.xxx.xxx
(localhost) via TCP. Every line of them is tagged with
hydra-t.test.xxx.xxx. That's working after you show me the code.

But some other logs still use shortname of FQDN (hydra-t), like rsyslog
itself and snmpd.

Is it relative to $template in rsyslog config file? We use "$template
RSYSLOG_TraditionalForwardFormat" currently.

Many thanks,
Patrick


Rainer Gerhards wrote:
> Hi Patrick,
> 
> the code in question is this:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008aac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171
> 
> doesn't look hard to add a global option to prevent it, I'll see when I
> can do it. In the meantime, you can simply comment out the strip of the
> local domain (line 1174 I guess, but you need to check if it works).
> 
> Rainer
> 
> On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> Hi Rainer,
> 
> I will appreciate if you could show me the magic code.
> 
> Best regards,
> Patrick
> 
> Rainer Gerhards wrote:
>>>> Hi Patrick,
>>>>
>>>> this is part of the sysklogd legacy code. While there is no longer much
>>>> of sysklogd in rsyslog, this part actually is. I need to figure out if
>>>> you can turn it off, I remember to have added such an option.
>>>>
>>>> Rainer
>>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
>>>>> Sent: Wednesday, November 12, 2008 11:06 AM
>>>>> To: rsyslog-users
>>>>> Subject: [rsyslog] FQDN with rsyslogd
>>>>>
>>>> Dear Rainer,
>>>>
>>>> Have a look at online man doc, maybe it's outdated. I find one thing I
>>>> really care about.
>>>>
>>>> ######################################################################
>>>> If the remote host is located in the same domain  as  the  host,
>>>> rsyslogd  is running on, only the simple hostname will be logged
>>>> instead
>>>> of the whole fqdn.
>>>> ######################################################################
>>>>
>>>> Currently we are using rsyslog as client to send logs to central
>>>> loghost, actually in the same domain. But I wanna need FQDN in logs,
>>>> not
>>>> simple hostname.
>>>>
>>>> How need I do?
>>>>
>>>> Many thanks,
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJHOcKkHhYtFevC+MRAjafAJ9aSXu91K9Hj6xj+tGs0SNWjk/vswCgiE9C
c6P4BTbHAnCa8+/hcw/MkGc=
=GeS4
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Fri Nov 14 08:24:21 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Nov 2008 08:24:21 +0100
Subject: [rsyslog] FQDN with rsyslogd
In-Reply-To: <491CE70A.8050108@net-m.de>
References: <491AAA8B.5010805@net-m.de>	<577465F99B41C842AAFBE9ED71E70ABA44F5E7@grfint2.intern.adiscon.com>	<491B8CD8.3020701@net-m.de><1226566582.19905.25.camel@rgf9dev.intern.adiscon.com>
	<491CE70A.8050108@net-m.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F621@grfint2.intern.adiscon.com>

Hi Patrick,

I think it would be good if you file a feature request in bugzilla. I
will try to have a look, but these messages (or more precisely
hostnames) stem back to some places where the hostname is generated. I
need to review the code more in-depth to see if there is one ultimate
place and, if not, if there is a reason for multiple places or if they
could be restructured to a single code module (very likely!).

There are many other things in the queue, but I'll check if that's
something easy to do to shuffle in between...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> Sent: Friday, November 14, 2008 3:49 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] FQDN with rsyslogd
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi Rainer,
> 
> It works. Thank you very much. But I also find an issue:
> 
> ###################################################################
> 2008-11-14T03:32:47.137221+01:00 hydra-t kernel: imklog 3.18.4, log
> source = /proc/kmsg started.
> 2008-11-14T03:32:47.137273+01:00 hydra-t rsyslogd: [origin
> software="rsyslogd" swVersion="3.18.4" x-pid="651"
> x-info="http://www.rsyslog.com"] restart
> ...
> 2008-11-14T03:34:51.680382+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,679 [INFO] [main] [:] [] [] [] []
> [] [] [] [] mbg.servlet.BillingGateway:
> /opt/as/APP/mbg/WEB-INF/conf/MBGBillingProviderConfig.xml loaded
> 2008-11-14T03:34:51.785187+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,784 [INFO] [main] [:] [] [] [] []
> [] [] [] [] base.db.MBGCounterManager: Starting MBGCounterManager
> [dataSourceName:jdbc/dataSource/factory]
> 2008-11-14T03:34:51.785724+01:00 hydra-t.test.xxx.xxx MBG:
> iris.tomcat20010 2008-11-14 03:34:51,785 [INFO] [main] [:] [] [] [] []
> [] [] [] [] mbg.servlet.BillingGateway: Initialization finish. Start
> waiting for request ...
> ...
> 2008-11-14T03:37:15.555764+01:00 hydra-t snmpd[1699]: Connection from
> UDP: [192.168.4.15]:34136
> 2008-11-14T03:37:15.555829+01:00 hydra-t snmpd[1699]: Received SNMP
> packet(s) from UDP: [192.168.4.15]:34136
> 2008-11-14T03:37:16.357732+01:00 hydra-t snmpd[1699]: Connection from
> UDP: [192.168.4.15]:34136
> #####################################################################
> 
> We use "Log4j" to produce application(MBG) logs to
hydra-t.test.xxx.xxx
> (localhost) via TCP. Every line of them is tagged with
> hydra-t.test.xxx.xxx. That's working after you show me the code.
> 
> But some other logs still use shortname of FQDN (hydra-t), like
rsyslog
> itself and snmpd.
> 
> Is it relative to $template in rsyslog config file? We use "$template
> RSYSLOG_TraditionalForwardFormat" currently.
> 
> Many thanks,
> Patrick
> 
> 
> Rainer Gerhards wrote:
> > Hi Patrick,
> >
> > the code in question is this:
> >
> >
>
http://git.adiscon.com/?p=rsyslog.git;a=blob;f=runtime/net.c;h=44c9008a
> ac0efd70fdc385c0b1b5974d9913e573;hb=HEAD#l1171
> >
> > doesn't look hard to add a global option to prevent it, I'll see
when
> I
> > can do it. In the meantime, you can simply comment out the strip of
> the
> > local domain (line 1174 I guess, but you need to check if it works).
> >
> > Rainer
> >
> > On Thu, 2008-11-13 at 03:11 +0100, Patrick Shen wrote:
> > Hi Rainer,
> >
> > I will appreciate if you could show me the magic code.
> >
> > Best regards,
> > Patrick
> >
> > Rainer Gerhards wrote:
> >>>> Hi Patrick,
> >>>>
> >>>> this is part of the sysklogd legacy code. While there is no
longer
> much
> >>>> of sysklogd in rsyslog, this part actually is. I need to figure
> out if
> >>>> you can turn it off, I remember to have added such an option.
> >>>>
> >>>> Rainer
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>>>> bounces at lists.adiscon.com] On Behalf Of Patrick Shen
> >>>>> Sent: Wednesday, November 12, 2008 11:06 AM
> >>>>> To: rsyslog-users
> >>>>> Subject: [rsyslog] FQDN with rsyslogd
> >>>>>
> >>>> Dear Rainer,
> >>>>
> >>>> Have a look at online man doc, maybe it's outdated. I find one
> thing I
> >>>> really care about.
> >>>>
> >>>>
> ######################################################################
> >>>> If the remote host is located in the same domain  as  the  host,
> >>>> rsyslogd  is running on, only the simple hostname will be logged
> >>>> instead
> >>>> of the whole fqdn.
> >>>>
> ######################################################################
> >>>>
> >>>> Currently we are using rsyslog as client to send logs to central
> >>>> loghost, actually in the same domain. But I wanna need FQDN in
> logs,
> >>>> not
> >>>> simple hostname.
> >>>>
> >>>> How need I do?
> >>>>
> >>>> Many thanks,
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> 
> iD8DBQFJHOcKkHhYtFevC+MRAjafAJ9aSXu91K9Hj6xj+tGs0SNWjk/vswCgiE9C
> c6P4BTbHAnCa8+/hcw/MkGc=
> =GeS4
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Nov 18 10:41:35 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 10:41:35 +0100
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>

Hi all,

I have been asked if it is possible that the property replacer provides
an option of emitting a Unix timestamp for timestamp fields. This sounds
reasonable, but there are a number of subtleties. They are outlined in
the forum thread, with this post:

http://kb.monitorware.com/post14667.html#p14667

being probably a good wrap-up of the problems.

I would appreciate if someone from the mailing list could point me to a
resource, or share an idea, or whatever, on how I may tackle the
problem. Note that the primary concern is that I need to generate the
correct timestamp for messages from different time zones (problems
outlined in mentioned post).

Any feedback is highly welcome.

Thanks,
Rainer


From denemici at libero.it  Tue Nov 18 10:57:04 2008
From: denemici at libero.it (denemici at libero.it)
Date: Tue, 18 Nov 2008 10:57:04 +0100 (CET)
Subject: [rsyslog] undefined symbol: mysql_init
Message-ID: <19844959.335851227002224564.JavaMail.defaultUser@defaultHost>

Hi all,
I have upgrade Rsyslog from the 1.13.1 version (perfectly working with 
mysql) to the 3.20.0.
I have installed the new version from the source package, 
i have followed this step,
1) ./configure --enable-mysql
2) make
3) make 
install

Installation went fine but when i try to run rsyslog i have this error


Could not load module '/usr/local/lib/rsyslog/ommysql.so', dlopen: 
/usr/local/lib/rsyslog/ommysql.so: undefined symbol: mysql_init

If useful this 
a piece of configure phase:
.....
checking mysql/mysql.h usability... yes

checking mysql/mysql.h presence... yes
checking for mysql/mysql.h... yes

checking for mysql_config... yes
checking for mysql_init in -lmysqlclient... 
yes
...
Rsyslog is installed on Red Hat Enterprise Linux AS U4

Linux hostname 
2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 x86_64 x86_64 
GNU/Linux

I try to find this error on the forum but i haven't found nothing :(


Thanks for any help.

Bye
denemici


From mikel at irontec.com  Tue Nov 18 14:10:20 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Tue, 18 Nov 2008 14:10:20 +0100
Subject: [rsyslog] milliseconds timestamp
Message-ID: <4922BEBC.3050606@irontec.com>

Hello
I am using phplogcon viewer and I have loosed milliseconds timestamps.

I watched that "DATE" in mysql databases is date-time type.

Does rsyslog gives the date in milliseconds? is possible to see this 
milliseconds din phplogcon?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From hks.private at gmail.com  Tue Nov 18 15:46:32 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 09:46:32 -0500
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>

On Tue, Nov 18, 2008 at 4:41 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Hi all,
>
> I have been asked if it is possible that the property replacer provides
> an option of emitting a Unix timestamp for timestamp fields. This sounds
> reasonable, but there are a number of subtleties. They are outlined in
> the forum thread, with this post:
>
> http://kb.monitorware.com/post14667.html#p14667
>
> being probably a good wrap-up of the problems.
>
> I would appreciate if someone from the mailing list could point me to a
> resource, or share an idea, or whatever, on how I may tackle the
> problem. Note that the primary concern is that I need to generate the
> correct timestamp for messages from different time zones (problems
> outlined in mentioned post).


Perhaps I'm overlooking a subtlety of the calculation, but why not
convert the time with respect to the host's locale, then apply the the
offset in seconds?

On the subject of leap seconds: epoch time has no such concept. This
is one of the (many) reasons it's crap and should be deserted in favor
of proper ISO timestamps or something similar, but it at least makes
your job a bit easier. To epoch time, every day is 86400 seconds, no
matter what.

-HKS


From rgerhards at hq.adiscon.com  Tue Nov 18 15:49:02 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 15:49:02 +0100
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>

> Perhaps I'm overlooking a subtlety of the calculation, but why not
> convert the time with respect to the host's locale, then apply the the
> offset in seconds?

Lol, there is a saying in German "you don't see the forest because of
the trees" which means you don't see the obvious thing, because it is so
obvious you are overlooking it. This sounds like a perfect case. Of
course, this solution is quite simple. Lol, at least I am glad I asked
;)

Thanks for hinting me.

Rainer


From hks.private at gmail.com  Tue Nov 18 15:51:29 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 09:51:29 -0500
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F673@grfint2.intern.adiscon.com>
Message-ID: <ac9ded160811180651l72d85273if6ebea5c979e0c7b@mail.gmail.com>

Glad to help.

-HKS


On Tue, Nov 18, 2008 at 9:49 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
>> Perhaps I'm overlooking a subtlety of the calculation, but why not
>> convert the time with respect to the host's locale, then apply the the
>> offset in seconds?
>
> Lol, there is a saying in German "you don't see the forest because of
> the trees" which means you don't see the obvious thing, because it is so
> obvious you are overlooking it. This sounds like a perfect case. Of
> course, this solution is quite simple. Lol, at least I am glad I asked
> ;)
>
> Thanks for hinting me.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Tue Nov 18 16:38:56 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 16:38:56 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <4922BEBC.3050606@irontec.com>
References: <4922BEBC.3050606@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>

Hi Mikel,

I have talked to Andre, phpLogCon's lead developer. He says MySQL does
not support millisecond resolution in timestamps. So a work-around would
be to store them to another field and add this as a second field to the
phpLogCon view. I think it would probably smart to enable phpLogCon to
combine two such fields, but I leave this to Andre...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Tuesday, November 18, 2008 2:10 PM
> To: rsyslog-users
> Subject: [rsyslog] milliseconds timestamp
> 
> Hello
> I am using phplogcon viewer and I have loosed milliseconds timestamps.
> 
> I watched that "DATE" in mysql databases is date-time type.
> 
> Does rsyslog gives the date in milliseconds? is possible to see this
> milliseconds din phplogcon?
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From alorbach at ro1.adiscon.com  Tue Nov 18 17:56:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Tue, 18 Nov 2008 17:56:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>

I think it wouldn't be so difficult to implement, if the milliseconds
were simply stored in a second field.
We can add custom new fields into phpLogCon very easily. 

best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Dienstag, 18. November 2008 16:39
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> Hi Mikel,
> 
> I have talked to Andre, phpLogCon's lead developer. He says MySQL does
> not support millisecond resolution in timestamps. So a work-around
> would
> be to store them to another field and add this as a second field to
the
> phpLogCon view. I think it would probably smart to enable phpLogCon to
> combine two such fields, but I leave this to Andre...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > Sent: Tuesday, November 18, 2008 2:10 PM
> > To: rsyslog-users
> > Subject: [rsyslog] milliseconds timestamp
> >
> > Hello
> > I am using phplogcon viewer and I have loosed milliseconds
> timestamps.
> >
> > I watched that "DATE" in mysql databases is date-time type.
> >
> > Does rsyslog gives the date in milliseconds? is possible to see this
> > milliseconds din phplogcon?
> >
> > --
> > Mikel Jimenez Fernandez
> > Irontec, Internet y Sistemas sobre GNU/LinuX -
http://www.irontec.com
> > +34 94.404.81.82
> >
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Nov 18 18:00:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Nov 2008 18:00:04 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>

Andre and all,

the rsyslog engine has the necessary plumbing and I know of at least one
user who stores subsecond timestamps in this way. As far as rsyslog is
concerned, this just requires a special timestamp. So if you would
support that in phpLogCon (what I would find a good idea ;)), I would
include a standard template into rsyslog (hardcoded). That would make it
easier for people to work with it. But maybe we get along by just
properly documenting it, because the schema must be modified in any
case.

Another alternative would be to update the default schema, but I am not
sure it that is actually a good idea. On the other hand, high-precision
timestamps hopefully get into more widespread use...

Thoughts from the community are appreciated.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Tuesday, November 18, 2008 5:56 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> I think it wouldn't be so difficult to implement, if the milliseconds
> were simply stored in a second field.
> We can add custom new fields into phpLogCon very easily.
> 
> best regards,
> Andre Lorbach
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Dienstag, 18. November 2008 16:39
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > Hi Mikel,
> >
> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
> does
> > not support millisecond resolution in timestamps. So a work-around
> > would
> > be to store them to another field and add this as a second field to
> the
> > phpLogCon view. I think it would probably smart to enable phpLogCon
> to
> > combine two such fields, but I leave this to Andre...
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > To: rsyslog-users
> > > Subject: [rsyslog] milliseconds timestamp
> > >
> > > Hello
> > > I am using phplogcon viewer and I have loosed milliseconds
> > timestamps.
> > >
> > > I watched that "DATE" in mysql databases is date-time type.
> > >
> > > Does rsyslog gives the date in milliseconds? is possible to see
> this
> > > milliseconds din phplogcon?
> > >
> > > --
> > > Mikel Jimenez Fernandez
> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> http://www.irontec.com
> > > +34 94.404.81.82
> > >
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From friedl at hq.adiscon.com  Tue Nov 18 18:03:16 2008
From: friedl at hq.adiscon.com (Florian Riedl)
Date: Tue, 18 Nov 2008 18:03:16 +0100
Subject: [rsyslog] rsyslog 4.1.0 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F681@grfint2.intern.adiscon.com>

Hi all,

We have released rsyslog 4.1.0 today. This is the initial development
branch version of the upcoming v4 series. The main theme is performance
enhancement, 4.1.0 contains many enhancements which make the engine
process many more events per second than the previous versions. This was
made possible thanks to large changes inside the rsyslog core, and
consequently there is ample room for bugs. So this version should be
used with care. Note that it also contains some additional bugfixes and
small feature enhancements which are not found in other versions.

Please report any issue you see when working with that version. We are
especially interested in any anomaly on IPv6 enabled systems without
IPv6 addresses. There is a problem report for such an environment, which
we could not yet reproduce.

In case you wonder: there is no explicit schedule for the first
v4-stable version. But expect it no earlier than February 2009.

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-139.phtml

Changelog:

http://www.rsyslog.com/Article308.phtml

Feedback is very appreciated for this new release.

Florian Riedl
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You
can contribute by reporting bugs, improve the software, or donate money
or equipment.

Commercial support contracts for rsyslog are available, and they help
finance continued maintenance.  Adiscon GmbH, a privately held German
company, is currently funding rsyslog development. We are always looking
for interesting development projects. For details on how to help, please
see http://www.rsyslog.com/doc-how2help.html .



From aoz.syn at gmail.com  Tue Nov 18 18:30:11 2008
From: aoz.syn at gmail.com (RB)
Date: Tue, 18 Nov 2008 10:30:11 -0700
Subject: [rsyslog] help requested: correctly calculating Unix Timestamps
In-Reply-To: <ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA44F656@grfint2.intern.adiscon.com>
	<ac9ded160811180646y4c626ffcvd74a2c1f24b086ba@mail.gmail.com>
Message-ID: <4255c2570811180930m77862fddu984f045127d7116f@mail.gmail.com>

> On the subject of leap seconds: epoch time has no such concept. This
> is one of the (many) reasons it's crap and should be deserted in favor
> of proper ISO timestamps or something similar, but it at least makes
> your job a bit easier. To epoch time, every day is 86400 seconds, no
> matter what.

I think your statement is mildly misleading.  As I understand it,
POSIX.1 epoch (as opposed to TAI) is ambiguous when it comes to leap
seconds, since certain timestamps can be interpreted as two separate
ISO timestamps - i.e. some values are repeated.  Therefore, POSIX.1
epochs most certainly account for leap seconds, but make conversions
of certain stamps difficult.

Regardless, you're right - ISO stamps (or TAI with a leap-second
table) is the way to go when your timestamps must be absolutely
precise across leap boundaries.


From mikel at irontec.com  Tue Nov 18 21:06:04 2008
From: mikel at irontec.com (mikel)
Date: Tue, 18 Nov 2008 21:06:04 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
Message-ID: <a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>


Dear Andre

So, in near future will be possible?

Or in far future? 

Thanx

On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
<rgerhards at hq.adiscon.com> wrote:
> Andre and all,
> 
> the rsyslog engine has the necessary plumbing and I know of at least one
> user who stores subsecond timestamps in this way. As far as rsyslog is
> concerned, this just requires a special timestamp. So if you would
> support that in phpLogCon (what I would find a good idea ;)), I would
> include a standard template into rsyslog (hardcoded). That would make it
> easier for people to work with it. But maybe we get along by just
> properly documenting it, because the schema must be modified in any
> case.
> 
> Another alternative would be to update the default schema, but I am not
> sure it that is actually a good idea. On the other hand, high-precision
> timestamps hopefully get into more widespread use...
> 
> Thoughts from the community are appreciated.
> 
> Thanks,
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>> Sent: Tuesday, November 18, 2008 5:56 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] milliseconds timestamp
>>
>> I think it wouldn't be so difficult to implement, if the milliseconds
>> were simply stored in a second field.
>> We can add custom new fields into phpLogCon very easily.
>>
>> best regards,
>> Andre Lorbach
>>
>> > -----Original Message-----
>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> > Sent: Dienstag, 18. November 2008 16:39
>> > To: rsyslog-users
>> > Subject: Re: [rsyslog] milliseconds timestamp
>> >
>> > Hi Mikel,
>> >
>> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
>> does
>> > not support millisecond resolution in timestamps. So a work-around
>> > would
>> > be to store them to another field and add this as a second field to
>> the
>> > phpLogCon view. I think it would probably smart to enable phpLogCon
>> to
>> > combine two such fields, but I leave this to Andre...
>> >
>> > Rainer
>> >
>> > > -----Original Message-----
>> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> > > Sent: Tuesday, November 18, 2008 2:10 PM
>> > > To: rsyslog-users
>> > > Subject: [rsyslog] milliseconds timestamp
>> > >
>> > > Hello
>> > > I am using phplogcon viewer and I have loosed milliseconds
>> > timestamps.
>> > >
>> > > I watched that "DATE" in mysql databases is date-time type.
>> > >
>> > > Does rsyslog gives the date in milliseconds? is possible to see
>> this
>> > > milliseconds din phplogcon?
>> > >
>> > > --
>> > > Mikel Jimenez Fernandez
>> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
>> http://www.irontec.com
>> > > +34 94.404.81.82
>> > >
>> > >
>> > > _______________________________________________
>> > > rsyslog mailing list
>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > > http://www.rsyslog.com
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From rsyslog at clark-communications.com  Tue Nov 18 21:23:51 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 12:23:51 -0800
Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
Message-ID: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>


Hello,

I have motivated the creation of a port of rsyslog to OpenBSD.
The port is intended to fully support the OpenBSD ports framework, and  
will build an OpenBSD binary package
suitable for installation via pkg_add.  See the message below for a  
copy of the port.

I have done some testing of this port on OpenBSD 4.4 (stable) under  
amd64.

I would very much appreciate any additional testing and/or comments  
from the rsyslog community.

Best regards,

Don


Begin forwarded message:
> From: Don Jackson <don.jackson at gmail.com>
> Date: November 18, 2008 11:07:45 AM PST
> To: ports at openbsd.org
> Subject: NEW: sysutils/rsyslog-3.20.0
>
>
> Tested on 4.4/amd64, definitely interested in more testing.
>
> $ cat ./pkg/DESCR
>
> A syslogd replacement
>
>
>
>
>
>
>


From rsyslog at clark-communications.com  Tue Nov 18 21:57:02 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 12:57:02 -0800
Subject: [rsyslog] rsyslogd security questions/comments
Message-ID: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>


Hello,

I was reading the man page for rsyslogd today, and saw:

      SECURITY THREATS

      There  is the potential for the rsyslogd daemon to be used
      as a conduit for a denial of service attack.  A rogue pro-
      gram(mer) could very easily flood the rsyslogd daemon with
      syslog messages resulting in the log files  consuming  all
      the remaining space on the filesystem.  Activating logging
      over the inet domain sockets will of course expose a  sys-
      tem  to  risks  outside  of programs or individuals on the
      local machine.

      There are a number of methods of protecting a machine:

      1.     Implement kernel firewalling to limit  which  hosts
             or networks have access to the 514/UDP socket.

      2.     Logging  can be directed to an isolated or non-root
             filesystem which, if filled, will  not  impair  the
             machine.

      3.     The  ext2  filesystem can be used which can be con-
             figured to limit a certain percentage of a filesys-
             tem  to  usage  by  root only.  NOTE that this will
             require rsyslogd to be run as a  non-root  process.
             ALSO  NOTE  that  this will prevent usage of remote
             logging on the default port since rsyslogd will  be
             unable to bind to the 514/UDP socket.

I had the following questions:

Would it be possible (optionally) to have rsyslogd chroot to a  
particular directory on startup?

That seems the safest.  One could configure a disk partition for log  
messages, configure rsyslogd to log there,
and also chroot to a directory on that partition, so if the rsyslogd  
process itself is compromised,
it can't do other damage.

There must be a way to have a daemon run as a non-root user, and also  
to open ports < 1024.
This seems to be done all the time on *bsd machines:

# ps -aux
USER       PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED       TIME  
COMMAND
root         1  0.0  0.0   428   356 ??  Is    Thu02PM    0:00.01 / 
sbin/init
_dhcp    22078  0.0  0.0   396   432 ??  Is    Thu03PM    0:00.01  
dhclient: bge0 (dhclient)
_syslogd 27943  0.0  0.0   452   812 ??  S     Thu03PM    0:00.19  
syslogd -a /var/empty/dev/log

I'm not sure how this is done, but it looks like chroot also supports  
changing the userid...

Just some thoughts,

Best regards,

Don









From david at lang.hm  Tue Nov 18 22:10:13 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 13:10:13 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
Message-ID: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, Don Jackson wrote:

> I had the following questions:
>
> Would it be possible (optionally) to have rsyslogd chroot to a
> particular directory on startup?
> That seems the safest.  One could configure a disk partition for log
> messages, configure rsyslogd to log there,

chroot doesn't help. if you have rsyslog set to log to a seperate 
partition it can only fill that partition, but it can fill _all of that 
partition even if you chroot into a subdirectory on that partition.

> and also chroot to a directory on that partition, so if the rsyslogd
> process itself is compromised,
> it can't do other damage.

that gives you protection if you are receiving logs from the network, but 
if you are receiving logs from /dev/log (local logs) you can't go into a 
chroot effectivly

> There must be a way to have a daemon run as a non-root user, and also
> to open ports < 1024.
> This seems to be done all the time on *bsd machines:

the POSIX standard still calls for ports < 1024 to require root to bind to 
them, different systems have different ways to be non-compliant with the 
standard and let you do so anyway. what OS are you using?

David Lang

> # ps -aux
> USER       PID %CPU %MEM   VSZ   RSS TT  STAT  STARTED       TIME
> COMMAND
> root         1  0.0  0.0   428   356 ??  Is    Thu02PM    0:00.01 /
> sbin/init
> _dhcp    22078  0.0  0.0   396   432 ??  Is    Thu03PM    0:00.01
> dhclient: bge0 (dhclient)
> _syslogd 27943  0.0  0.0   452   812 ??  S     Thu03PM    0:00.19
> syslogd -a /var/empty/dev/log
>
> I'm not sure how this is done, but it looks like chroot also supports
> changing the userid...
>
> Just some thoughts,
>
> Best regards,
>
> Don
>
>
>
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From hks.private at gmail.com  Tue Nov 18 22:57:58 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 16:57:58 -0500
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
Message-ID: <ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>

On Tue, Nov 18, 2008 at 4:10 PM,  <david at lang.hm> wrote:
> On Tue, 18 Nov 2008, Don Jackson wrote:
>
>> I had the following questions:
>>
>> Would it be possible (optionally) to have rsyslogd chroot to a
>> particular directory on startup?
>> That seems the safest.  One could configure a disk partition for log
>> messages, configure rsyslogd to log there,
>
> chroot doesn't help. if you have rsyslog set to log to a seperate
> partition it can only fill that partition, but it can fill _all of that
> partition even if you chroot into a subdirectory on that partition.

Yes, but chrooting precludes any possibility of a rogue syslog agent
filling up /other/ partitions. In the event that a security compromise
were found in the rsyslogd service itself, it also confines an
attacker's damage to the chrooted environment. Paired with non-root
privileges, that's decent insurance against a full-on machine ownage.


>> and also chroot to a directory on that partition, so if the rsyslogd
>> process itself is compromised,
>> it can't do other damage.
>
> that gives you protection if you are receiving logs from the network, but
> if you are receiving logs from /dev/log (local logs) you can't go into a
> chroot effectivly
>
>> There must be a way to have a daemon run as a non-root user, and also
>> to open ports < 1024.
>> This seems to be done all the time on *bsd machines:
>
> the POSIX standard still calls for ports < 1024 to require root to bind to
> them, different systems have different ways to be non-compliant with the
> standard and let you do so anyway. what OS are you using?

OpenBSD has solved problems like this with their own daemons by
following two (general) stages of startup:

 - Privileged, where the daemon reads its config file and opens the
necessary sockets (but doesn't yet listen on them). It then chroots
itself and drops its privileges:
 - Unprivileged, where the daemon begins communication with the rest
of the environment and performing whatever job is required of it

For a concise discussion of how this applies to Apache (the best
explanation I've found in their docs), see:
http://www.openbsd.org/faq/faq10.html#httpdchroot

-HKS


From rsyslog at clark-communications.com  Tue Nov 18 23:02:46 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Tue, 18 Nov 2008 14:02:46 -0800
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
Message-ID: <85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>


On Nov 18, 2008, at 1:10 PM, david at lang.hm wrote:

> On Tue, 18 Nov 2008, Don Jackson wrote:
>
>> I had the following questions:
>>
>> Would it be possible (optionally) to have rsyslogd chroot to a
>> particular directory on startup?
>> That seems the safest.  One could configure a disk partition for log
>> messages, configure rsyslogd to log there,
>
> chroot doesn't help. if you have rsyslog set to log to a seperate
> partition it can only fill that partition, but it can fill _all of  
> that
> partition even if you chroot into a subdirectory on that partition.

Agree that file systems or partitions can get filled up no matter what  
you do.
But it seems to me the safest thing to do is to enable rsyslog to be  
configured to
use a specific file system on a dedicated partition, and then always  
monitor that.
Clearly that is possible with rsyslog now, but chrooting would make  
this even more restrictive/secure.

If the (r)syslog clients are also using rsyslog, then they could be  
configured for reliable delivery,
and queue log msgs in the event the rsyslog "server" gets hosed,so one  
you fixed your rsyslog server,
they would then be able to deliver their logs.

Seems like that would be the best overall approach.

>
>> and also chroot to a directory on that partition, so if the rsyslogd
>> process itself is compromised,
>> it can't do other damage.
>
> that gives you protection if you are receiving logs from the  
> network, but
> if you are receiving logs from /dev/log (local logs) you can't go  
> into a
> chroot effectivly

Protecting the "server" from attacks from networked clients seems like  
a good idea.

To answer your later question, I use OpenBSD a lot.
Here is how the stock syslogd on OpenBSD provides a /dev/log for each  
chrooted app:

NAME
      syslogd - log systems messages

SYNOPSIS
      syslogd [-dnu] [-a path] [-f config_file] [-m mark_interval]
              [-p log_socket] [-s reporting_socket]

DESCRIPTION
      syslogd reads and logs messages to the system console, log  
files, pipes
      to other programs, other machines and/or users as specified by  
its con-
      figuration file.

      The options are as follows:

      -a path
              Specify a location where syslogd should place an  
additional log
              socket.  Up to about 20 additional logging sockets can  
be speci-
              fied.  The primary use for this is to place additional  
log sock-
              ets in /dev/log of various chroot filespaces.

So you can arrange for each chroot'ed application to have it's own  
copy of /dev/log in its chroot.

It looks like syslogd on OpenBSD itself doesn't chroot.  I am still  
thinking it might be possible to do so (see below), but maybe I am  
totally wrong about this.

>
>> There must be a way to have a daemon run as a non-root user, and also
>> to open ports < 1024.
>> This seems to be done all the time on *bsd machines:
>
> the POSIX standard still calls for ports < 1024 to require root to  
> bind to
> them, different systems have different ways to be non-compliant with  
> the
> standard and let you do so anyway. what OS are you using?

Sorry, I was not being clear nor accurate here.

What I meant to say was, OK, start up as root, open the resources you  
need, and then chroot, both to change your userid to something that is  
non-root ( user _rsyslog ?), and also
to put rsyslog into a restricted subset file system.  Could you not  
open up the network ports needed, and also /dev/log (for the benefit  
of apps running on that machine that will
write there?) before the call to chroot?

If not, it looks like what syslogd on OpenBSD does is separate itself  
into to processes, the main process runs as root, and then it spawns a  
child that runs as user _syslogd,
it is this child process that listens to log sockets, while the parent  
process gives the child access to write log files.

I'm just brainstorming here.  Perhaps what I am thinking is not  
possible.

Don




From david at lang.hm  Tue Nov 18 23:06:48 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 14:06:48 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
Message-ID: <alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, (private) HKS wrote:

> On Tue, Nov 18, 2008 at 4:10 PM,  <david at lang.hm> wrote:
>> On Tue, 18 Nov 2008, Don Jackson wrote:
>>
>>> I had the following questions:
>>>
>>> Would it be possible (optionally) to have rsyslogd chroot to a
>>> particular directory on startup?
>>> That seems the safest.  One could configure a disk partition for log
>>> messages, configure rsyslogd to log there,
>>
>> chroot doesn't help. if you have rsyslog set to log to a seperate
>> partition it can only fill that partition, but it can fill _all of that
>> partition even if you chroot into a subdirectory on that partition.
>
> Yes, but chrooting precludes any possibility of a rogue syslog agent
> filling up /other/ partitions. In the event that a security compromise
> were found in the rsyslogd service itself, it also confines an
> attacker's damage to the chrooted environment. Paired with non-root
> privileges, that's decent insurance against a full-on machine ownage.

true, but that's a very different problem than the one that Don quoted in 
his message

>>> and also chroot to a directory on that partition, so if the rsyslogd
>>> process itself is compromised,
>>> it can't do other damage.
>>
>> that gives you protection if you are receiving logs from the network, but
>> if you are receiving logs from /dev/log (local logs) you can't go into a
>> chroot effectivly
>>
>>> There must be a way to have a daemon run as a non-root user, and also
>>> to open ports < 1024.
>>> This seems to be done all the time on *bsd machines:
>>
>> the POSIX standard still calls for ports < 1024 to require root to bind to
>> them, different systems have different ways to be non-compliant with the
>> standard and let you do so anyway. what OS are you using?
>
> OpenBSD has solved problems like this with their own daemons by
> following two (general) stages of startup:
>
> - Privileged, where the daemon reads its config file and opens the
> necessary sockets (but doesn't yet listen on them). It then chroots
> itself and drops its privileges:
> - Unprivileged, where the daemon begins communication with the rest
> of the environment and performing whatever job is required of it
>
> For a concise discussion of how this applies to Apache (the best
> explanation I've found in their docs), see:
> http://www.openbsd.org/faq/faq10.html#httpdchroot

Yes, many applications drop privilages (and optionally chroot) after 
starting up.

I suspect that changing rsyslog to do this would be a fair bit of work 
(among other things it can't do a full restart without full privilages, so 
the historic HUP behavior couldn't work)

David Lang


From david at lang.hm  Tue Nov 18 23:13:43 2008
From: david at lang.hm (david at lang.hm)
Date: Tue, 18 Nov 2008 14:13:43 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
Message-ID: <alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>

On Tue, 18 Nov 2008, Don Jackson wrote:

> On Nov 18, 2008, at 1:10 PM, david at lang.hm wrote:
>
>> On Tue, 18 Nov 2008, Don Jackson wrote:
>>
>>
>>> and also chroot to a directory on that partition, so if the rsyslogd
>>> process itself is compromised,
>>> it can't do other damage.
>>
>> that gives you protection if you are receiving logs from the
>> network, but
>> if you are receiving logs from /dev/log (local logs) you can't go
>> into a
>> chroot effectivly
>
> Protecting the "server" from attacks from networked clients seems like
> a good idea.
>
> To answer your later question, I use OpenBSD a lot.
> Here is how the stock syslogd on OpenBSD provides a /dev/log for each
> chrooted app:

the issue isn't providing a /dev/log for each chroot app, that can be done 
easily from a syslogd that's not in the chroot by opening additional 
pipes, the problem is that if the syslogd itself is inside a chroot it 
cannot open anything _outside_ of that chroot, so it can't provide the 
/dev/log device for the main system.

and you don't want to open that before you go into the chroot becouse one 
of the ways to break out of a chroot involves having a file handle open to 
something outside of the chroot.

>>> There must be a way to have a daemon run as a non-root user, and also
>>> to open ports < 1024.
>>> This seems to be done all the time on *bsd machines:
>>
>> the POSIX standard still calls for ports < 1024 to require root to
>> bind to
>> them, different systems have different ways to be non-compliant with
>> the
>> standard and let you do so anyway. what OS are you using?
>
> Sorry, I was not being clear nor accurate here.
>
> What I meant to say was, OK, start up as root, open the resources you
> need, and then chroot, both to change your userid to something that is
> non-root ( user _rsyslog ?), and also
> to put rsyslog into a restricted subset file system.  Could you not
> open up the network ports needed, and also /dev/log (for the benefit
> of apps running on that machine that will
> write there?) before the call to chroot?

changing the userid will work, doing a chroot will not.

> If not, it looks like what syslogd on OpenBSD does is separate itself
> into to processes, the main process runs as root, and then it spawns a
> child that runs as user _syslogd,
> it is this child process that listens to log sockets, while the parent
> process gives the child access to write log files.

rsyslog is multi-threaded, not multi-process. all the threads have access 
to everything that any one thread has access to, so it wouldn't help to 
seperate things out without serious surgery to rsyslog.

you could do something like have one copy of rsyslog running inside a 
chroot, and another one running outside the chroot with the one outside 
the chroot not listening to the network but forwarding everything it 
receives to the one inside the chroot.

this may require a little bit of work to make work (depending on how 
outbound network connections are made from rsyslog)

David Lang


From hks.private at gmail.com  Tue Nov 18 23:29:43 2008
From: hks.private at gmail.com ((private) HKS)
Date: Tue, 18 Nov 2008 17:29:43 -0500
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
Message-ID: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>

FWIW, while I believe this discussion is relevant and the security
changes proposed are important, I don't see this as a high priority
feature. The scripting language/syntax, greater performance, continued
stability enhancements, and true RELP support are all far more
important. As far as security goes, I would much rather see two-way
host authentication than a chroot/unprivilieged framework implemented.

Of course, all of the above would be just fine with me.

-HKS


From rgerhards at hq.adiscon.com  Wed Nov 19 09:39:18 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 09:39:18 +0100
Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
In-Reply-To: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>
References: <FFE0AA82-C4B0-4501-85C6-B0977ACB3214@clark-communications.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F683@grfint2.intern.adiscon.com>

Hi Don,

this is very good news. Please let me know if you run into any problems
that require an upstream fix. I have also seen your other message, but
will read through that thread before I reply.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Don Jackson
> Sent: Tuesday, November 18, 2008 9:24 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Port of rsyslog 3.20.0 to OpenBSD
> 
> 
> Hello,
> 
> I have motivated the creation of a port of rsyslog to OpenBSD.
> The port is intended to fully support the OpenBSD ports framework, and
> will build an OpenBSD binary package
> suitable for installation via pkg_add.  See the message below for a
> copy of the port.
> 
> I have done some testing of this port on OpenBSD 4.4 (stable) under
> amd64.
> 
> I would very much appreciate any additional testing and/or comments
> from the rsyslog community.
> 
> Best regards,
> 
> Don
> 
> 
> Begin forwarded message:
> > From: Don Jackson <don.jackson at gmail.com>
> > Date: November 18, 2008 11:07:45 AM PST
> > To: ports at openbsd.org
> > Subject: NEW: sysutils/rsyslog-3.20.0
> >
> >
> > Tested on 4.4/amd64, definitely interested in more testing.
> >
> > $ cat ./pkg/DESCR
> >
> > A syslogd replacement
> >
> >
> >
> >
> >
> >
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 09:46:58 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 09:46:58 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
Message-ID: <1227084418.19905.38.camel@rgf9dev.intern.adiscon.com>

Hi HKS,

just one point for now:

On Tue, 2008-11-18 at 17:29 -0500, (private) HKS wrote:
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

Two-way host authentication is available in the stable branch since
3.20.0. This is done by configuring the proper certficate settings in
TLS mode. As syslog-TLS will soon turn into a full-blown RFC, I expect
to see interoperable implementations in the not so distant future.

Rainer



From alorbach at ro1.adiscon.com  Wed Nov 19 10:37:15 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 19 Nov 2008 10:37:15 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com>
	<a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>

I think this would be possible yes, we need to define a common name for
the database and property field in rsyslog, then we can start adding
support for it in phplogcon. 

regards,
Andre

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of mikel
> Sent: Dienstag, 18. November 2008 21:06
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> 
> Dear Andre
> 
> So, in near future will be possible?
> 
> Or in far future?
> 
> Thanx
> 
> On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> <rgerhards at hq.adiscon.com> wrote:
> > Andre and all,
> >
> > the rsyslog engine has the necessary plumbing and I know of at least
> one
> > user who stores subsecond timestamps in this way. As far as rsyslog
> is
> > concerned, this just requires a special timestamp. So if you would
> > support that in phpLogCon (what I would find a good idea ;)), I
would
> > include a standard template into rsyslog (hardcoded). That would
make
> it
> > easier for people to work with it. But maybe we get along by just
> > properly documenting it, because the schema must be modified in any
> > case.
> >
> > Another alternative would be to update the default schema, but I am
> not
> > sure it that is actually a good idea. On the other hand, high-
> precision
> > timestamps hopefully get into more widespread use...
> >
> > Thoughts from the community are appreciated.
> >
> > Thanks,
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> >> Sent: Tuesday, November 18, 2008 5:56 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] milliseconds timestamp
> >>
> >> I think it wouldn't be so difficult to implement, if the
> milliseconds
> >> were simply stored in a second field.
> >> We can add custom new fields into phpLogCon very easily.
> >>
> >> best regards,
> >> Andre Lorbach
> >>
> >> > -----Original Message-----
> >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> >> > Sent: Dienstag, 18. November 2008 16:39
> >> > To: rsyslog-users
> >> > Subject: Re: [rsyslog] milliseconds timestamp
> >> >
> >> > Hi Mikel,
> >> >
> >> > I have talked to Andre, phpLogCon's lead developer. He says MySQL
> >> does
> >> > not support millisecond resolution in timestamps. So a
work-around
> >> > would
> >> > be to store them to another field and add this as a second field
> to
> >> the
> >> > phpLogCon view. I think it would probably smart to enable
> phpLogCon
> >> to
> >> > combine two such fields, but I leave this to Andre...
> >> >
> >> > Rainer
> >> >
> >> > > -----Original Message-----
> >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> >> > > To: rsyslog-users
> >> > > Subject: [rsyslog] milliseconds timestamp
> >> > >
> >> > > Hello
> >> > > I am using phplogcon viewer and I have loosed milliseconds
> >> > timestamps.
> >> > >
> >> > > I watched that "DATE" in mysql databases is date-time type.
> >> > >
> >> > > Does rsyslog gives the date in milliseconds? is possible to see
> >> this
> >> > > milliseconds din phplogcon?
> >> > >
> >> > > --
> >> > > Mikel Jimenez Fernandez
> >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> >> http://www.irontec.com
> >> > > +34 94.404.81.82
> >> > >
> >> > >
> >> > > _______________________________________________
> >> > > rsyslog mailing list
> >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > > http://www.rsyslog.com
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 10:38:45 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 10:38:45 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>

Andre,

So do you think we should add this to the standard schema? I am a bit
hesitant, as not all folks will probably want that and it keeps backward
compatibility. Maybe it is better to just us a standard name but not
include it in the standard schema.

What do you (and others) think?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Wednesday, November 19, 2008 10:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> I think this would be possible yes, we need to define a common name
for
> the database and property field in rsyslog, then we can start adding
> support for it in phplogcon.
> 
> regards,
> Andre
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of mikel
> > Sent: Dienstag, 18. November 2008 21:06
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> >
> > Dear Andre
> >
> > So, in near future will be possible?
> >
> > Or in far future?
> >
> > Thanx
> >
> > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > <rgerhards at hq.adiscon.com> wrote:
> > > Andre and all,
> > >
> > > the rsyslog engine has the necessary plumbing and I know of at
> least
> > one
> > > user who stores subsecond timestamps in this way. As far as
rsyslog
> > is
> > > concerned, this just requires a special timestamp. So if you would
> > > support that in phpLogCon (what I would find a good idea ;)), I
> would
> > > include a standard template into rsyslog (hardcoded). That would
> make
> > it
> > > easier for people to work with it. But maybe we get along by just
> > > properly documenting it, because the schema must be modified in
any
> > > case.
> > >
> > > Another alternative would be to update the default schema, but I
am
> > not
> > > sure it that is actually a good idea. On the other hand, high-
> > precision
> > > timestamps hopefully get into more widespread use...
> > >
> > > Thoughts from the community are appreciated.
> > >
> > > Thanks,
> > > Rainer
> > >
> > >> -----Original Message-----
> > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > >> To: rsyslog-users
> > >> Subject: Re: [rsyslog] milliseconds timestamp
> > >>
> > >> I think it wouldn't be so difficult to implement, if the
> > milliseconds
> > >> were simply stored in a second field.
> > >> We can add custom new fields into phpLogCon very easily.
> > >>
> > >> best regards,
> > >> Andre Lorbach
> > >>
> > >> > -----Original Message-----
> > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > >> > Sent: Dienstag, 18. November 2008 16:39
> > >> > To: rsyslog-users
> > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > >> >
> > >> > Hi Mikel,
> > >> >
> > >> > I have talked to Andre, phpLogCon's lead developer. He says
> MySQL
> > >> does
> > >> > not support millisecond resolution in timestamps. So a
> work-around
> > >> > would
> > >> > be to store them to another field and add this as a second
field
> > to
> > >> the
> > >> > phpLogCon view. I think it would probably smart to enable
> > phpLogCon
> > >> to
> > >> > combine two such fields, but I leave this to Andre...
> > >> >
> > >> > Rainer
> > >> >
> > >> > > -----Original Message-----
> > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > >> > > To: rsyslog-users
> > >> > > Subject: [rsyslog] milliseconds timestamp
> > >> > >
> > >> > > Hello
> > >> > > I am using phplogcon viewer and I have loosed milliseconds
> > >> > timestamps.
> > >> > >
> > >> > > I watched that "DATE" in mysql databases is date-time type.
> > >> > >
> > >> > > Does rsyslog gives the date in milliseconds? is possible to
> see
> > >> this
> > >> > > milliseconds din phplogcon?
> > >> > >
> > >> > > --
> > >> > > Mikel Jimenez Fernandez
> > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > >> http://www.irontec.com
> > >> > > +34 94.404.81.82
> > >> > >
> > >> > >
> > >> > > _______________________________________________
> > >> > > rsyslog mailing list
> > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > > http://www.rsyslog.com
> > >> > _______________________________________________
> > >> > rsyslog mailing list
> > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> > http://www.rsyslog.com
> > >> _______________________________________________
> > >> rsyslog mailing list
> > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > >> http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From alorbach at ro1.adiscon.com  Wed Nov 19 10:48:25 2008
From: alorbach at ro1.adiscon.com (Andre Lorbach)
Date: Wed, 19 Nov 2008 10:48:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
Message-ID: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>

By not adding it into the standard schema, you mean the default table
layout for the database? 
We may could reuse an existing unused field instead?

--
Andre 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Mittwoch, 19. November 2008 10:39
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> Andre,
> 
> So do you think we should add this to the standard schema? I am a bit
> hesitant, as not all folks will probably want that and it keeps
> backward
> compatibility. Maybe it is better to just us a standard name but not
> include it in the standard schema.
> 
> What do you (and others) think?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > Sent: Wednesday, November 19, 2008 10:37 AM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > I think this would be possible yes, we need to define a common name
> for
> > the database and property field in rsyslog, then we can start adding
> > support for it in phplogcon.
> >
> > regards,
> > Andre
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of mikel
> > > Sent: Dienstag, 18. November 2008 21:06
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] milliseconds timestamp
> > >
> > >
> > > Dear Andre
> > >
> > > So, in near future will be possible?
> > >
> > > Or in far future?
> > >
> > > Thanx
> > >
> > > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > > <rgerhards at hq.adiscon.com> wrote:
> > > > Andre and all,
> > > >
> > > > the rsyslog engine has the necessary plumbing and I know of at
> > least
> > > one
> > > > user who stores subsecond timestamps in this way. As far as
> rsyslog
> > > is
> > > > concerned, this just requires a special timestamp. So if you
> would
> > > > support that in phpLogCon (what I would find a good idea ;)), I
> > would
> > > > include a standard template into rsyslog (hardcoded). That would
> > make
> > > it
> > > > easier for people to work with it. But maybe we get along by
just
> > > > properly documenting it, because the schema must be modified in
> any
> > > > case.
> > > >
> > > > Another alternative would be to update the default schema, but I
> am
> > > not
> > > > sure it that is actually a good idea. On the other hand, high-
> > > precision
> > > > timestamps hopefully get into more widespread use...
> > > >
> > > > Thoughts from the community are appreciated.
> > > >
> > > > Thanks,
> > > > Rainer
> > > >
> > > >> -----Original Message-----
> > > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > > >> To: rsyslog-users
> > > >> Subject: Re: [rsyslog] milliseconds timestamp
> > > >>
> > > >> I think it wouldn't be so difficult to implement, if the
> > > milliseconds
> > > >> were simply stored in a second field.
> > > >> We can add custom new fields into phpLogCon very easily.
> > > >>
> > > >> best regards,
> > > >> Andre Lorbach
> > > >>
> > > >> > -----Original Message-----
> > > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > >> > Sent: Dienstag, 18. November 2008 16:39
> > > >> > To: rsyslog-users
> > > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > > >> >
> > > >> > Hi Mikel,
> > > >> >
> > > >> > I have talked to Andre, phpLogCon's lead developer. He says
> > MySQL
> > > >> does
> > > >> > not support millisecond resolution in timestamps. So a
> > work-around
> > > >> > would
> > > >> > be to store them to another field and add this as a second
> field
> > > to
> > > >> the
> > > >> > phpLogCon view. I think it would probably smart to enable
> > > phpLogCon
> > > >> to
> > > >> > combine two such fields, but I leave this to Andre...
> > > >> >
> > > >> > Rainer
> > > >> >
> > > >> > > -----Original Message-----
> > > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > >> > > To: rsyslog-users
> > > >> > > Subject: [rsyslog] milliseconds timestamp
> > > >> > >
> > > >> > > Hello
> > > >> > > I am using phplogcon viewer and I have loosed milliseconds
> > > >> > timestamps.
> > > >> > >
> > > >> > > I watched that "DATE" in mysql databases is date-time type.
> > > >> > >
> > > >> > > Does rsyslog gives the date in milliseconds? is possible to
> > see
> > > >> this
> > > >> > > milliseconds din phplogcon?
> > > >> > >
> > > >> > > --
> > > >> > > Mikel Jimenez Fernandez
> > > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > > >> http://www.irontec.com
> > > >> > > +34 94.404.81.82
> > > >> > >
> > > >> > >
> > > >> > > _______________________________________________
> > > >> > > rsyslog mailing list
> > > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > > http://www.rsyslog.com
> > > >> > _______________________________________________
> > > >> > rsyslog mailing list
> > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> > http://www.rsyslog.com
> > > >> _______________________________________________
> > > >> rsyslog mailing list
> > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > >> http://www.rsyslog.com
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 10:49:28 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 10:49:28 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F688@grfint2.intern.adiscon.com>

No, I mean that we define a field name, but leave it to the user if he
or she adds/creates the field to the schema (actually two, one for
received and one for generated).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> Sent: Wednesday, November 19, 2008 10:48 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] milliseconds timestamp
> 
> By not adding it into the standard schema, you mean the default table
> layout for the database?
> We may could reuse an existing unused field instead?
> 
> --
> Andre
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Mittwoch, 19. November 2008 10:39
> > To: rsyslog-users
> > Subject: Re: [rsyslog] milliseconds timestamp
> >
> > Andre,
> >
> > So do you think we should add this to the standard schema? I am a
bit
> > hesitant, as not all folks will probably want that and it keeps
> > backward
> > compatibility. Maybe it is better to just us a standard name but not
> > include it in the standard schema.
> >
> > What do you (and others) think?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > Sent: Wednesday, November 19, 2008 10:37 AM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] milliseconds timestamp
> > >
> > > I think this would be possible yes, we need to define a common
name
> > for
> > > the database and property field in rsyslog, then we can start
> adding
> > > support for it in phplogcon.
> > >
> > > regards,
> > > Andre
> > >
> > > > -----Original Message-----
> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > bounces at lists.adiscon.com] On Behalf Of mikel
> > > > Sent: Dienstag, 18. November 2008 21:06
> > > > To: rsyslog-users
> > > > Subject: Re: [rsyslog] milliseconds timestamp
> > > >
> > > >
> > > > Dear Andre
> > > >
> > > > So, in near future will be possible?
> > > >
> > > > Or in far future?
> > > >
> > > > Thanx
> > > >
> > > > On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
> > > > <rgerhards at hq.adiscon.com> wrote:
> > > > > Andre and all,
> > > > >
> > > > > the rsyslog engine has the necessary plumbing and I know of at
> > > least
> > > > one
> > > > > user who stores subsecond timestamps in this way. As far as
> > rsyslog
> > > > is
> > > > > concerned, this just requires a special timestamp. So if you
> > would
> > > > > support that in phpLogCon (what I would find a good idea ;)),
I
> > > would
> > > > > include a standard template into rsyslog (hardcoded). That
> would
> > > make
> > > > it
> > > > > easier for people to work with it. But maybe we get along by
> just
> > > > > properly documenting it, because the schema must be modified
in
> > any
> > > > > case.
> > > > >
> > > > > Another alternative would be to update the default schema, but
> I
> > am
> > > > not
> > > > > sure it that is actually a good idea. On the other hand, high-
> > > > precision
> > > > > timestamps hopefully get into more widespread use...
> > > > >
> > > > > Thoughts from the community are appreciated.
> > > > >
> > > > > Thanks,
> > > > > Rainer
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
> > > > >> Sent: Tuesday, November 18, 2008 5:56 PM
> > > > >> To: rsyslog-users
> > > > >> Subject: Re: [rsyslog] milliseconds timestamp
> > > > >>
> > > > >> I think it wouldn't be so difficult to implement, if the
> > > > milliseconds
> > > > >> were simply stored in a second field.
> > > > >> We can add custom new fields into phpLogCon very easily.
> > > > >>
> > > > >> best regards,
> > > > >> Andre Lorbach
> > > > >>
> > > > >> > -----Original Message-----
> > > > >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > > > >> > Sent: Dienstag, 18. November 2008 16:39
> > > > >> > To: rsyslog-users
> > > > >> > Subject: Re: [rsyslog] milliseconds timestamp
> > > > >> >
> > > > >> > Hi Mikel,
> > > > >> >
> > > > >> > I have talked to Andre, phpLogCon's lead developer. He says
> > > MySQL
> > > > >> does
> > > > >> > not support millisecond resolution in timestamps. So a
> > > work-around
> > > > >> > would
> > > > >> > be to store them to another field and add this as a second
> > field
> > > > to
> > > > >> the
> > > > >> > phpLogCon view. I think it would probably smart to enable
> > > > phpLogCon
> > > > >> to
> > > > >> > combine two such fields, but I leave this to Andre...
> > > > >> >
> > > > >> > Rainer
> > > > >> >
> > > > >> > > -----Original Message-----
> > > > >> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > > >> > > bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> > > > >> > > Sent: Tuesday, November 18, 2008 2:10 PM
> > > > >> > > To: rsyslog-users
> > > > >> > > Subject: [rsyslog] milliseconds timestamp
> > > > >> > >
> > > > >> > > Hello
> > > > >> > > I am using phplogcon viewer and I have loosed
milliseconds
> > > > >> > timestamps.
> > > > >> > >
> > > > >> > > I watched that "DATE" in mysql databases is date-time
> type.
> > > > >> > >
> > > > >> > > Does rsyslog gives the date in milliseconds? is possible
> to
> > > see
> > > > >> this
> > > > >> > > milliseconds din phplogcon?
> > > > >> > >
> > > > >> > > --
> > > > >> > > Mikel Jimenez Fernandez
> > > > >> > > Irontec, Internet y Sistemas sobre GNU/LinuX -
> > > > >> http://www.irontec.com
> > > > >> > > +34 94.404.81.82
> > > > >> > >
> > > > >> > >
> > > > >> > > _______________________________________________
> > > > >> > > rsyslog mailing list
> > > > >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> > > http://www.rsyslog.com
> > > > >> > _______________________________________________
> > > > >> > rsyslog mailing list
> > > > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> > http://www.rsyslog.com
> > > > >> _______________________________________________
> > > > >> rsyslog mailing list
> > > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > >> http://www.rsyslog.com
> > > > > _______________________________________________
> > > > > rsyslog mailing list
> > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > > http://www.rsyslog.com
> > > >
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 19 13:32:08 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:32:08 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><ac9ded160811181357u1ea1e907l70def810944f570f@mail.gmail.com>
	<alpine.DEB.1.10.0811181404230.32032@asgard.lang.hm>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68A@grfint2.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, November 18, 2008 11:07 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
[snip]

> Yes, many applications drop privilages (and optionally chroot) after
> starting up.
> 
> I suspect that changing rsyslog to do this would be a fair bit of work
> (among other things it can't do a full restart without full
privilages,
> so
> the historic HUP behavior couldn't work)

Yes, the traditional HUP behavior is simply incompatible with dropping
privileges. But I assume that someone who configures rsyslogd in such a
way knows what he does and thus changes config-reload scripts away from
HUP to a real reload.

Rainer


From rgerhards at hq.adiscon.com  Wed Nov 19 13:46:01 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:46:01 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
Message-ID: <1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>

Hi all,

So, let me use this reply as a warp-up of my thoughts. Basically, I
agree to HKS argument. I also agree to David Lang's argument that some
degree of design change would be necessary to get the requested features
done 100% correct.

But, of course, it all depends on the effort required. And more security
is always welcome. So I took time out today to think about the options
and experiment a bit. I came to the conclusion that some improvement can
probably be made in short time. I experimentally implemented a new
$PrivDropToUser config directive, which basically issues a setuid() call
with the user in question. It is far from being bullet-proof and will
never be totally save without architecture changes. But I think it still
can considerably improve security and when we can achieve this with a
few hours of work, it's probably worth the effort (a bit more work would
improve its security, but I don't do this unless I get feedback it makes
sense at all).

HOWEVER, there are also some things that simply don't work out. For
example, imklog reads the proc file system under Linux and, even though
it is opened as root, these reads fail after dropping to another user.
There may be other issues, I have not further investigated that.

I have created a bugzilla enhancement request for it:

http://bugzilla.adiscon.com/show_bug.cgi?id=105

You may want to subscribe to that bug for updates. 

The experimental code is available here:

http://git.adiscon.com/?p=rsyslog.git;a=shortlog;h=refs/heads/droppriv

I would appreciate feedback

a) on the general issue
b) on the experimental code itself

Many thanks,
Rainer

On Tue, 2008-11-18 at 17:29 -0500, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.
> 
> Of course, all of the above would be just fine with me.
> 
> -HKS
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From mbiebl at gmail.com  Wed Nov 19 13:53:12 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 13:53:12 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
Message-ID: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>

2008/11/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> So, let me use this reply as a warp-up of my thoughts. Basically, I
> agree to HKS argument. I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested features
> done 100% correct.
>
> But, of course, it all depends on the effort required. And more security
> is always welcome. So I took time out today to think about the options
> and experiment a bit. I came to the conclusion that some improvement can
> probably be made in short time. I experimentally implemented a new
> $PrivDropToUser config directive, which basically issues a setuid() call
> with the user in question. It is far from being bullet-proof and will
> never be totally save without architecture changes. But I think it still
> can considerably improve security and when we can achieve this with a
> few hours of work, it's probably worth the effort (a bit more work would
> improve its security, but I don't do this unless I get feedback it makes
> sense at all).
>
> HOWEVER, there are also some things that simply don't work out. For
> example, imklog reads the proc file system under Linux and, even though
> it is opened as root, these reads fail after dropping to another user.
> There may be other issues, I have not further investigated that.

/dev/klog could be made accessible via an ACL or a separate group.
(e.g. the init script of the distro could chgrp or setfacl /dev/klog).
Distros will also have to make sure, that the rsyslog daemon has write
access to /var/log (or wherever they put their log files by default,
but that is easily solvable too).

More tricky will be the already mentioned opening of privileged ports I think.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Wed Nov 19 13:55:29 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 13:55:29 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>

2008/11/19 Michael Biebl <mbiebl at gmail.com>:

>
> /dev/klog could be made accessible via an ACL or a separate group.
^^^^^^^^^^^^
Argh, /proc/kmsg, I mean.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 19 13:56:26 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 13:56:26 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68B@grfint2.intern.adiscon.com>

Hi Michael,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, November 19, 2008 1:53 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> 2008/11/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Hi all,
> >
> > So, let me use this reply as a warp-up of my thoughts. Basically, I
> > agree to HKS argument. I also agree to David Lang's argument that
> some
> > degree of design change would be necessary to get the requested
> features
> > done 100% correct.
> >
> > But, of course, it all depends on the effort required. And more
> security
> > is always welcome. So I took time out today to think about the
> options
> > and experiment a bit. I came to the conclusion that some improvement
> can
> > probably be made in short time. I experimentally implemented a new
> > $PrivDropToUser config directive, which basically issues a setuid()
> call
> > with the user in question. It is far from being bullet-proof and
will
> > never be totally save without architecture changes. But I think it
> still
> > can considerably improve security and when we can achieve this with
a
> > few hours of work, it's probably worth the effort (a bit more work
> would
> > improve its security, but I don't do this unless I get feedback it
> makes
> > sense at all).
> >
> > HOWEVER, there are also some things that simply don't work out. For
> > example, imklog reads the proc file system under Linux and, even
> though
> > it is opened as root, these reads fail after dropping to another
> user.
> > There may be other issues, I have not further investigated that.
> 
> /dev/klog could be made accessible via an ACL or a separate group.
> (e.g. the init script of the distro could chgrp or setfacl /dev/klog).
> Distros will also have to make sure, that the rsyslog daemon has write
> access to /var/log (or wherever they put their log files by default,
> but that is easily solvable too).
> 
> More tricky will be the already mentioned opening of privileged ports
I
> think.

The experimental code handles that. It drops privileges after processing
the config file. There is a window of insecurity between when the
listeners start and the privileges are dropped, and that can not be
solved without an architecture change. But in essence, the "port open"
problem is solved (of course, HUP will always be of non-restart type). I
think security is improved even with this Window of insecurity, so if
the rest works OK, I would consider the new code a valuable addition.

Rainer 


From mbiebl at gmail.com  Wed Nov 19 14:07:34 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Nov 2008 14:07:34 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
Message-ID: <aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>

2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
>
>>
>> /dev/klog could be made accessible via an ACL or a separate group.
> ^^^^^^^^^^^^
> Argh, /proc/kmsg, I mean.

Looks like one can't set an ACL on a procfs unfortunately.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 19 14:10:24 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Nov 2008 14:10:24 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><aa76cdac0811190455p59b77e1bqafd788fe5e4de72a@mail.gmail.com>
	<aa76cdac0811190507r7f426f97mffedc8d1411e2fc2@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F68D@grfint2.intern.adiscon.com>

An out-of-process implementation for the plugins would solve all such
issues. But that takes time... A work-around may be to run two rsyslogd
processes concurrently. Definitely not a mainstream or default config,
but could be done. Also, I wonder if the kernel log works on BSD (not
tested), in which case the experimental code would work there without
any bad side effects (better phrased: with side-effects yet to be seen,
as I said I did so far not invest any serious time in testing).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, November 19, 2008 2:08 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> > 2008/11/19 Michael Biebl <mbiebl at gmail.com>:
> >
> >>
> >> /dev/klog could be made accessible via an ACL or a separate group.
> > ^^^^^^^^^^^^
> > Argh, /proc/kmsg, I mean.
> 
> Looks like one can't set an ACL on a procfs unfortunately.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Thu Nov 20 16:14:50 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Thu, 20 Nov 2008 10:14:50 -0500
Subject: [rsyslog] documentation: <object>blah, what is object?
Message-ID: <49257EEA.2000706@canonical.com>

I've been looking over the rsyslog documentation and I see a lot of
directives that begin with the nomenclature "<object>".  I see examples
of this tag being assigned to "Action" and "MainMsg".

Is there an explanation of what <object> is all about?

Thanks,

Peter


From rgerhards at hq.adiscon.com  Thu Nov 20 17:10:04 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:10:04 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <49257EEA.2000706@canonical.com>
References: <49257EEA.2000706@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>

I guess you mean the queue documentation. It looks like I have forgotten
that piece of information ;) <object> is "Action" for action queues and
"MainMsg" for the main message queue. Will add that soon, thanks for
pointing out.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Thursday, November 20, 2008 4:15 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] documentation: <object>blah, what is object?
> 
> I've been looking over the rsyslog documentation and I see a lot of
> directives that begin with the nomenclature "<object>".  I see
examples
> of this tag being assigned to "Action" and "MainMsg".
> 
> Is there an explanation of what <object> is all about?
> 
> Thanks,
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Nov 20 17:11:39 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:11:39 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>

Lol, actually it is described at the *bottom* of the document. I will
add a refrecence to that at the front ;)

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, November 20, 2008 5:10 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> 
> I guess you mean the queue documentation. It looks like I have
> forgotten
> that piece of information ;) <object> is "Action" for action queues
and
> "MainMsg" for the main message queue. Will add that soon, thanks for
> pointing out.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> > Sent: Thursday, November 20, 2008 4:15 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] documentation: <object>blah, what is object?
> >
> > I've been looking over the rsyslog documentation and I see a lot of
> > directives that begin with the nomenclature "<object>".  I see
> examples
> > of this tag being assigned to "Action" and "MainMsg".
> >
> > Is there an explanation of what <object> is all about?
> >
> > Thanks,
> >
> > Peter
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Nov 20 17:16:49 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Nov 2008 17:16:49 +0100
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com><577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>

I "fixed" this. The bottom part could simply be moved up, I think it now
makes more sense. You may want to have a look at the online version:

http://www.rsyslog.com/doc-queues.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, November 20, 2008 5:12 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> 
> Lol, actually it is described at the *bottom* of the document. I will
> add a refrecence to that at the front ;)
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, November 20, 2008 5:10 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] documentation: <object>blah, what is object?
> >
> > I guess you mean the queue documentation. It looks like I have
> > forgotten
> > that piece of information ;) <object> is "Action" for action queues
> and
> > "MainMsg" for the main message queue. Will add that soon, thanks for
> > pointing out.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> > > Sent: Thursday, November 20, 2008 4:15 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] documentation: <object>blah, what is object?
> > >
> > > I've been looking over the rsyslog documentation and I see a lot
of
> > > directives that begin with the nomenclature "<object>".  I see
> > examples
> > > of this tag being assigned to "Action" and "MainMsg".
> > >
> > > Is there an explanation of what <object> is all about?
> > >
> > > Thanks,
> > >
> > > Peter
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Thu Nov 20 17:32:40 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Thu, 20 Nov 2008 11:32:40 -0500
Subject: [rsyslog] documentation: <object>blah, what is object?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>
References: <49257EEA.2000706@canonical.com><577465F99B41C842AAFBE9ED71E70ABA44F6B7@grfint2.intern.adiscon.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6B8@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6B9@grfint2.intern.adiscon.com>
Message-ID: <49259128.2040506@canonical.com>

Thanks for your prompt response.

Peter

Rainer Gerhards wrote:
> I "fixed" this. The bottom part could simply be moved up, I think it now
> makes more sense. You may want to have a look at the online version:
> 
> http://www.rsyslog.com/doc-queues.html
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Thursday, November 20, 2008 5:12 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
>>
>> Lol, actually it is described at the *bottom* of the document. I will
>> add a refrecence to that at the front ;)
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>> Sent: Thursday, November 20, 2008 5:10 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] documentation: <object>blah, what is object?
>>>
>>> I guess you mean the queue documentation. It looks like I have
>>> forgotten
>>> that piece of information ;) <object> is "Action" for action queues
>> and
>>> "MainMsg" for the main message queue. Will add that soon, thanks for
>>> pointing out.
>>>
>>> Rainer
>>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
>>>> Sent: Thursday, November 20, 2008 4:15 PM
>>>> To: rsyslog at lists.adiscon.com
>>>> Subject: [rsyslog] documentation: <object>blah, what is object?
>>>>
>>>> I've been looking over the rsyslog documentation and I see a lot
> of
>>>> directives that begin with the nomenclature "<object>".  I see
>>> examples
>>>> of this tag being assigned to "Action" and "MainMsg".
>>>>
>>>> Is there an explanation of what <object> is all about?
>>>>
>>>> Thanks,
>>>>
>>>> Peter
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From janfrode at tanso.net  Fri Nov 21 13:11:02 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 13:11:02 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
Message-ID: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>

For my usage, I need two modes of operation for syslog daemons. 

    1 - local syslog. Requires privileges to on local devices (/dev/log,
        /dev/klogd or similar), write to local log-files, and send to
        remote log server.

    2 - central log server. Only listen on the needed network ports
        (514/udp/tcp), and write to local log files (possibly also send
        to other remote log servers).

For #1 I think it's OK not being able to chroot, keep more privileges,
etc.,  as the attacks against it will mostly be from local processes.

#2 needs to be *very* openly available on the network ports, since all 
my servers needs to send logs to it. #2 will also be holding a lot more
sensitive data than #1, so I think this server needs to be protected as
much as possible. If chroot'ing, or dropping privileges prevents it from
reading from local /proc og /dev, I think that wouldn't matter much. One
could always run two instances on these few central servers, i.e. #1
sending to #2.


  -jf



From rgerhards at hq.adiscon.com  Fri Nov 21 14:50:28 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 14:50:28 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>

So it looks like the new idea, though not perfect, seems to provide some
value, at least under some circumstances. It also looks trivially to
implement. Most effort is probably to tell people precisely why it is
not a fully security guard. I'll see if I get this fully implemented
next week if nobody objects. I guess it's not more than a day's worth.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Friday, November 21, 2008 1:11 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> For my usage, I need two modes of operation for syslog daemons.
> 
>     1 - local syslog. Requires privileges to on local devices
> (/dev/log,
>         /dev/klogd or similar), write to local log-files, and send to
>         remote log server.
> 
>     2 - central log server. Only listen on the needed network ports
>         (514/udp/tcp), and write to local log files (possibly also
send
>         to other remote log servers).
> 
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
> 
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot
more
> sensitive data than #1, so I think this server needs to be protected
as
> much as possible. If chroot'ing, or dropping privileges prevents it
> from
> reading from local /proc og /dev, I think that wouldn't matter much.
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.
> 
> 
>   -jf
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Fri Nov 21 15:31:24 2008
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Nov 2008 06:31:24 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>

On Fri, 21 Nov 2008, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
>    1 - local syslog. Requires privileges to on local devices (/dev/log,
>        /dev/klogd or similar), write to local log-files, and send to
>        remote log server.
>
>    2 - central log server. Only listen on the needed network ports
>        (514/udp/tcp), and write to local log files (possibly also send
>        to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot more
> sensitive data than #1, so I think this server needs to be protected as
> much as possible.

just pointing out that on this central server the data will be in the same 
place as the daemon listening to the network, and accessable to that 
daemon, so chrooting, dropping privilages, etc won't protect the data that 
gets logged (it will allow you to protect other data that lives on the 
server, just not log data)

well, you could do something where rsyslog writes the file in the chroot, 
and when it rolls the file something outside the chroot picks it up and 
moves it elsewhere, but that's starting to get a little extreme.

> If chroot'ing, or dropping privileges prevents it from
> reading from local /proc og /dev, I think that wouldn't matter much. One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

correct.

David Lang


From david at lang.hm  Fri Nov 21 15:31:49 2008
From: david at lang.hm (david at lang.hm)
Date: Fri, 21 Nov 2008 06:31:49 -0800 (PST)
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6D3@grfint2.intern.adiscon.com>
Message-ID: <alpine.DEB.1.10.0811210631270.28119@asgard.lang.hm>

On Fri, 21 Nov 2008, Rainer Gerhards wrote:

> So it looks like the new idea, though not perfect, seems to provide some
> value, at least under some circumstances. It also looks trivially to
> implement. Most effort is probably to tell people precisely why it is
> not a fully security guard. I'll see if I get this fully implemented
> next week if nobody objects. I guess it's not more than a day's worth.

I agree that even the limited version has benifits.

David Lang

> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
>> Sent: Friday, November 21, 2008 1:11 PM
>> To: rsyslog at lists.adiscon.com
>> Subject: Re: [rsyslog] rsyslogd security questions/comments
>>
>> For my usage, I need two modes of operation for syslog daemons.
>>
>>     1 - local syslog. Requires privileges to on local devices
>> (/dev/log,
>>         /dev/klogd or similar), write to local log-files, and send to
>>         remote log server.
>>
>>     2 - central log server. Only listen on the needed network ports
>>         (514/udp/tcp), and write to local log files (possibly also
> send
>>         to other remote log servers).
>>
>> For #1 I think it's OK not being able to chroot, keep more privileges,
>> etc.,  as the attacks against it will mostly be from local processes.
>>
>> #2 needs to be *very* openly available on the network ports, since all
>> my servers needs to send logs to it. #2 will also be holding a lot
> more
>> sensitive data than #1, so I think this server needs to be protected
> as
>> much as possible. If chroot'ing, or dropping privileges prevents it
>> from
>> reading from local /proc og /dev, I think that wouldn't matter much.
>> One
>> could always run two instances on these few central servers, i.e. #1
>> sending to #2.
>>
>>
>>   -jf
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From peter.matulis at canonical.com  Fri Nov 21 15:33:07 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 21 Nov 2008 09:33:07 -0500
Subject: [rsyslog] available compare operations in properties-base filters
Message-ID: <4926C6A3.8030706@canonical.com>

For properties-based filtering, the documentation states that currently
there is only a single compare operation ('contains') available.  It
then goes on to describe additional ones.  So what operations are
implemented at this point?

Peter



From rgerhards at hq.adiscon.com  Fri Nov 21 15:38:16 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 15:38:16 +0100
Subject: [rsyslog] available compare operations in properties-base
	filters
In-Reply-To: <4926C6A3.8030706@canonical.com>
References: <4926C6A3.8030706@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D6@grfint2.intern.adiscon.com>

Peter,

all modes in the table are implemented. The "single compare available"
sentence is wrong, I think I simply forgot to update it as I added
modes. Will do that now :)

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 21, 2008 3:33 PM
> To: rsyslog-users
> Subject: [rsyslog] available compare operations in properties-base
> filters
> 
> For properties-based filtering, the documentation states that
currently
> there is only a single compare operation ('contains') available.  It
> then goes on to describe additional ones.  So what operations are
> implemented at this point?
> 
> Peter
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From aoz.syn at gmail.com  Fri Nov 21 16:26:14 2008
From: aoz.syn at gmail.com (RB)
Date: Fri, 21 Nov 2008 08:26:14 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
Message-ID: <4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>

On Fri, Nov 21, 2008 at 07:31,  <david at lang.hm> wrote:
> just pointing out that on this central server the data will be in the same
> place as the daemon listening to the network, and accessable to that
> daemon, so chrooting, dropping privilages, etc won't protect the data that
> gets logged (it will allow you to protect other data that lives on the
> server, just not log data)

As is the same with all chrooted processes - the chroot model prevents
untrusted processes from messing with each other's candy, not their
own.  If anyone that runs a chroot thinks otherwise, they're sorely
mistaken.  It's why web servers are often chrooted with no write
access: the only writing they can do is back through specific ports
(database, syslog, memcache, etc.)

> well, you could do something where rsyslog writes the file in the chroot,
> and when it rolls the file something outside the chroot picks it up and
> moves it elsewhere, but that's starting to get a little extreme.

I absolutely agree that approach is foolish - better to give the
process no write permissions at all and have it either log to a
database (INSERT-only user) or run a two-layered approach wherein the
chrooted log server passes the [scrubbed] logs it receives back over a
pipe or to a secondary syslog service.  Crash/subvert the front layer,
you have little place to go.

RB


From janfrode at tanso.net  Fri Nov 21 18:34:56 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 18:34:56 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
Message-ID: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, RB <aoz.syn at gmail.com> wrote:
> On Fri, Nov 21, 2008 at 07:31,  <david at lang.hm> wrote:
>> just pointing out that on this central server the data will be in the same
>> place as the daemon listening to the network, and accessable to that
>> daemon, so chrooting, dropping privilages, etc won't protect the data that
>> gets logged (it will allow you to protect other data that lives on the
>> server, just not log data)
>
> As is the same with all chrooted processes - the chroot model prevents
> untrusted processes from messing with each other's candy, not their
> own.  If anyone that runs a chroot thinks otherwise, they're sorely
> mistaken.

I think the chroot() limits the possibility of:

    - executing anything outside the chroot (f.ex. /bin/sh)
    - write a new file, and execute it (noexec on the chroot mountpoint)
    - overwrite something outside the chroot. I'm an itsy bit worried that 
      something like this:
    
        $template PerAppLogs,"/var/log/apps/%programname%.log"
        *.* -?PerAppLogs

      could be tricked to write to unexpected places if you mess with
      %programname%, or other similar variables that the sender has complete
      controle over.

so I would argue that my sensitive syslogged data is more secure in a
chroot environment where there are no other executables, than on a non-
chroot environment.



  -jf

-- 
To know recursion, you must first know recursion.



From rgerhards at hq.adiscon.com  Fri Nov 21 18:43:37 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 21 Nov 2008 18:43:37 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm><85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com><alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com><alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm><4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6D7@grfint2.intern.adiscon.com>

Just a quick note:

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Friday, November 21, 2008 6:35 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> I'm an itsy bit worried
> that
>       something like this:
> 
>         $template PerAppLogs,"/var/log/apps/%programname%.log"
>         *.* -?PerAppLogs
> 
>       could be tricked to write to unexpected places if you mess with
>       %programname%, or other similar variables that the sender has
> complete
>       controle over.

Definitely. Though not a complete cure, the secpath-* property replacer
options prevent at least slashes inside programname. I know this is not
100% bullet proof, but it should be applied. Otherwise, you don't need a
malicious user, a simple program error is sufficient to screw up things.

So the template is suggested to be

           $template
PerAppLogs,"/var/log/apps/%programname:::secpath-replace%.log"

>From the doc:
---
secpath-replace 	Replace slashes inside the field by an
underscore. (e.g. "a/b" becomes "a_b"). Useful for secure pathname
generation (with dynafiles).
---

Full source at http://www.rsyslog.com/doc-property_replacer.html

I just thought I mention it.

Rainer


From janfrode at tanso.net  Fri Nov 21 19:23:54 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Fri, 21 Nov 2008 19:23:54 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6D7@grfint2.intern.adiscon.com>
Message-ID: <slrngidv5q.jbu.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
>
> Definitely. Though not a complete cure, the secpath-* property replacer
> options prevent at least slashes inside programname. I know this is not
> 100% bullet proof, but it should be applied. Otherwise, you don't need a
> malicious user, a simple program error is sufficient to screw up things.


Oh, I didn't know about this one. Thanks!


  -jf



From aoz.syn at gmail.com  Fri Nov 21 20:08:51 2008
From: aoz.syn at gmail.com (RB)
Date: Fri, 21 Nov 2008 12:08:51 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>

On Fri, Nov 21, 2008 at 10:34, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> I think the chroot() limits the possibility of:
>
>    - executing anything outside the chroot (f.ex. /bin/sh)
>    - write a new file, and execute it (noexec on the chroot mountpoint)
>    - overwrite something outside the chroot. I'm an itsy bit worried that
>      something like this:

You are absolutely correct - an unbroken chroot() provides a security
boundary that prevents the process[es] therein from accessing
filesystem resources outside that boundary.  Permissions on the
directory structure at & below that point can provide additional
restrictions (like noexec).

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment where there are no other executables, than on a non-
> chroot environment.

Argue what you like, but your basis is flawed.  You seem to forget
that most exploits bring their own executable [shell]code with them
and often operate purely in-memory.  Therefore they don't strictly
need external executables; the lack thereof is more of a nuisance than
a roadblock.

System designers _must_ assume that any successful remote attacker may
access anything the application can, with the same permissions and
executing arbitrary code - chroot or not.  If the data you are logging
is sufficiently sensitive that no client should be able to read the
data and any possibility of their doing so would be catastrophic, you
are going to have to either establish a one-way transport out of that
syslog chroot() or limit your scope to the point that remote attackers
are no longer a possibility.


RB


From janfrode at tanso.net  Sat Nov 22 12:17:24 2008
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Sat, 22 Nov 2008 12:17:24 +0100
Subject: [rsyslog] rsyslogd security questions/comments
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>
Message-ID: <slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>

On 2008-11-21, RB <aoz.syn at gmail.com> wrote:
>
>> so I would argue that my sensitive syslogged data is more secure in a
>> chroot environment where there are no other executables, than on a non-
>> chroot environment.
>
> Argue what you like, but your basis is flawed.  You seem to forget
> that most exploits bring their own executable [shell]code with them
> and often operate purely in-memory.  Therefore they don't strictly
> need external executables; the lack thereof is more of a nuisance than
> a roadblock.

Did you see the comment from Rainer about the secpath-replace property? 
I think that proves my basis is not flawed. The chroot here protects
against code flaws, or configuration flaws that could otherwise give
attacker the possibility of overwriting system files.

Also you pointing at "most exploits bring their own executable .. operate
purely in-memory" argument is flawed when I'm arguing for chroot being 
*more* secure. Not 100% secure against all flaws, but *more* secure than
a non-chrooted process.

.. but dropping privileges is higher up on my wishlist, than chroot()..


  -jf

-- 
http://xkcd.com/386/ ;-)



From rsyslog at clark-communications.com  Sat Nov 22 20:18:22 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Sat, 22 Nov 2008 11:18:22 -0800
Subject: [rsyslog] rsyslog support for other chrooted apps
Message-ID: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>


NB, in this email, I am *not* referring to any potential chroot  
support within rsyslog itself!

In an OS like OpenBSD, some/many daemons run in a chroot jail, two  
common examples include postfix and named.

The stock syslogd on OpenBSD supports this by a command line option "- 
a" that adds new /dev/log devices.

Thus, on one of my machines, syslogd is run like this:

	syslogd -a /var/spool/postfix/dev/log -a /var/named/dev/log -a /var/ 
empty/dev/log

Which adds the three additional /dev/logs to the standard /dev/log,  
and syslogd reads them all.

In further testing of the OpenBSD port of rsyslog that I posted  
earlier this week,

	http://lists.adiscon.net/pipermail/rsyslog/2008-November/001395.html

I finally realized that it is going to difficult to replace the stock  
OpenBSD syslogd with rsyslogd unless it supports something like this.

Questions,

Does rsyslog support anything like this already?

If not, how difficult would it be to add this, and do people here  
think this is valuable addition, or not?

Don



From rsyslog at clark-communications.com  Sat Nov 22 21:34:07 2008
From: rsyslog at clark-communications.com (Don Jackson)
Date: Sat, 22 Nov 2008 12:34:07 -0800
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>


On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:

> For my usage, I need two modes of operation for syslog daemons.
>
>    1 - local syslog. Requires privileges to on local devices (/dev/ 
> log,
>        /dev/klogd or similar), write to local log-files, and send to
>        remote log server.
>
>    2 - central log server. Only listen on the needed network ports
>        (514/udp/tcp), and write to local log files (possibly also send
>        to other remote log servers).
>
> For #1 I think it's OK not being able to chroot, keep more privileges,
> etc.,  as the attacks against it will mostly be from local processes.
>
> #2 needs to be *very* openly available on the network ports, since all
> my servers needs to send logs to it. #2 will also be holding a lot  
> more
> sensitive data than #1, so I think this server needs to be protected  
> as
> much as possible. If chroot'ing, or dropping privileges prevents it  
> from
> reading from local /proc og /dev, I think that wouldn't matter much.  
> One
> could always run two instances on these few central servers, i.e. #1
> sending to #2.

Yes, your two modes as defined above are exactly how I run syslog, and  
I think the distinction between the two use cases
local-syslog and central-log-server is very useful and important in  
this discussion.

And I agree with your prioritization of security between the two  
modes, the local-syslog does not need to open a network port
to listen for log messages, and so it is only at risk from other  
processes on the same box.

The central-log-server does need to open and read a network port, and  
needs to be open to lots of devices on the network, and thus it is at  
the most risk.
That is the version that most needs privilege separation and chroot.   
And I agree that the server that runs the central-log-server could  
also run an instance of the local-syslog, the local-syslog would  
handle /dev/log on that machine, and forward msgs to be centrally  
logged onto the central-log-server running on the same machine.

 From another sub-thread:

> On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:

> Yes, the traditional HUP behavior is simply incompatible with dropping
> privileges. But I assume that someone who configures rsyslogd in  
> such a
> way knows what he does and thus changes config-reload scripts away  
> from
> HUP to a real reload.

I am not sure about the details of the implementation, but named on  
OpenBSD supports privilege separation, and it provides
some of your HUP functionality, although possibly in different ways:

Here is how named looks from ps:

	root     	23404  0.0  0.0  2148   880 ??  Is     2:19PM    0:00.00  
named: [priv] (named)
	named    21080  0.0  0.2  8744  9452 ??  I      2:19PM    0:00.24  
named -4

Here is what the named man page says about HUP:

   SIGNALS
        In routine operation, signals should not be used to con-
        trol the nameserver; rndc should be used instead.

        SIGHUP Force a reload of the server.

        SIGINT, SIGTERM
               Shut down the server.

        The result of sending any other signals to the server is
        undefined.

And, as stated above, rndc is the preferred way of controlling named  
(asking it to reload zone files, etc.)

The config file for rsyslogd could/should reside within the chroot, so  
it should always be accessible.


On yet another sub-thread:

On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
>  I also agree to David Lang's argument that some
> degree of design change would be necessary to get the requested  
> features
> done 100% correct.

It is generally difficult to add security ex post facto, and the more  
code that is written prior to the implementation of those design  
changes will tend to increase the difficultly/cost of that  
implementation.

On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> FWIW, while I believe this discussion is relevant and the security
> changes proposed are important, I don't see this as a high priority
> feature. The scripting language/syntax, greater performance, continued
> stability enhancements, and true RELP support are all far more
> important. As far as security goes, I would much rather see two-way
> host authentication than a chroot/unprivilieged framework implemented.

I don't know what the author's goals for rsyslog are, so it is  
difficult to prioritize.

What *I* would like rsyslog to be is the clear choice to replace  
syslog on all the machines in my network, on all the OSes I care about  
(OpenBSD, Solaris, FreeBSD, etc),
basically "total world domination of the syslog space" :-)

I would ask Ranier and others here to reflect on the need/importance  
of security vis-a-vis their ultimate goals for rsyslog, and if  
security is going to be important (I think it is crucial, but you need  
to come to that conclusion yourself), and the cost of implementing  
security design changes is going to increase over time, to consider  
making those design changes sooner rather than later.

In closing, I am learning a lot and enjoying the thoughtful and  
patient responses to my original email on this topic, kudos to the  
rsyslog community, it is a welcome change from some other mailing  
lists :-)

Best regards,

Don















From aoz.syn at gmail.com  Sat Nov 22 23:49:08 2008
From: aoz.syn at gmail.com (RB)
Date: Sat, 22 Nov 2008 15:49:08 -0700
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm>
	<4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com>
	<slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com>
	<slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
Message-ID: <4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>

On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust <janfrode at tanso.net> wrote:
> Did you see the comment from Rainer about the secpath-replace property?
> I think that proves my basis is not flawed. The chroot here protects
> against code flaws, or configuration flaws that could otherwise give
> attacker the possibility of overwriting system files.

I most certainly did, but all the secpath-replace property protects
against is an administrator foolishly trusting unvalidated data from a
remote host to generate file paths.  Chroot does not protect against
code flaws, full stop.  That is the continuing flaw in your argument.
Chroot only prevents code flaws from affecting anything _outside_ of
the chroot environment.

> Also you pointing at "most exploits bring their own executable .. operate
> purely in-memory" argument is flawed when I'm arguing for chroot being
> *more* secure. Not 100% secure against all flaws, but *more* secure than
> a non-chrooted process.

You mistake what I am arguing - there is no question that chroot()
helps protect the host system's files.  What it does _not_ protect
against is abuse of anything within the chroot().  Your statement was:

> so I would argue that my sensitive syslogged data is more secure in a
> chroot environment

If your log data is being stored inside that chroot, you are utterly
wrong.  Please actually understand what chroot() does before you
continue making wild, unfounded statements.


From rgerhards at hq.adiscon.com  Mon Nov 24 11:08:16 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:08:16 +0100
Subject: [rsyslog] rsyslog support for other chrooted apps
In-Reply-To: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>
References: <2861B59B-273F-4FBB-8C02-A48003523B52@clark-communications.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DA@grfint2.intern.adiscon.com>

Hi Don,

I guess you were lost in the docs (it needs to be restrucuted some
day... I begun with that, but that didn't yet result in improved ease of
use, to phrase it politely ;)).

You use case is described in the relevant module's doc:

http://www.rsyslog.com/doc-imuxsock.html

Please let me know if that solves the issue.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Don Jackson
> Sent: Saturday, November 22, 2008 8:18 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog support for other chrooted apps
> 
> 
> NB, in this email, I am *not* referring to any potential chroot
> support within rsyslog itself!
> 
> In an OS like OpenBSD, some/many daemons run in a chroot jail, two
> common examples include postfix and named.
> 
> The stock syslogd on OpenBSD supports this by a command line option "-
> a" that adds new /dev/log devices.
> 
> Thus, on one of my machines, syslogd is run like this:
> 
> 	syslogd -a /var/spool/postfix/dev/log -a /var/named/dev/log -a
> /var/
> empty/dev/log
> 
> Which adds the three additional /dev/logs to the standard /dev/log,
> and syslogd reads them all.
> 
> In further testing of the OpenBSD port of rsyslog that I posted
> earlier this week,
> 
> 	http://lists.adiscon.net/pipermail/rsyslog/2008-
> November/001395.html
> 
> I finally realized that it is going to difficult to replace the stock
> OpenBSD syslogd with rsyslogd unless it supports something like this.
> 
> Questions,
> 
> Does rsyslog support anything like this already?
> 
> If not, how difficult would it be to add this, and do people here
> think this is valuable addition, or not?
> 
> Don
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dhaivat.desai at yahoo.com  Mon Nov 24 11:23:57 2008
From: dhaivat.desai at yahoo.com (Dhaivat Desai)
Date: Mon, 24 Nov 2008 15:53:57 +0530 (IST)
Subject: [rsyslog]  defining own input module in rsyslog
Message-ID: <10144.52888.qm@web94909.mail.in2.yahoo.com>

Hi,

I am using imuxsock module of rsyslog.
I want to make a new module with some changes in the existing one.
in imuxsock.c the macros and callbacks are very confusing.
are there any steps i need to follow for developing my own module working with RSyslog?

i want to change imuxsock module work with stream protocol. as current implementation doesnt support binary message logging, i want to add binary message logging also in the same.

moreover, in the rsyslog.config i want to add one more directive $BINARYFILEPATH which specifies the pathname for binary file which all the binary messages will be logged into.

what changes will i require to get my code working?

Thanks,
Dhaivat....









      Add more friends to your messenger and enjoy! Go to http://messenger.yahoo.com/invite/

From rgerhards at hq.adiscon.com  Mon Nov 24 11:41:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:41:25 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com><ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com><1227098761.19905.49.camel@rgf9dev.intern.adiscon.com><aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com><slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com><alpine.DEB.1.10.0811210628170.28119@asgard.lang.hm><4255c2570811210726h91f937et9fbfe53c0f4dce81@mail.gmail.com><slrngidsa0.isq.janfrode@lc4eb5760521341.ibm.com><4255c2570811211108n4ab1d082q9f84b26b12089d17@mail.gmail.com><slrngifqi4.aag.janfrode@lc4eb5760521341.ibm.com>
	<4255c2570811221449j5198b8afif331019ba60c443@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DC@grfint2.intern.adiscon.com>

Hi all,

please let me jump into the middle of that discussion.

Maybe it helps if someone else comments. From my (external) point of
view, it looks like your arguments hit more or less the same targets,
but it also looks like you seem to apply different co-notations to
things. So let me try to formalize things a bit.

Let me think of security as a probability of security breach. S_curr is
the security of the reference system without a root jail. S_total is the
security of a hypothetical system that is "totally secure" (knowing well
that no such system exists). In other words, the probability S_total
equals 0.

I think the common ground is that a root jail does not worsen security.
Note that I do not say it improves security, only that it does not
reduce a system's security. S_jail is the security of a system that is
otherwise identical to the reference system, but with a root jail. Than
S_jail <= s_curr, because we assume that the security of the system is
not reduced.

I think it is also common ground that the probability of a security
breach is reduced if the number of attack vectors is reduced, without
any new attack vectors being added. [There is one generic "attack
vector", the "thought of being secure and thus becoming careless" which
always increases as risk is reduced - I will not include that vector in
my thoughts]

We seem to be in agreement that a root jail is able to prevent some
attacks from being successful. I can't enumerate them and it is probably
useless to try to do so (because attackers invent new attacks each day),
but there exist some attacks which can be prevented by a root jail. I do
not try to weigh them by their importance.

For obvious reasons, there exist other attacks which are not affected by
the root jail. Some of them have been mentions, like the class of
in-memory based attacks, code injection and many more. 

I tend to think that the set of attack vectors that can be prevented by
a root jail is much smaller than the set of those which can not. I also
tend to think that the later class contains the more serious attack
vectors.

But even then, a root jail seems to remove a subset of the attack
vectors that otherwise exists and so it reduces the probably of security
breach. So it benefits security. We can only argue that it does not
benefit security if we can show that in all cases we can think of (and
those we can not), security is not improved. However, some cases have
been show, where it improves, so it can not be that security is not
improved in all cases. As such, a root jail improves security, or more
precisely the probability of a security breach is

0 < S_jail < S_curr

We can identify the benefit we gain is the difference between the
reference system's probability of security breach and the system with
the jail. Be S_impr this improvement, than

S_impr = S_curr - S_jail

Now the root jail is just one potential security measure. We could now
try to calculate S_impr for all kinds of security measures, for example
a privilege drop. I find it hard to do the actual probability
calculations, but I would guess that S_impr_privdop > S_impr_jail.

Based on the improvements, one may finally decide what to implement
first (either at the code or admin level), all of this of course
weighted with the importance of the numbers.

In any case, I think I have show that both is correct:

- the root jail is a security improvement
- there exist numerous other improvements, many of them probably more
efficient than the jail

And if I got you right, I think your arguments meant exactly this ;)

For rsyslog as a project, this also means we need to weigh security
measures against functionality. Based on this discussion, I wonder if it
would be a useful undertaking to try document at least the security
options at hand and try to provide some helpful notes for those that are
not so deeply knowledgeable about these issues (including me, someone
who is always surprised by the numerous ways people find to circumvent
what one has considered to be secure ;)).

Feedback is appreciated.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of RB
> Sent: Saturday, November 22, 2008 11:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslogd security questions/comments
> 
> On Sat, Nov 22, 2008 at 04:17, Jan-Frode Myklebust
<janfrode at tanso.net>
> wrote:
> > Did you see the comment from Rainer about the secpath-replace
> property?
> > I think that proves my basis is not flawed. The chroot here protects
> > against code flaws, or configuration flaws that could otherwise give
> > attacker the possibility of overwriting system files.
> 
> I most certainly did, but all the secpath-replace property protects
> against is an administrator foolishly trusting unvalidated data from a
> remote host to generate file paths.  Chroot does not protect against
> code flaws, full stop.  That is the continuing flaw in your argument.
> Chroot only prevents code flaws from affecting anything _outside_ of
> the chroot environment.
> 
> > Also you pointing at "most exploits bring their own executable ..
> operate
> > purely in-memory" argument is flawed when I'm arguing for chroot
> being
> > *more* secure. Not 100% secure against all flaws, but *more* secure
> than
> > a non-chrooted process.
> 
> You mistake what I am arguing - there is no question that chroot()
> helps protect the host system's files.  What it does _not_ protect
> against is abuse of anything within the chroot().  Your statement was:
> 
> > so I would argue that my sensitive syslogged data is more secure in
a
> > chroot environment
> 
> If your log data is being stored inside that chroot, you are utterly
> wrong.  Please actually understand what chroot() does before you
> continue making wild, unfounded statements.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 11:46:30 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 11:46:30 +0100
Subject: [rsyslog] defining own input module in rsyslog
In-Reply-To: <10144.52888.qm@web94909.mail.in2.yahoo.com>
References: <10144.52888.qm@web94909.mail.in2.yahoo.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6DD@grfint2.intern.adiscon.com>

Hi,

I suggest that you have a look at the imtemplate modul, which was
specifically written to help understand the interface. Please come back
once you have reviewed it. I would also appreciate if you could do that
as a web-based forum thread, because that would make it easier for me to
refer other people with the same intention to that thread. The forum
link is

http://www.rsyslog.com/forum

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Dhaivat Desai
> Sent: Monday, November 24, 2008 11:24 AM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] defining own input module in rsyslog
> 
> Hi,
> 
> I am using imuxsock module of rsyslog.
> I want to make a new module with some changes in the existing one.
> in imuxsock.c the macros and callbacks are very confusing.
> are there any steps i need to follow for developing my own module
> working with RSyslog?
> 
> i want to change imuxsock module work with stream protocol. as current
> implementation doesnt support binary message logging, i want to add
> binary message logging also in the same.
> 
> moreover, in the rsyslog.config i want to add one more directive
> $BINARYFILEPATH which specifies the pathname for binary file which all
> the binary messages will be logged into.
> 
> what changes will i require to get my code working?
> 
> Thanks,
> Dhaivat....
> 
> 
> 
> 
> 
> 
> 
> 
> 
>       Add more friends to your messenger and enjoy! Go to
> http://messenger.yahoo.com/invite/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 12:55:00 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 12:55:00 +0100
Subject: [rsyslog] rsyslogd security questions/comments
In-Reply-To: <2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>
References: <D8C9AF77-E2BF-4532-90B2-34C5898D3068@clark-communications.com>
	<alpine.DEB.1.10.0811181306180.32032@asgard.lang.hm>
	<85EB8DD7-0A94-431E-992D-6C4DE4685A53@clark-communications.com>
	<alpine.DEB.1.10.0811181407170.32032@asgard.lang.hm>
	<ac9ded160811181429m234922dq97a1f6253b4fbeae@mail.gmail.com>
	<1227098761.19905.49.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811190453y5e2a324fyee9e8aab2ba8ba8b@mail.gmail.com>
	<slrngid9am.gee.janfrode@lc4eb5760521341.ibm.com>
	<2A00BB12-EC3F-4E4C-ACAA-49ACFC1ADDAD@clark-communications.com>
Message-ID: <1227527700.5282.21.camel@rgf9dev.intern.adiscon.com>

Hi Don,

thanks for your careful thoughts. I would also appreciate if you could
try out the experimental branch, but on the other hand I think I will be
able to create a regular devel branch based on some of the discussion
this week.

On Sat, 2008-11-22 at 12:34 -0800, Don Jackson wrote:
> On Nov 21, 2008, at 4:11 AM, Jan-Frode Myklebust wrote:
> 
> > For my usage, I need two modes of operation for syslog daemons.
> >
> >    1 - local syslog. Requires privileges to on local devices (/dev/ 
> > log,
> >        /dev/klogd or similar), write to local log-files, and send to
> >        remote log server.
> >
> >    2 - central log server. Only listen on the needed network ports
> >        (514/udp/tcp), and write to local log files (possibly also send
> >        to other remote log servers).
> >
> > For #1 I think it's OK not being able to chroot, keep more privileges,
> > etc.,  as the attacks against it will mostly be from local processes.
> >
> > #2 needs to be *very* openly available on the network ports, since all
> > my servers needs to send logs to it. #2 will also be holding a lot  
> > more
> > sensitive data than #1, so I think this server needs to be protected  
> > as
> > much as possible. If chroot'ing, or dropping privileges prevents it  
> > from
> > reading from local /proc og /dev, I think that wouldn't matter much.  
> > One
> > could always run two instances on these few central servers, i.e. #1
> > sending to #2.
> 
> Yes, your two modes as defined above are exactly how I run syslog, and  
> I think the distinction between the two use cases
> local-syslog and central-log-server is very useful and important in  
> this discussion.

Even further, I think this could be different (but compatible) use cases
in a to-be-written rsyslog security doc.

> 
> And I agree with your prioritization of security between the two  
> modes, the local-syslog does not need to open a network port
> to listen for log messages, and so it is only at risk from other  
> processes on the same box.
> 
> The central-log-server does need to open and read a network port, and  
> needs to be open to lots of devices on the network, and thus it is at  
> the most risk.
> That is the version that most needs privilege separation and chroot.   
> And I agree that the server that runs the central-log-server could  
> also run an instance of the local-syslog, the local-syslog would  
> handle /dev/log on that machine, and forward msgs to be centrally  
> logged onto the central-log-server running on the same machine.
> 
>  From another sub-thread:
> 
> > On Nov 19, 2008, at 4:32 AM, Rainer Gerhards wrote:
> 
> > Yes, the traditional HUP behavior is simply incompatible with dropping
> > privileges. But I assume that someone who configures rsyslogd in  
> > such a
> > way knows what he does and thus changes config-reload scripts away  
> > from
> > HUP to a real reload.
> 
> I am not sure about the details of the implementation, but named on  
> OpenBSD supports privilege separation, and it provides
> some of your HUP functionality, although possibly in different ways:
[snip]

It would be too much info to go in full detail, but HUP is ugly and it
is a full restart. I'd like to phase out that mode, but will probably
need to keep it for "admin compatibility". Anyhow, I do not see any
issue if, in a secure system, HUP does not work as a full restart. (One
way to make it work would be to have a master running all the time and
the actual rsyslogd being a child where the parent handles HUP and
restarts the "real" rsyslogd - but I don't like to make it so compliated
"just" to satisfy old habits... [who likes this can always contribute
the code ;)]).

> On yet another sub-thread:
> 
> On Nov 19, 2008, at 4:46 AM, Rainer Gerhards wrote:
> >  I also agree to David Lang's argument that some
> > degree of design change would be necessary to get the requested  
> > features
> > done 100% correct.
> 
> It is generally difficult to add security ex post facto, and the more  
> code that is written prior to the implementation of those design  
> changes will tend to increase the difficultly/cost of that  
> implementation.

That's totally right, and rsyslog is a good example. First of all it is
important to remember that it was not designed from scratch. We forked
it off from sysklogd. And while almost no sysklogd code remains today,
we had a long chain of design decisions which were based on the need to
keep compatible with sysklogd. If you traverse that chain, you'll see
why some things are as they are. 

The security approach was not to be worse than sysklogd and better where
possible, but there always was a strong focus on deliverving something.
You may argue that it is foolish not doing the "right thing" right from
the beginning. My view is that we live in an imperfect (and risky) world
and it is important to get some things done. So I don't want to limit
myself in using a better solution because that better solution is not
better in some aspect than the previous solution, which I would be using
otherwise. 

In somewhat more formal words. If sysklogd's number if features is
F_sysklogd and it's probably of security breach is S_sysklog and the
exact same concepts for rsyslog are F_rsyslog and S_rsyslog, then I
don't see any reason not to use rsyslog over sysklogd if F_rsyslog >
F_sysklogd and R_rsyslog <= R_sysklogd. In even other words, if I would
not use rsyslog because it is not as secure as sysklogd, I would ditch
it without getting any benefit and I would also not get the benefits
from the new features it offers.

One may of course argue that rsyslog's addition feature inherently
increase the risk of using it, in which case S_rsyslog < S_sysklogd. The
only argument I have against this is that you can disable most of these
new features so that you can limit the potential risk exposure.

In any case, you are right that the ultimate goal should be to keep as
secure *as possible* and this is why I immediately followed up on your
questions. 

But I think it is still very important to not let security be a show
stopper in a case if it is not worsened. I think if I had started the
rsyslog project with a totally secure design, rsyslog would be obsolete
by now because anybody would already have moved over to syslog-ng and no
other solution would have a chance to rise.

I am not saying that syslog-ng is a bad project, nor am I saying it is
less secure than rsyslog (right now, it probably is more secure). But a
world with just syslog-ng (or one with just rsyslog!) is a monoculture a
monoculture has inherent and very large security risk.

I sincerely believe (bash me) that Windows is *not* bad software (or at
least not worse than Linux). It "just" is (well, "was" ;)) a monoculture
and that lead to much more interest in attacking it. The financial
benefit of attacking it was much greater than the benefit of attacking
Linux.

If you look from that angle (and I do), you'll find that a world with a
not-so-secure rsyslog AND syslog-ng is more secure than a world with a
totally-secure rsyslog (that nobody uses) and and always-present
syslog-ng (being a monoculture on the vast majority of systems). So less
security in a "subsystem" brought more security to the overall picture.

Still, I strongly favor an as-secure-as possible rsyslog, but I need to
abide to the constraints given (and with the financial crises and loss
of sponsorship these constraints for the rsyslog project unfortunately
have not become more permitting...).

> 
> On Nov 18, 2008, at 2:29 PM, (private) HKS wrote:
> > FWIW, while I believe this discussion is relevant and the security
> > changes proposed are important, I don't see this as a high priority
> > feature. The scripting language/syntax, greater performance, continued
> > stability enhancements, and true RELP support are all far more
> > important. As far as security goes, I would much rather see two-way
> > host authentication than a chroot/unprivilieged framework implemented.
> 
> I don't know what the author's goals for rsyslog are, so it is  
> difficult to prioritize.
> 
> What *I* would like rsyslog to be is the clear choice to replace  
> syslog on all the machines in my network, on all the OSes I care about  
> (OpenBSD, Solaris, FreeBSD, etc),
> basically "total world domination of the syslog space" :-)

as said above, "total world dominition" is sweet fruit, but it has a big
risk associated with it...

> 
> I would ask Ranier and others here to reflect on the need/importance  
> of security vis-a-vis their ultimate goals for rsyslog, and if  
> security is going to be important (I think it is crucial, but you need  
> to come to that conclusion yourself), and the cost of implementing  
> security design changes is going to increase over time, to consider  
> making those design changes sooner rather than later.

I hope I outlined these thoughts above. Please keep asking/suggesting.
My mind is always open to change and I like good arguments ;)

Rainer
> 
> In closing, I am learning a lot and enjoying the thoughtful and  
> patient responses to my original email on this topic, kudos to the  
> rsyslog community, it is a welcome change from some other mailing  
> lists :-)
> 
> Best regards,
> 
> Don
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From pete.philips at secerno.com  Mon Nov 24 13:36:35 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 12:36:35 +0000
Subject: [rsyslog] Logging lost TCP connections
Message-ID: <492A9FD3.2040700@secerno.com>

Hi.

I've googled this extensively but can can't find a solution. Please could anyone tell
me if there is an option to enable logging of lost TCP connections? I'd like the
event recorded locally if contact is lost with a remote TCP syslog server.

Many thanks.


Pete Philips.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 14:36:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 14:36:44 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492A9FD3.2040700@secerno.com>
References: <492A9FD3.2040700@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>

Hi Pete,

I think there is no such option, but I will check with the code.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 1:37 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Logging lost TCP connections
> 
> Hi.
> 
> I've googled this extensively but can can't find a solution. Please
> could anyone tell
> me if there is an option to enable logging of lost TCP connections?
I'd
> like the
> event recorded locally if contact is lost with a remote TCP syslog
> server.
> 
> Many thanks.
> 
> 
> Pete Philips.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 15:17:38 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:17:38 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>

Hi Pete,

I checked, but this information is only available in the debug log.
Having said that, it is probably not hard to add it. What exactly would
you like to see (e.g. complete closes or just aborted ones)?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, November 24, 2008 2:37 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Hi Pete,
> 
> I think there is no such option, but I will check with the code.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Pete Philips
> > Sent: Monday, November 24, 2008 1:37 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Logging lost TCP connections
> >
> > Hi.
> >
> > I've googled this extensively but can can't find a solution. Please
> > could anyone tell
> > me if there is an option to enable logging of lost TCP connections?
> I'd
> > like the
> > event recorded locally if contact is lost with a remote TCP syslog
> > server.
> >
> > Many thanks.
> >
> >
> > Pete Philips.
> > --
> > Pete Philips
> > Secerno Ltd
> > Email: pete.philips at secerno.com
> > PGP key: http://www.secerno.com/pgp/pete.gpg
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:27:02 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:27:02 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
Message-ID: <492AB9B6.4040805@secerno.com>

Rainer,

Thanks for your help on this.

Rainer Gerhards wrote:
> I checked, but this information is only available in the debug log.
> Having said that, it is probably not hard to add it. What exactly would
> you like to see (e.g. complete closes or just aborted ones)?

I'm not sure of the exact terminology but I'd like it to log a record locally everytime
it becomes not possible to send a TCP syslog message to a remote server. Perhaps if
this would be useful to others then it could be added to a future release?

Thanks.


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 15:29:45 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:29:45 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492AB9B6.4040805@secerno.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>
	<492AB9B6.4040805@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 3:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Rainer,
> 
> Thanks for your help on this.
> 
> Rainer Gerhards wrote:
> > I checked, but this information is only available in the debug log.
> > Having said that, it is probably not hard to add it. What exactly
> would
> > you like to see (e.g. complete closes or just aborted ones)?
> 
> I'm not sure of the exact terminology but I'd like it to log a record
> locally everytime
> it becomes not possible to send a TCP syslog message to a remote
> server. Perhaps if
> this would be useful to others then it could be added to a future
> release?

Ummm... I was on the wrong side, checking the receiver. So you would
like to log if the send does not succeed?

Which brings up the question: do you have reasons for accepting the
message loss? I am asking because you can instruct rsyslog to retry
sending the data when the remote server is ready again. For an example,
see here:

http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html

Rainer
> 
> Thanks.
> 
> 
> Pete.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:44:02 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:44:02 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
Message-ID: <492ABDB2.2040001@secerno.com>

Rainer Gerhards wrote:
> Ummm... I was on the wrong side, checking the receiver. So you would
> like to log if the send does not succeed?

Yes exactly.

> Which brings up the question: do you have reasons for accepting the
> message loss? I am asking because you can instruct rsyslog to retry
> sending the data when the remote server is ready again. For an example,
> see here:
> 
> http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html

I've seen that page and would like to use the disk buffering capability of
rsyslog to make logging failures unnecessary. Unfortunately the spec for the system
I am delivering requires logging of failed delivery attempts and not disk
buffering :-(


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Mon Nov 24 15:45:32 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:45:32 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <492ABDB2.2040001@secerno.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>
	<492ABDB2.2040001@secerno.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>

OK, I will look into that, but I can not promise how trivial this may be
;) I keep you posted.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Pete Philips
> Sent: Monday, November 24, 2008 3:44 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> Rainer Gerhards wrote:
> > Ummm... I was on the wrong side, checking the receiver. So you would
> > like to log if the send does not succeed?
> 
> Yes exactly.
> 
> > Which brings up the question: do you have reasons for accepting the
> > message loss? I am asking because you can instruct rsyslog to retry
> > sending the data when the remote server is ready again. For an
> example,
> > see here:
> >
> > http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html
> 
> I've seen that page and would like to use the disk buffering
capability
> of
> rsyslog to make logging failures unnecessary. Unfortunately the spec
> for the system
> I am delivering requires logging of failed delivery attempts and not
> disk
> buffering :-(
> 
> 
> Pete.
> --
> Pete Philips
> Secerno Ltd
> Email: pete.philips at secerno.com
> PGP key: http://www.secerno.com/pgp/pete.gpg
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Nov 24 15:49:57 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 24 Nov 2008 15:49:57 +0100
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com><492ABDB2.2040001@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F6EA@grfint2.intern.adiscon.com>

Well... I was too quick. The issue is that no matter what we do, we
cannot simply log failed *tcp sends*. The reason is that actions fail
some place inside the rule engine. At the point where it fails, it is
unaware if it is a TCP send, file store, sql store, etc, etc, ... Of
course the tcp output plugin can emit the failure message, but the tcp
output plugin does not know if this is an ultimate failure or not - this
depends on the rule engine configuration.

So if anything is set to report errors, you can either instrument the
plugin and live with potentially false positives OR you can instrument
the rule engine and generate messages in any instance (well, we could
create an action-specific failure notice, which probably can be
configured to what you exactly wanted to have - probably the
solution...).

In any case, feedback from your side is appreciated.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, November 24, 2008 3:46 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Logging lost TCP connections
> 
> OK, I will look into that, but I can not promise how trivial this may
> be
> ;) I keep you posted.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Pete Philips
> > Sent: Monday, November 24, 2008 3:44 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Logging lost TCP connections
> >
> > Rainer Gerhards wrote:
> > > Ummm... I was on the wrong side, checking the receiver. So you
> would
> > > like to log if the send does not succeed?
> >
> > Yes exactly.
> >
> > > Which brings up the question: do you have reasons for accepting
the
> > > message loss? I am asking because you can instruct rsyslog to
retry
> > > sending the data when the remote server is ready again. For an
> > example,
> > > see here:
> > >
> > > http://www.rsyslog.com/doc-rsyslog_reliable_forwarding.html
> >
> > I've seen that page and would like to use the disk buffering
> capability
> > of
> > rsyslog to make logging failures unnecessary. Unfortunately the spec
> > for the system
> > I am delivering requires logging of failed delivery attempts and not
> > disk
> > buffering :-(
> >
> >
> > Pete.
> > --
> > Pete Philips
> > Secerno Ltd
> > Email: pete.philips at secerno.com
> > PGP key: http://www.secerno.com/pgp/pete.gpg
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pete.philips at secerno.com  Mon Nov 24 15:51:14 2008
From: pete.philips at secerno.com (Pete Philips)
Date: Mon, 24 Nov 2008 14:51:14 +0000
Subject: [rsyslog] Logging lost TCP connections
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
References: <492A9FD3.2040700@secerno.com>	<577465F99B41C842AAFBE9ED71E70ABA44F6E5@grfint2.intern.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F6E6@grfint2.intern.adiscon.com>	<492AB9B6.4040805@secerno.com><577465F99B41C842AAFBE9ED71E70ABA44F6E8@grfint2.intern.adiscon.com>	<492ABDB2.2040001@secerno.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F6E9@grfint2.intern.adiscon.com>
Message-ID: <492ABF62.2060809@secerno.com>

Rainer Gerhards wrote:
> OK, I will look into that, but I can not promise how trivial this may be
> ;) I keep you posted.

Thanks very much for all your help.

Regards,


Pete.
-- 
Pete Philips
Secerno Ltd
Email: pete.philips at secerno.com
PGP key: http://www.secerno.com/pgp/pete.gpg


From rgerhards at hq.adiscon.com  Wed Nov 26 12:20:48 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 12:20:48 +0100
Subject: [rsyslog] rsyslog documentation
Message-ID: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>

Hi all,

as you probably know, I keep rsyslog's documentation in html files, with
the man pages containing the bare minimum to make sense of the system.
This need arises from the sheer volume of the information that must be
provided.

However, this makes the documentation pretty closed (everything needs to
go through me) and consequently I have seen very few doc contributions.
It probably also makes it more complicated than needed to translate the
documentation.

Initially, I considered a purely web-based doc vs. a html-file based
doc. I went for the file-based doc as this can easily be distributed
together with every rsyslog version.

However, looking at other projects, many have adapted a web-based
approach where version differences are flagged within the documentation.
The advantage of a web-based doc is that we can use thinks like a wiki
to generate it. That, I think, would make it much easier for users to
contribute doc or at least samples. Also, it looks like a web-based doc
is also more convenient for most users. Everyone has a browser open and
checks the web, but who installs doc packages? ;)

So I am now consider changing to a web-based system, too. I'd probably
consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
point. Since I created it, it receives a slow but steady traffic
increase and has now around the same number of hits than the rsyslog
main site. I would move over the current file-based doc into that system
and at the same time see that I can improve the structure and usability
of the doc.

With the many new and powerful features that appeared in rsyslog over
the past couple of month, I think it is very important to make them
accessible by a sufficiently good doc. The current one is, to phrase it
politely, not good. It probably even hinders adoption of rsyslog in some
cases.

With a web-based system open to user contributions I hope to solve this
issues.

Please let me know your thoughts. All feedback is deeply appreciated.

Many thanks,
Rainer Gerhards



From mbiebl at gmail.com  Wed Nov 26 13:10:38 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 26 Nov 2008 13:10:38 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
Message-ID: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>

2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>
> However, looking at other projects, many have adapted a web-based
> approach where version differences are flagged within the documentation.
> The advantage of a web-based doc is that we can use thinks like a wiki
> to generate it. That, I think, would make it much easier for users to
> contribute doc or at least samples. Also, it looks like a web-based doc
> is also more convenient for most users. Everyone has a browser open and
> checks the web, but who installs doc packages? ;)
>
> So I am now consider changing to a web-based system, too. I'd probably
> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
> point. Since I created it, it receives a slow but steady traffic
> increase and has now around the same number of hits than the rsyslog
> main site. I would move over the current file-based doc into that system
> and at the same time see that I can improve the structure and usability
> of the doc.
>
> With the many new and powerful features that appeared in rsyslog over
> the past couple of month, I think it is very important to make them
> accessible by a sufficiently good doc. The current one is, to phrase it
> politely, not good. It probably even hinders adoption of rsyslog in some
> cases.
>
> With a web-based system open to user contributions I hope to solve this
> issues.
>
> Please let me know your thoughts. All feedback is deeply appreciated.

Imho it would be very unfortunate, if the documentation would only be
available online.

A wiki can be a very good *additional* source of documentation (for
stuff like best practices, tips and tricks), but it doesn't substitute
a well written manual.

What I would rather like to see (and I know, I'm repeating myself
here), is to have the existing documentation a bit more structured and
easier accessible. I posted examples like the exim [1] or postgresql
[2] documentation, which I think are excellent.

The postgresql documentation is interesting. If you follow [2] the
link, you can see, that users are able to add comments, which could be
helpful to improve the overall documentation.
There is still a "static" version though, also available as pdf, which
you could print and even use as a book.

Cheers,
Michael


[1] http://www.exim.org/exim-html-current/doc/html/spec_html/index.html
http://www.exim.org/exim-pdf-current/doc/spec.pdf
[2] http://www.postgresql.org/docs/8.3/interactive/index.html
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Wed Nov 26 13:16:02 2008
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 26 Nov 2008 13:16:02 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>

2008/11/26 Michael Biebl <mbiebl at gmail.com>:
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.

And distribute easily with the rsyslog release tarball

Cheers,
Michael

P.S: Maybe a book could be an additional source of income.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Wed Nov 26 13:28:25 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 13:28:25 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>

Hi Michael,

thanks for the feedback.

On Wed, 2008-11-26 at 13:10 +0100, Michael Biebl wrote:
> Imho it would be very unfortunate, if the documentation would only be
> available online.
> 
> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.
> 
> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.

I think the PHP manual is also a good sample.

But actually this doesn't solve my main point: It is quite
time-consuming to create a good doc. I am ready to put a bit more time
into that (moving that away from development), but I would like to
encourage user contributions. Without bashing, it is one thing to show a
good example, but it is a totally different one to provide good
content ;)

The bottom line is that I know I could do better, but I have not
sufficient time (and probably not the best talent in that area) to
actually do it.

> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.

This is the main point. Maybe I can find a way to easily make user
comments available to the current doc set. That would help drive in user
contributions (at least I hope so). The problem again is that I simply
can not invest a few weeks of my time "just" to get the right doc
display system done.

It's a trade-off between time available, hoping to get contributions and
quality of doc.

Again, I value your thoughts very much, they go in the same direction of
what I hope to have, but it looks like I currently can not achieve this
and need to lower my demands in order to get working, solution - but one
that is better than the current state...

Rainer



From rgerhards at hq.adiscon.com  Wed Nov 26 13:43:12 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 13:43:12 +0100
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
Message-ID: <1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>

Hi all,

some times it seems to help ask around ;) Andre who cares for the web
sites (also the phpLogCon lead devel) has pointed out to me that we may
try the CMS' comment function. We have now enabled that. It seems to
solve the immediate need. So this looks like a way we can at least try
out.

Anyhow, if you have any additional thoughts please let me know them.
They probably help shape a medium-term doc strategy.

Thanks,
Rainer

On Wed, 2008-11-26 at 13:28 +0100, Rainer Gerhards wrote:
> Hi Michael,
> 
> thanks for the feedback.
> 
> On Wed, 2008-11-26 at 13:10 +0100, Michael Biebl wrote:
> > Imho it would be very unfortunate, if the documentation would only be
> > available online.
> > 
> > A wiki can be a very good *additional* source of documentation (for
> > stuff like best practices, tips and tricks), but it doesn't substitute
> > a well written manual.
> > 
> > What I would rather like to see (and I know, I'm repeating myself
> > here), is to have the existing documentation a bit more structured and
> > easier accessible. I posted examples like the exim [1] or postgresql
> > [2] documentation, which I think are excellent.
> 
> I think the PHP manual is also a good sample.
> 
> But actually this doesn't solve my main point: It is quite
> time-consuming to create a good doc. I am ready to put a bit more time
> into that (moving that away from development), but I would like to
> encourage user contributions. Without bashing, it is one thing to show a
> good example, but it is a totally different one to provide good
> content ;)
> 
> The bottom line is that I know I could do better, but I have not
> sufficient time (and probably not the best talent in that area) to
> actually do it.
> 
> > The postgresql documentation is interesting. If you follow [2] the
> > link, you can see, that users are able to add comments, which could be
> > helpful to improve the overall documentation.
> 
> This is the main point. Maybe I can find a way to easily make user
> comments available to the current doc set. That would help drive in user
> contributions (at least I hope so). The problem again is that I simply
> can not invest a few weeks of my time "just" to get the right doc
> display system done.
> 
> It's a trade-off between time available, hoping to get contributions and
> quality of doc.
> 
> Again, I value your thoughts very much, they go in the same direction of
> what I hope to have, but it looks like I currently can not achieve this
> and need to lower my demands in order to get working, solution - but one
> that is better than the current state...
> 
> Rainer
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com



From rfujita at redhat.com  Wed Nov 26 13:19:03 2008
From: rfujita at redhat.com (=?ISO-2022-JP?B?GyRCTkcbKEIgGyRCRiNFRBsoQg==?=)
Date: Wed, 26 Nov 2008 21:19:03 +0900
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<aa76cdac0811260416j7fba8cc9k9568abfce797b2da@mail.gmail.com>
Message-ID: <E5D6C192-A983-4807-8B63-686A2D61A018@redhat.com>

Hi,

> P.S: Maybe a book could be an additional source of income.

+1

Best Rio.

On 2008/11/26, at 21:16, Michael Biebl wrote:

> 2008/11/26 Michael Biebl <mbiebl at gmail.com>:
>>
>> The postgresql documentation is interesting. If you follow [2] the
>> link, you can see, that users are able to add comments, which could  
>> be
>> helpful to improve the overall documentation.
>> There is still a "static" version though, also available as pdf,  
>> which
>> you could print and even use as a book.
>
> And distribute easily with the rsyslog release tarball
>
> Cheers,
> Michael
>
> P.S: Maybe a book could be an additional source of income.
>
>
> -- 
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################



From rfujita at redhat.com  Wed Nov 26 13:18:34 2008
From: rfujita at redhat.com (=?ISO-2022-JP?B?GyRCTkcbKEIgGyRCRiNFRBsoQg==?=)
Date: Wed, 26 Nov 2008 21:18:34 +0900
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <0C7538F5-2324-4EB7-9A55-02E28965ABDA@redhat.com>

Hi all,

> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/ 
> index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html

Both these links are made with DocBook.

Best Rio.

On 2008/11/26, at 21:10, Michael Biebl wrote:

> 2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> However, looking at other projects, many have adapted a web-based
>> approach where version differences are flagged within the  
>> documentation.
>> The advantage of a web-based doc is that we can use thinks like a  
>> wiki
>> to generate it. That, I think, would make it much easier for users to
>> contribute doc or at least samples. Also, it looks like a web-based  
>> doc
>> is also more convenient for most users. Everyone has a browser open  
>> and
>> checks the web, but who installs doc packages? ;)
>>
>> So I am now consider changing to a web-based system, too. I'd  
>> probably
>> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
>> point. Since I created it, it receives a slow but steady traffic
>> increase and has now around the same number of hits than the rsyslog
>> main site. I would move over the current file-based doc into that  
>> system
>> and at the same time see that I can improve the structure and  
>> usability
>> of the doc.
>>
>> With the many new and powerful features that appeared in rsyslog over
>> the past couple of month, I think it is very important to make them
>> accessible by a sufficiently good doc. The current one is, to  
>> phrase it
>> politely, not good. It probably even hinders adoption of rsyslog in  
>> some
>> cases.
>>
>> With a web-based system open to user contributions I hope to solve  
>> this
>> issues.
>>
>> Please let me know your thoughts. All feedback is deeply appreciated.
>
> Imho it would be very unfortunate, if the documentation would only be
> available online.
>
> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.
>
> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.
>
> Cheers,
> Michael
>
>
> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/ 
> index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html
> -- 
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com

########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################



From mrdemeanour at jackpot.uk.net  Wed Nov 26 13:24:42 2008
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Wed, 26 Nov 2008 12:24:42 +0000
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <492D400A.2010105@jackpot.uk.net>

Michael Biebl wrote:
>> 
>> With a web-based system open to user contributions I hope to solve
>> this issues.
>> 
>> Please let me know your thoughts. All feedback is deeply
>> appreciated.
> 
> Imho it would be very unfortunate, if the documentation would only be
>  available online.

+1.

Many projects now use Docbook, to generate variously HTML, PDF and manpages.

On most of my servers, I have no X GUI set up; HTML docs are therefore a
pain in the neck, unless the HTML is extremely clean (and even then it's
a hassle).

The documentation should be a part of the project, and it should be
possible to check it out, modify it (e.g. translate it) and 'build' it.

I think wiki docs are great, but I also think the foundation for a
documentation wiki should be the authorised version of the docs. Users
can then comment on that authorised version, which should be the docs
that ship with the product. It might be possible to rig some mechanism
for automagically pulling content from the wiki back into the
repository; but I'm not sure how to go about that.

Thanks for this excellent project, and for being so committed to it!
-- 
Jack.


From mikel at irontec.com  Wed Nov 26 16:24:32 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 16:24:32 +0100
Subject: [rsyslog] rsyslog freeze dude
Message-ID: <492D6A30.9060105@irontec.com>

Hello
If I configure rsyslog client like this:

WorkDirectory /root/rsyslog
$ActionQueueType LinkedList
$ActionQueueFileName myfile
$ActionResumeRetryCount -1

and

*.* @@somehost.domain.com

Is possible that lost of conectivity (internet down) of this server 
during days, produce that server freeze? or not be possible to login to 
the machine?

If I do /etc/init.d/rsyslog restart it takes long long time to restart.

Please Rainer I hope you help me. thanks


-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 16:39:36 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:39:36 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D6A30.9060105@irontec.com>
References: <492D6A30.9060105@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>

Hi Mikel,

which version are you using? The "ssh hang" can probably be related to
the initial thought that imuxsock messages can be delayed. This has been
changed in recent releases.

Other than that, please note that actions are retried in intervals and
these intervals get longer each time the action is retried without
success. This is done in order to save unnecessary work. I think there
currently is no config setting to change the default intervals.

It may be useful to get a shutdown debug log from such an occurrence.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Wednesday, November 26, 2008 4:25 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog freeze dude
> 
> Hello
> If I configure rsyslog client like this:
> 
> WorkDirectory /root/rsyslog
> $ActionQueueType LinkedList
> $ActionQueueFileName myfile
> $ActionResumeRetryCount -1
> 
> and
> 
> *.* @@somehost.domain.com
> 
> Is possible that lost of conectivity (internet down) of this server
> during days, produce that server freeze? or not be possible to login
to
> the machine?
> 
> If I do /etc/init.d/rsyslog restart it takes long long time to
restart.
> 
> Please Rainer I hope you help me. thanks
> 
> 
> --
> Mikel Jimenez Fernandez
> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
> +34 94.404.81.82
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Nov 26 16:54:42 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:54:42 +0100
Subject: [rsyslog] rsyslog 4.1.1 (devel) released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F714@grfint2.intern.adiscon.com>

Hi all,

rsyslog 4.1.1, a member of the development branch, has been released
today. Most importantly, it now contains the ability to drop privileges,
thus enabling it to be executed without root permissions even if
listening to privileged network ports. The release also contains a fix
for a memory leak in the postgres output module and a fix for the klog
input module, which did not compile under BSD.

Download:
http://www.rsyslog.com/Article317.phtml

Change Log:
http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-140.phtml

The "privilege drop" functionality was discussed on this list and, while
not perfect, seems to provide some useful service. A number of
restrictions is described here:

http://wiki.rsyslog.com/index.php/Security

I would like to express my gratitude for all opinions raised. If you
have any more comments, please let them flow. Also feel free to add
anything of interest to the Wiki. Obviously, I would be very interested
in feedback from folks who really try the new functionality out.

I hope this release is useful.

Rainer



From mikel at irontec.com  Wed Nov 26 16:54:51 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 16:54:51 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
Message-ID: <492D714B.4050107@irontec.com>

Rainer Gerhards escribi?:
> Hi Mikel,
>
> which version are you using? The "ssh hang" can probably be related to
> the initial thought that imuxsock messages can be delayed. This has been
> changed in recent releases.
>
> Other than that, please note that actions are retried in intervals and
> these intervals get longer each time the action is retried without
> success. This is done in order to save unnecessary work. I think there
> currently is no config setting to change the default intervals.
>
> It may be useful to get a shutdown debug log from such an occurrence.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Wednesday, November 26, 2008 4:25 PM
>> To: rsyslog-users
>> Subject: [rsyslog] rsyslog freeze dude
>>
>> Hello
>> If I configure rsyslog client like this:
>>
>> WorkDirectory /root/rsyslog
>> $ActionQueueType LinkedList
>> $ActionQueueFileName myfile
>> $ActionResumeRetryCount -1
>>
>> and
>>
>> *.* @@somehost.domain.com
>>
>> Is possible that lost of conectivity (internet down) of this server
>> during days, produce that server freeze? or not be possible to login
>>     
> to
>   
>> the machine?
>>
>> If I do /etc/init.d/rsyslog restart it takes long long time to
>>     
> restart.
>   
>> Please Rainer I hope you help me. thanks
>>
>>
>> --
>> Mikel Jimenez Fernandez
>> Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
>> +34 94.404.81.82
>>
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Hello Rainer


root at ironpbx:~# rsyslogd -v
rsyslogd 3.16.1, compiled with:
      FEATURE_REGEXP:           Yes
      FEATURE_LARGEFILE:        Yes
      FEATURE_NETZIP (message compression):     Yes
      GSSAPI Kerberos 5 support:                No
      FEATURE_DEBUG (debug build, slow code):   No
      Runtime Instrumentation (slow code):      No

See http://www.rsyslog.com for more information.



Say me if you need more information.

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 16:56:52 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 16:56:52 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D714B.4050107@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
	<492D714B.4050107@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>

Hi Mikel,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
> Sent: Wednesday, November 26, 2008 4:55 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog freeze dude
 
> root at ironpbx:~# rsyslogd -v
> rsyslogd 3.16.1, compiled with:
>       FEATURE_REGEXP:           Yes
>       FEATURE_LARGEFILE:        Yes
>       FEATURE_NETZIP (message compression):     Yes
>       GSSAPI Kerberos 5 support:                No
>       FEATURE_DEBUG (debug build, slow code):   No
>       Runtime Instrumentation (slow code):      No
> 
> See http://www.rsyslog.com for more information.

Please upgrade to the recent v3-stable. That will probably solve the big
issue, if not all.

Rainer


From mikel at irontec.com  Wed Nov 26 17:03:15 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 17:03:15 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
Message-ID: <492D7343.1040101@irontec.com>

Rainer Gerhards escribi?:
> Hi Mikel,
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>> Sent: Wednesday, November 26, 2008 4:55 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog freeze dude
>>     
>  
>   
>> root at ironpbx:~# rsyslogd -v
>> rsyslogd 3.16.1, compiled with:
>>       FEATURE_REGEXP:           Yes
>>       FEATURE_LARGEFILE:        Yes
>>       FEATURE_NETZIP (message compression):     Yes
>>       GSSAPI Kerberos 5 support:                No
>>       FEATURE_DEBUG (debug build, slow code):   No
>>       Runtime Instrumentation (slow code):      No
>>
>> See http://www.rsyslog.com for more information.
>>     
>
> Please upgrade to the recent v3-stable. That will probably solve the big
> issue, if not all.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
OK

What is the last stable version?

There is a debian package?
Other thing:
The workDirectory is necesary that exist?
For example: WorkDirectory /rsyslog/work is a directory in / called 
rsyslog and another one inside this called work?

If they aren?t can cause a bug?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 17:13:10 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 17:13:10 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D7343.1040101@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
	<492D7343.1040101@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>

> OK
> 
> What is the last stable version?

I checked, it is 3.20.0 (and while checking I noticed that the project status was once again outdated ;))

> 
> There is a debian package?

This I don't know.

> Other thing:
> The workDirectory is necesary that exist?
Yes, absolutely

> For example: WorkDirectory /rsyslog/work is a directory in / called
> rsyslog and another one inside this called work?
> 
> If they aren?t can cause a bug?

I'd say yes.

Rainer


From mikel at irontec.com  Wed Nov 26 17:22:37 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Wed, 26 Nov 2008 17:22:37 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
Message-ID: <492D77CD.3000204@irontec.com>

Rainer Gerhards escribi?:
>> OK
>>
>> What is the last stable version?
>>     
>
> I checked, it is 3.20.0 (and while checking I noticed that the project status was once again outdated ;))
>
>   
>> There is a debian package?
>>     
>
> This I don't know.
>
>   
>> Other thing:
>> The workDirectory is necesary that exist?
>>     
> Yes, absolutely
>
>   
>> For example: WorkDirectory /rsyslog/work is a directory in / called
>> rsyslog and another one inside this called work?
>>
>> If they aren?t can cause a bug?
>>     
>
> I'd say yes.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   

Before installing this version, would be possible to do more debugging?

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Wed Nov 26 17:35:13 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 26 Nov 2008 17:35:13 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492D77CD.3000204@irontec.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
	<492D77CD.3000204@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>

> Before installing this version, would be possible to do more
debugging?

Sorry, I don't want to upset you, but I have no time to look at things
already fixed. Quite seriously, we have paid support options for this. I
see that it can make sense for you to stay with the older version, but
there is absolutely no benefit to the community. Sorry...

Rainer


From david at lang.hm  Wed Nov 26 18:57:05 2008
From: david at lang.hm (david at lang.hm)
Date: Wed, 26 Nov 2008 09:57:05 -0800 (PST)
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
Message-ID: <alpine.DEB.1.10.0811260954070.22089@asgard.lang.hm>

On Wed, 26 Nov 2008, Michael Biebl wrote:

> 2008/11/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> However, looking at other projects, many have adapted a web-based
>> approach where version differences are flagged within the documentation.
>> The advantage of a web-based doc is that we can use thinks like a wiki
>> to generate it. That, I think, would make it much easier for users to
>> contribute doc or at least samples. Also, it looks like a web-based doc
>> is also more convenient for most users. Everyone has a browser open and
>> checks the web, but who installs doc packages? ;)
>>
>> So I am now consider changing to a web-based system, too. I'd probably
>> consider the rsyslog wiki (http://wiki.rsyslog.com) a good starting
>> point. Since I created it, it receives a slow but steady traffic
>> increase and has now around the same number of hits than the rsyslog
>> main site. I would move over the current file-based doc into that system
>> and at the same time see that I can improve the structure and usability
>> of the doc.
>>
>> With the many new and powerful features that appeared in rsyslog over
>> the past couple of month, I think it is very important to make them
>> accessible by a sufficiently good doc. The current one is, to phrase it
>> politely, not good. It probably even hinders adoption of rsyslog in some
>> cases.
>>
>> With a web-based system open to user contributions I hope to solve this
>> issues.
>>
>> Please let me know your thoughts. All feedback is deeply appreciated.
>
> Imho it would be very unfortunate, if the documentation would only be
> available online.

I definantly agree. whatever is done online needs to have some way of 
having a snapshot of it done to be distributed for offline access.

> A wiki can be a very good *additional* source of documentation (for
> stuff like best practices, tips and tricks), but it doesn't substitute
> a well written manual.

the other problem with a wiki is that unless you have the time to review 
the changes there is a real risk that the documentation will be wrong. 
this is true of any user-contributed documentation, but with a wiki there 
is no way for other users to know what has been checked and is correct vs 
what is wrong.

the suggestion elsewhere in this thread to have an authoritative source 
with the ability for people to make comments does a good job of 
differentiating between the two.

David Lang

> What I would rather like to see (and I know, I'm repeating myself
> here), is to have the existing documentation a bit more structured and
> easier accessible. I posted examples like the exim [1] or postgresql
> [2] documentation, which I think are excellent.
>
> The postgresql documentation is interesting. If you follow [2] the
> link, you can see, that users are able to add comments, which could be
> helpful to improve the overall documentation.
> There is still a "static" version though, also available as pdf, which
> you could print and even use as a book.
>
> Cheers,
> Michael
>
>
> [1] http://www.exim.org/exim-html-current/doc/html/spec_html/index.html
> http://www.exim.org/exim-pdf-current/doc/spec.pdf
> [2] http://www.postgresql.org/docs/8.3/interactive/index.html
>


From hks.private at gmail.com  Wed Nov 26 21:20:51 2008
From: hks.private at gmail.com ((private) HKS)
Date: Wed, 26 Nov 2008 15:20:51 -0500
Subject: [rsyslog] rsyslog documentation
In-Reply-To: <1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>
References: <1227698448.22599.9.camel@rgf9dev.intern.adiscon.com>
	<aa76cdac0811260410w73b01226w98ef39b355096355@mail.gmail.com>
	<1227702505.22599.19.camel@rgf9dev.intern.adiscon.com>
	<1227703392.22599.23.camel@rgf9dev.intern.adiscon.com>
Message-ID: <ac9ded160811261220i7fb06ff7r7cfe3a69f010db91@mail.gmail.com>

On Wed, Nov 26, 2008 at 7:43 AM, Rainer Gerhards
<rgerhards at hq.adiscon.com> wrote:
> Hi all,
>
> some times it seems to help ask around ;) Andre who cares for the web
> sites (also the phpLogCon lead devel) has pointed out to me that we may
> try the CMS' comment function. We have now enabled that. It seems to
> solve the immediate need. So this looks like a way we can at least try
> out.
>
> Anyhow, if you have any additional thoughts please let me know them.
> They probably help shape a medium-term doc strategy.
>
> Thanks,
> Rainer


I agree with most contributors so far, that a typical user-contributed
wiki alone is not sufficient. For a perfect example of a great project
crippled by abysmal wiki documentation, see OpenNMS.

Wikis /can/ work as official documentation, provided that they are
carefully thought through and official sources are the main
contributors. A general structure also needs to be in place so that
things don't spread out into a web of maze-like links with multiple
pages describing the same subject, albeit with small and crucial
differences. Their main benefits are, obviously, some measure of
self-correction and allowing users to share their solutions to
problems the developers may not have encountered.


Don't know if that helps, but I hope so.

-HKS


From mikel at irontec.com  Thu Nov 27 10:07:39 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 27 Nov 2008 10:07:39 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
Message-ID: <492E635B.80607@irontec.com>

Rainer Gerhards escribi?:
>> Before installing this version, would be possible to do more
>>     
> debugging?
>
> Sorry, I don't want to upset you, but I have no time to look at things
> already fixed. Quite seriously, we have paid support options for this. I
> see that it can make sense for you to stay with the older version, but
> there is absolutely no benefit to the community. Sorry...
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Ok Rainer
Thanks, really. I?m going to use the stable version.

Last question:
Why could be the reason, if I block internet in this host, and when pass 
10-12 hours, obviously not logging to remote (because firewall), but why 
not locally to?

I do:
logger "message"

And I have not notice in /var/log/syslog or /var/log/messages, and in 
normal situations yes...

Thanks

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From mikel at irontec.com  Thu Nov 27 10:19:25 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Thu, 27 Nov 2008 10:19:25 +0100
Subject: [rsyslog] milliseconds timestamp
In-Reply-To: <F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
References: <4922BEBC.3050606@irontec.com><577465F99B41C842AAFBE9ED71E70ABA44F677@grfint2.intern.adiscon.com><F1745A9F580ACD44BBA4BAC95A69F04F449EAE@rmail.adiscon.com><577465F99B41C842AAFBE9ED71E70ABA44F680@grfint2.intern.adiscon.com><a0d9b4bcf394a88aaa5841ea67c1a7f2@irontec.com><F1745A9F580ACD44BBA4BAC95A69F04F449EB0@rmail.adiscon.com>	<577465F99B41C842AAFBE9ED71E70ABA44F686@grfint2.intern.adiscon.com>
	<F1745A9F580ACD44BBA4BAC95A69F04F449EB2@rmail.adiscon.com>
Message-ID: <492E661D.8020501@irontec.com>

Andre Lorbach escribi?:
> By not adding it into the standard schema, you mean the default table
> layout for the database? 
> We may could reuse an existing unused field instead?
>
> --
> Andre 
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>> Sent: Mittwoch, 19. November 2008 10:39
>> To: rsyslog-users
>> Subject: Re: [rsyslog] milliseconds timestamp
>>
>> Andre,
>>
>> So do you think we should add this to the standard schema? I am a bit
>> hesitant, as not all folks will probably want that and it keeps
>> backward
>> compatibility. Maybe it is better to just us a standard name but not
>> include it in the standard schema.
>>
>> What do you (and others) think?
>>
>> Rainer
>>
>>     
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>>> Sent: Wednesday, November 19, 2008 10:37 AM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>
>>> I think this would be possible yes, we need to define a common name
>>>       
>> for
>>     
>>> the database and property field in rsyslog, then we can start adding
>>> support for it in phplogcon.
>>>
>>> regards,
>>> Andre
>>>
>>>       
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of mikel
>>>> Sent: Dienstag, 18. November 2008 21:06
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>
>>>>
>>>> Dear Andre
>>>>
>>>> So, in near future will be possible?
>>>>
>>>> Or in far future?
>>>>
>>>> Thanx
>>>>
>>>> On Tue, 18 Nov 2008 18:00:04 +0100, "Rainer Gerhards"
>>>> <rgerhards at hq.adiscon.com> wrote:
>>>>         
>>>>> Andre and all,
>>>>>
>>>>> the rsyslog engine has the necessary plumbing and I know of at
>>>>>           
>>> least
>>>       
>>>> one
>>>>         
>>>>> user who stores subsecond timestamps in this way. As far as
>>>>>           
>> rsyslog
>>     
>>>> is
>>>>         
>>>>> concerned, this just requires a special timestamp. So if you
>>>>>           
>> would
>>     
>>>>> support that in phpLogCon (what I would find a good idea ;)), I
>>>>>           
>>> would
>>>       
>>>>> include a standard template into rsyslog (hardcoded). That would
>>>>>           
>>> make
>>>       
>>>> it
>>>>         
>>>>> easier for people to work with it. But maybe we get along by
>>>>>           
> just
>   
>>>>> properly documenting it, because the schema must be modified in
>>>>>           
>> any
>>     
>>>>> case.
>>>>>
>>>>> Another alternative would be to update the default schema, but I
>>>>>           
>> am
>>     
>>>> not
>>>>         
>>>>> sure it that is actually a good idea. On the other hand, high-
>>>>>           
>>>> precision
>>>>         
>>>>> timestamps hopefully get into more widespread use...
>>>>>
>>>>> Thoughts from the community are appreciated.
>>>>>
>>>>> Thanks,
>>>>> Rainer
>>>>>
>>>>>           
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>> bounces at lists.adiscon.com] On Behalf Of Andre Lorbach
>>>>>> Sent: Tuesday, November 18, 2008 5:56 PM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>>>
>>>>>> I think it wouldn't be so difficult to implement, if the
>>>>>>             
>>>> milliseconds
>>>>         
>>>>>> were simply stored in a second field.
>>>>>> We can add custom new fields into phpLogCon very easily.
>>>>>>
>>>>>> best regards,
>>>>>> Andre Lorbach
>>>>>>
>>>>>>             
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>>> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
>>>>>>> Sent: Dienstag, 18. November 2008 16:39
>>>>>>> To: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] milliseconds timestamp
>>>>>>>
>>>>>>> Hi Mikel,
>>>>>>>
>>>>>>> I have talked to Andre, phpLogCon's lead developer. He says
>>>>>>>               
>>> MySQL
>>>       
>>>>>> does
>>>>>>             
>>>>>>> not support millisecond resolution in timestamps. So a
>>>>>>>               
>>> work-around
>>>       
>>>>>>> would
>>>>>>> be to store them to another field and add this as a second
>>>>>>>               
>> field
>>     
>>>> to
>>>>         
>>>>>> the
>>>>>>             
>>>>>>> phpLogCon view. I think it would probably smart to enable
>>>>>>>               
>>>> phpLogCon
>>>>         
>>>>>> to
>>>>>>             
>>>>>>> combine two such fields, but I leave this to Andre...
>>>>>>>
>>>>>>> Rainer
>>>>>>>
>>>>>>>               
>>>>>>>> -----Original Message-----
>>>>>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>>>>>> bounces at lists.adiscon.com] On Behalf Of Mikel Jimenez
>>>>>>>> Sent: Tuesday, November 18, 2008 2:10 PM
>>>>>>>> To: rsyslog-users
>>>>>>>> Subject: [rsyslog] milliseconds timestamp
>>>>>>>>
>>>>>>>> Hello
>>>>>>>> I am using phplogcon viewer and I have loosed milliseconds
>>>>>>>>                 
>>>>>>> timestamps.
>>>>>>>               
>>>>>>>> I watched that "DATE" in mysql databases is date-time type.
>>>>>>>>
>>>>>>>> Does rsyslog gives the date in milliseconds? is possible to
>>>>>>>>                 
>>> see
>>>       
>>>>>> this
>>>>>>             
>>>>>>>> milliseconds din phplogcon?
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mikel Jimenez Fernandez
>>>>>>>> Irontec, Internet y Sistemas sobre GNU/LinuX -
>>>>>>>>                 
>>>>>> http://www.irontec.com
>>>>>>             
>>>>>>>> +34 94.404.81.82
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>                 
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Hello Andre/Rainer

Have do you decide how to progress with that?

Can I help

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Fri Nov 28 09:50:29 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 09:50:29 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492E635B.80607@irontec.com>
References: <492D6A30.9060105@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>
	<492D714B.4050107@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>
	<492D7343.1040101@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>
	<492D77CD.3000204@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>
	<492E635B.80607@irontec.com>
Message-ID: <1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>

Mikel,

sorry, I was busy yesterday. See inline...
On Thu, 2008-11-27 at 10:07 +0100, Mikel Jimenez wrote:

> Last question:
> Why could be the reason, if I block internet in this host, and when pass 
> 10-12 hours, obviously not logging to remote (because firewall), but why 
> not locally to?
> 
> I do:
> logger "message"
> 
> And I have not notice in /var/log/syslog or /var/log/messages, and in 
> normal situations yes...

If I am right (the update will prove that) the reason is that the local
log socket is not read (because rsyslog delays imuxsock) and thus fills
up. Then, other processes trying to write to that socket will block
during the write process.

Please note that the rsyslog core will discard messages after some time,
but the rate at which this happens is too slow to get the system back to
normal. As I said, this should disappear with the recent stable version.

HTH
Rainer



From mikel at irontec.com  Fri Nov 28 10:41:37 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Fri, 28 Nov 2008 10:41:37 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com>
	<1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
Message-ID: <492FBCD1.7010509@irontec.com>

Rainer Gerhards escribi?:
> Mikel,
>
> sorry, I was busy yesterday. See inline...
> On Thu, 2008-11-27 at 10:07 +0100, Mikel Jimenez wrote:
>
>   
>> Last question:
>> Why could be the reason, if I block internet in this host, and when pass 
>> 10-12 hours, obviously not logging to remote (because firewall), but why 
>> not locally to?
>>
>> I do:
>> logger "message"
>>
>> And I have not notice in /var/log/syslog or /var/log/messages, and in 
>> normal situations yes...
>>     
>
> If I am right (the update will prove that) the reason is that the local
> log socket is not read (because rsyslog delays imuxsock) and thus fills
> up. Then, other processes trying to write to that socket will block
> during the write process.
>
> Please note that the rsyslog core will discard messages after some time,
> but the rate at which this happens is too slow to get the system back to
> normal. As I said, this should disappear with the recent stable version.
>
> HTH
> Rainer
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Thanks for your explication Rainer!! I will install 3.20 right now!!

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From rgerhards at hq.adiscon.com  Fri Nov 28 10:43:22 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 10:43:22 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <492FBCD1.7010509@irontec.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com><1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>
	<492FBCD1.7010509@irontec.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>

> Thanks for your explication Rainer!! I will install 3.20 right now!!

Please let me know if the problem then re-appears. I don't think so, but
if it does this is definitely something we should look into ASAP. I'd
also appreciate if you let me know if the issue is fixed after the
upgrade.

Rainer


From mikel at irontec.com  Fri Nov 28 10:46:05 2008
From: mikel at irontec.com (Mikel Jimenez)
Date: Fri, 28 Nov 2008 10:46:05 +0100
Subject: [rsyslog] rsyslog freeze dude
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>
References: <492D6A30.9060105@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F713@grfint2.intern.adiscon.com>	<492D714B.4050107@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F715@grfint2.intern.adiscon.com>	<492D7343.1040101@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F716@grfint2.intern.adiscon.com>	<492D77CD.3000204@irontec.com>	<577465F99B41C842AAFBE9ED71E70ABA44F718@grfint2.intern.adiscon.com>	<492E635B.80607@irontec.com><1227862229.15897.2.camel@rgf9dev.intern.adiscon.com>	<492FBCD1.7010509@irontec.com>
	<577465F99B41C842AAFBE9ED71E70ABA44F746@grfint2.intern.adiscon.com>
Message-ID: <492FBDDD.2070308@irontec.com>

Rainer Gerhards escribi?:
>> Thanks for your explication Rainer!! I will install 3.20 right now!!
>>     
>
> Please let me know if the problem then re-appears. I don't think so, but
> if it does this is definitely something we should look into ASAP. I'd
> also appreciate if you let me know if the issue is fixed after the
> upgrade.
>
> Rainer
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   
Ok, I will do it

-- 
Mikel Jimenez Fernandez
Irontec, Internet y Sistemas sobre GNU/LinuX - http://www.irontec.com
+34 94.404.81.82




From peter.matulis at canonical.com  Fri Nov 28 17:59:56 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 11:59:56 -0500
Subject: [rsyslog] property based filtering with regex operator
Message-ID: <4930238C.5040208@canonical.com>

Hi.  I tried the following rule to match any occurrence of 'sda' but it
did not work:

:msg, regex, ".*sda.*"         /var/log/properties/sd_regex

Replacing the regex with simply "sda" matches.  Do I not need to take
the entire message into account?

Peter


From peter.matulis at canonical.com  Fri Nov 28 19:05:54 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 13:05:54 -0500
Subject: [rsyslog] QueueDiscardSeverity defaults
Message-ID: <49303302.3090200@canonical.com>

Hi.  The documentation states that for both the main queue and any
action queue the default value for QueueDiscardSeverity is '4'.  In
another place it is stated to be '8' (to prevent any message loss).

Can anyone also confirm whether both numerical as well as textual
severity can be given as a value?  The docs state that text is ok for
MainMsg but not for Action.

Peter


From rgerhards at hq.adiscon.com  Fri Nov 28 19:54:44 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 19:54:44 +0100
Subject: [rsyslog] property based filtering with regex operator
In-Reply-To: <4930238C.5040208@canonical.com>
References: <4930238C.5040208@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F754@grfint2.intern.adiscon.com>

Hi Peter,

can you run this through the debug log? That spits out additional
information, maybe that points to the source of the problem.

It's evening over here, I do not have a test system right at hand and
will be off for family things over the weekend. If it's not urgent, I
can run a test on Monday.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 6:00 PM
> To: rsyslog-users
> Subject: [rsyslog] property based filtering with regex operator
> 
> Hi.  I tried the following rule to match any occurrence of 'sda' but
it
> did not work:
> 
> :msg, regex, ".*sda.*"         /var/log/properties/sd_regex
> 
> Replacing the regex with simply "sda" matches.  Do I not need to take
> the entire message into account?
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Nov 28 19:57:08 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 28 Nov 2008 19:57:08 +0100
Subject: [rsyslog] QueueDiscardSeverity defaults
In-Reply-To: <49303302.3090200@canonical.com>
References: <49303302.3090200@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F755@grfint2.intern.adiscon.com>

Peter,

I think that's another doc issue.

The default is 8 now, it was 4 but this has shown to be very
unproductive. I think textual priorities can now be given, but will need
to double-check.

I'll see that I update the doc ASAP.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 7:06 PM
> To: rsyslog-users
> Subject: [rsyslog] QueueDiscardSeverity defaults
> 
> Hi.  The documentation states that for both the main queue and any
> action queue the default value for QueueDiscardSeverity is '4'.  In
> another place it is stated to be '8' (to prevent any message loss).
> 
> Can anyone also confirm whether both numerical as well as textual
> severity can be given as a value?  The docs state that text is ok for
> MainMsg but not for Action.
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From peter.matulis at canonical.com  Fri Nov 28 20:29:26 2008
From: peter.matulis at canonical.com (Peter Matulis)
Date: Fri, 28 Nov 2008 14:29:26 -0500
Subject: [rsyslog] broken link to 4.1.1 release
Message-ID: <49304696.4000403@canonical.com>

The download link to the latest devel release (4.1.1) is broken:


http://www.rsyslog.com/Downloads-req-getit-lid-140.phtml


Peter


From rgerhards at hq.adiscon.com  Sat Nov 29 16:54:31 2008
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 29 Nov 2008 16:54:31 +0100
Subject: [rsyslog] broken link to 4.1.1 release
In-Reply-To: <49304696.4000403@canonical.com>
References: <49304696.4000403@canonical.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA44F75A@grfint2.intern.adiscon.com>

Hi Peter,

thanks for letting me know. It is fixed now.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Peter Matulis
> Sent: Friday, November 28, 2008 8:29 PM
> To: rsyslog-users
> Subject: [rsyslog] broken link to 4.1.1 release
> 
> The download link to the latest devel release (4.1.1) is broken:
> 
> 
> http://www.rsyslog.com/Downloads-req-getit-lid-140.phtml
> 
> 
> Peter
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com