From mbiebl at gmail.com  Sat Dec  1 00:14:35 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Sat, 1 Dec 2007 00:14:35 +0100
Subject: [rsyslog] gss-api support for rsyslog
In-Reply-To: <473B2961.9010705@redhat.com>
References: <473B2961.9010705@redhat.com>
Message-ID: <aa76cdac0711301514x711721a7i2767e8a99e175f79@mail.gmail.com>

2007/11/14, theinric at redhat.com <theinric at redhat.com>:
> Hi,
>
> here is a patch that enables rsyslog to use gss-api for authentication
> and encryption.

Hi,

ist seems not all of the patch has made it into CVS. The part in
Makefile.am is missing, so the compilation fails. Please apply the
(inlined) patch ;-)

Index: Makefile.am
===================================================================
RCS file: /cvsroot/rsyslog/rsyslog/Makefile.am,v
retrieving revision 1.36
diff -u -3 -p -r1.36 Makefile.am
--- Makefile.am 30 Nov 2007 13:43:12 -0000      1.36
+++ Makefile.am 30 Nov 2007 23:10:27 -0000
@@ -63,7 +63,7 @@ rsyslogd_SOURCES = \
        gss-misc.h

 rsyslogd_CPPFLAGS = -D_PATH_MODDIR=\"$(pkglibdir)/\" $(pthreads_cflags)
-rsyslogd_LDADD = $(zlib_libs) $(pthreads_libs) -ldl
+rsyslogd_LDADD = $(zlib_libs) $(pthreads_libs) $(gss_libs) -ldl
 rsyslogd_LDFLAGS = -export-dynamic

 man_MANS = rfc3195d.8 rklogd.8 rsyslogd.8 rsyslog.conf.5



Cheers,
Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From sur5r at sur5r.net  Sat Dec  1 22:36:44 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Sat, 1 Dec 2007 22:36:44 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
Message-ID: <20071201223644.036dccd6@marathon.haufe>

Hi all!

As already mentioned in the forum: I've created an output module for
PostgreSQL.

The attached patch does the following:

* add plugins/ompgsql/*
* extend configure.ac and Makefile.am
* msg.h
** struct msg_t: add pszTIMESTAMP_PgSQL, add pszRcvdAt_PgSQL, fix typo
* msg.c
** getTimeGenerated(): add handling of tplFmtPgSQLDate
** getTimeReported(): add handling of tplFmtPgSQLDate
* syslogd.h:
** add definition of formatTimestampToPgSQL()
* syslogd.c:
** add StdPgSQLFmt
** mainThread(): include StdPgSQLFmt
* template.h:
** add tplFmtPgSQLDate to tplFormatTypes
* template.c:
** doOptions(): date-pgsql -> tplFmtPgSQLDate
** tplPrintList(): add tplFmtPgSQLDate to switch()

Please note that the patch has not yet been tested thoroughly, so until
that has been done, it is to be considered alpha quality.

If anyone wants to try it: any feedback is appreciated.

Regards,
sur5r

P.S.: I hope the patch survives mailman... ;)
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: rsyslog_ompgsql.diff
Url: http://lists.adiscon.net/pipermail/rsyslog/attachments/20071201/3543bb53/attachment.txt 

From sur5r at sur5r.net  Sat Dec  1 23:01:26 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Sat, 1 Dec 2007 23:01:26 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
References: <20071201223644.036dccd6@marathon.haufe>
Message-ID: <20071201230126.3537f120@marathon.haufe>

Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 22:36:44 +0100
in 20071201223644.036dccd6 at marathon.haufe:

> The attached patch does the following:
> 
> * add plugins/ompgsql/*

Erm, yes. Maybe i should have noticed, that cvs doesn't recurse into
new subdirectories... so here are the files for the plugins/ompgsql
directory.

From sur5r at sur5r.net  Sat Dec  1 23:10:11 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Sat, 1 Dec 2007 23:10:11 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
References: <20071201223644.036dccd6@marathon.haufe>
	<20071201230126.3537f120@marathon.haufe>
Message-ID: <20071201231011.5a27a361@marathon.haufe>

Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 23:01:26 +0100
in 20071201230126.3537f120 at marathon.haufe:

> Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 22:36:44 +0100
> in 20071201223644.036dccd6 at marathon.haufe:
> 
> > The attached patch does the following:
> > 
> > * add plugins/ompgsql/*
> 
> Erm, yes. Maybe i should have noticed, that cvs doesn't recurse into
> new subdirectories... so here are the files for the plugins/ompgsql
> directory.
Ok, mailman doesn't like any mimetypes but text/plain... next try *grml*
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Makefile.am
Url: http://lists.adiscon.net/pipermail/rsyslog/attachments/20071201/0f6fc89b/attachment.txt 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: createDB.sql
Url: http://lists.adiscon.net/pipermail/rsyslog/attachments/20071201/0f6fc89b/attachment-0001.txt 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ompgsql.c
Url: http://lists.adiscon.net/pipermail/rsyslog/attachments/20071201/0f6fc89b/attachment-0002.txt 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ompgsql.h
Url: http://lists.adiscon.net/pipermail/rsyslog/attachments/20071201/0f6fc89b/attachment-0003.txt 

From mbiebl at gmail.com  Sun Dec  2 23:10:07 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 2 Dec 2007 23:10:07 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
In-Reply-To: <20071201231011.5a27a361@marathon.haufe>
References: <20071201223644.036dccd6@marathon.haufe>
	<20071201230126.3537f120@marathon.haufe>
	<20071201231011.5a27a361@marathon.haufe>
Message-ID: <aa76cdac0712021410l2a218cftb5a6883c38bacd32@mail.gmail.com>

2007/12/1, Jakob Haufe <sur5r at sur5r.net>:
> Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 23:01:26 +0100
> in 20071201230126.3537f120 at marathon.haufe:
>
> > Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 22:36:44 +0100
> > in 20071201223644.036dccd6 at marathon.haufe:
> >
> > > The attached patch does the following:
> > >
> > > * add plugins/ompgsql/*
> >
> > Erm, yes. Maybe i should have noticed, that cvs doesn't recurse into
> > new subdirectories... so here are the files for the plugins/ompgsql
> > directory.
> Ok, mailman doesn't like any mimetypes but text/plain... next try *grml*

Hi,

just a very minor nitpick in Makefile.am:
pkglibdir is by default defined to be $(libdir)/@PACKAGE@/ [1], so
there is no need to redefine it. Simply delete the first line.
plugins/ommysql/Makefile.am should be fixed accordingly.

It also seems that the configure.ac bits are missing.

Cheers,
Michael

[1] http://www.gnu.org/software/automake/manual/html_node/Uniform.html#Uniform
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Mon Dec  3 08:10:44 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 3 Dec 2007 08:10:44 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
In-Reply-To: <aa76cdac0712021410l2a218cftb5a6883c38bacd32@mail.gmail.com>
References: <20071201223644.036dccd6@marathon.haufe><20071201230126.3537f120@marathon.haufe><20071201231011.5a27a361@marathon.haufe>
	<aa76cdac0712021410l2a218cftb5a6883c38bacd32@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30856B@grfint2.intern.adiscon.com>

Hi,

thanks to all for the great contributions! Let me just quickly outline
my plan, so that nobody is disappointed ;) I'll publish a new 1.19.12
release today. However, that will just hold some cleanup and fixes. When
I have done that, I'll begin to merge the patches into the
already-modified 1.20.0. I am not yet sure how much issues I will see
because there are a number of code changes in that release. I am also
not sure yet in which order I apply the patches, because one might be
better than the other. I'll announce when I have merged and will
probably post a test tarball here on the list before officially
releasing. My target for release is mid to end of week.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, December 02, 2007 11:10 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] [patch] output module for PostgreSQL
> 
> 2007/12/1, Jakob Haufe <sur5r at sur5r.net>:
> > Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 23:01:26
+0100
> > in 20071201230126.3537f120 at marathon.haufe:
> >
> > > Jakob Haufe <sur5r at sur5r.net> wrote on Sat, 1 Dec 2007 22:36:44
> +0100
> > > in 20071201223644.036dccd6 at marathon.haufe:
> > >
> > > > The attached patch does the following:
> > > >
> > > > * add plugins/ompgsql/*
> > >
> > > Erm, yes. Maybe i should have noticed, that cvs doesn't recurse
> into
> > > new subdirectories... so here are the files for the
plugins/ompgsql
> > > directory.
> > Ok, mailman doesn't like any mimetypes but text/plain... next try
> *grml*
> 
> Hi,
> 
> just a very minor nitpick in Makefile.am:
> pkglibdir is by default defined to be $(libdir)/@PACKAGE@/ [1], so
> there is no need to redefine it. Simply delete the first line.
> plugins/ommysql/Makefile.am should be fixed accordingly.
> 
> It also seems that the configure.ac bits are missing.
> 
> Cheers,
> Michael
> 
> [1]
>
http://www.gnu.org/software/automake/manual/html_node/Uniform.html#Unif
> orm
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Mon Dec  3 10:01:24 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 3 Dec 2007 10:01:24 +0100
Subject: [rsyslog] rsyslog 1.19.12 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308570@grfint2.intern.adiscon.com>

As promised this morning, I have just released 1.19.12. It is a cleanup
release, no need to install it if you do not experience any issues.
Users of 1.19.11 using mysql should update, as the new -pthread switch
was not applied to ommysql in 1.19.11.

Download:

   http://www.rsyslog.com/Downloads-req-getit-lid-64.phtml

As always, feedback is appreciated.

Best regards,
Rainer Gerhards

Rsyslog homepage: http://www.rsyslog.com
My blog: http://rgerhards.blogspot.com


From sur5r at sur5r.net  Mon Dec  3 10:37:48 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Mon, 3 Dec 2007 10:37:48 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
References: <20071201223644.036dccd6@marathon.haufe>
	<20071201230126.3537f120@marathon.haufe>
	<20071201231011.5a27a361@marathon.haufe>
	<aa76cdac0712021410l2a218cftb5a6883c38bacd32@mail.gmail.com>
Message-ID: <20071203103748.781bfba2@mp-atlantis3.informatik.uni-mannheim.de>

On Sun, 2 Dec 2007 23:10:07 +0100
"Michael Biebl" <mbiebl at gmail.com> wrote:

> just a very minor nitpick in Makefile.am:
> pkglibdir is by default defined to be $(libdir)/@PACKAGE@/ [1], so
> there is no need to redefine it. Simply delete the first line.
> plugins/ommysql/Makefile.am should be fixed accordingly.

Thanks for pointing this out. Unfortunately, I have no in-depth
knowledge of autotools, so I'm thankful for any suggestions.

> It also seems that the configure.ac bits are missing.

They are there, though hard to find. 

MSGID 20071201223644.036dccd6 at marathon.haufe
contains the patch modifying existing files, namely:
msg.[ch], syslogd.[ch], template.[ch], configure.ac, Makefile.am

MSGID 20071201231011.5a27a361 at marathon.haufe
contains the files for plugins/ompgsql/*

Regards,
Jakob


From sur5r at sur5r.net  Mon Dec  3 11:09:12 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Mon, 3 Dec 2007 11:09:12 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
References: <20071201223644.036dccd6@marathon.haufe>
	<20071201230126.3537f120@marathon.haufe>
	<20071201231011.5a27a361@marathon.haufe>
	<aa76cdac0712021410l2a218cftb5a6883c38bacd32@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA30856B@grfint2.intern.adiscon.com>
Message-ID: <20071203110912.50657ea6@mp-atlantis3.informatik.uni-mannheim.de>

On Mon, 3 Dec 2007 08:10:44 +0100
"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:

> thanks to all for the great contributions! Let me just quickly outline
> my plan, so that nobody is disappointed ;) I'll publish a new 1.19.12
> release today. However, that will just hold some cleanup and fixes.
> When I have done that, I'll begin to merge the patches into the
> already-modified 1.20.0. I am not yet sure how much issues I will see
> because there are a number of code changes in that release. I am also
> not sure yet in which order I apply the patches, because one might be
> better than the other. I'll announce when I have merged and will
> probably post a test tarball here on the list before officially
> releasing. My target for release is mid to end of week.

That's perfectly ok for me. But I definitely would appreciate any
feedback on ompgsql before officially releasing it. I do not expect any
problems with it but you never know...

So if anybody is going to test it, please give feedback.

Regards,
sur5r


From rgerhards at hq.adiscon.com  Mon Dec  3 11:11:25 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 3 Dec 2007 11:11:25 +0100
Subject: [rsyslog] [patch] output module for PostgreSQL
In-Reply-To: <20071203110912.50657ea6@mp-atlantis3.informatik.uni-mannheim.de>
References: <20071201223644.036dccd6@marathon.haufe><20071201230126.3537f120@marathon.haufe><20071201231011.5a27a361@marathon.haufe><aa76cdac0712021410l2a218cftb5a6883c38bacd32@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30856B@grfint2.intern.adiscon.com>
	<20071203110912.50657ea6@mp-atlantis3.informatik.uni-mannheim.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308574@grfint2.intern.adiscon.com>

Just a quick update. I am right now integrating it and will review code
during this and once I am through. As I suspected, I need to do at least
some manual merging, as the source tree already changed and in some
places considerably (talking about 1.20.0!). So it would be good if you
could-re-test it once I can release an early test tarball.

I keep you updated.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Monday, December 03, 2007 11:09 AM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] [patch] output module for PostgreSQL
> 
> On Mon, 3 Dec 2007 08:10:44 +0100
> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:
> 
> > thanks to all for the great contributions! Let me just quickly
> outline
> > my plan, so that nobody is disappointed ;) I'll publish a new
1.19.12
> > release today. However, that will just hold some cleanup and fixes.
> > When I have done that, I'll begin to merge the patches into the
> > already-modified 1.20.0. I am not yet sure how much issues I will
see
> > because there are a number of code changes in that release. I am
also
> > not sure yet in which order I apply the patches, because one might
be
> > better than the other. I'll announce when I have merged and will
> > probably post a test tarball here on the list before officially
> > releasing. My target for release is mid to end of week.
> 
> That's perfectly ok for me. But I definitely would appreciate any
> feedback on ompgsql before officially releasing it. I do not expect
any
> problems with it but you never know...
> 
> So if anybody is going to test it, please give feedback.
> 
> Regards,
> sur5r
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From mbiebl at gmail.com  Tue Dec  4 03:58:21 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 4 Dec 2007 03:58:21 +0100
Subject: [rsyslog] [PATCH] Fix latest "NetBSD fix"
Message-ID: <aa76cdac0712031858w52c6c63bq529b7496cb8a5cf3@mail.gmail.com>

Hi,

the latest NetBSD related commits (configure.ac/Makefile.am) break the
compilation on Linux. Variable assignments mustn't have whitespace,
otherwise have_libdl and as a result HAVE_LIBDL is not defined.

To fix the issue though, I'd simply recommend to remove the hard coded
-ldl from Makefile.am instead of the proposed solution.
The AC_CHECK_LIB macro [2] already appends -ldl to LIBS, depending on
the outcome of the check [2]. The global LIBS variable (similar to
CFLAGS e.g.) is used for linking all binaries.

Linked (see, I learned my lesson :-p) is the patch [1] to address this.

The only downside of AC_CHECK_LIB is, that it assigns -ldl (and -lrt)
to the global LIBS variable. This means that all binaries are linked
with -ldl -lrt, as mentioned above, even if they don't need them (i.e.
ommysql/ompsql plugin,  rklogd rfc3195d).

I can prepare a patch to address this issue correctly, if wanted.

Cheers,
Michael

[1] http://debs.michaelbiebl.de/rsyslog/fix_dl_compilation.patch
[2] http://www.gnu.org/software/automake/manual/autoconf/Libraries.html

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Tue Dec  4 11:15:57 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 04 Dec 2007 11:15:57 +0100
Subject: [rsyslog] [PATCH] Fix latest "NetBSD fix"
In-Reply-To: <aa76cdac0712031858w52c6c63bq529b7496cb8a5cf3@mail.gmail.com>
References: <aa76cdac0712031858w52c6c63bq529b7496cb8a5cf3@mail.gmail.com>
Message-ID: <1196763357.12493.6.camel@localhost.localdomain>

Hi Michael,

many thanks for the information. Actually, you were quicker than me ;) I
worked with Bartosz on the issue yesterday, but needed to leave before I
could finalize it.

I would appreciate if you could provide the "big patch" you wrote about.
That will ensure that autotools is used in the best possible way :)

Rainer

On Tue, 2007-12-04 at 03:58 +0100, Michael Biebl wrote:
> Hi,
> 
> the latest NetBSD related commits (configure.ac/Makefile.am) break the
> compilation on Linux. Variable assignments mustn't have whitespace,
> otherwise have_libdl and as a result HAVE_LIBDL is not defined.
> 
> To fix the issue though, I'd simply recommend to remove the hard coded
> -ldl from Makefile.am instead of the proposed solution.
> The AC_CHECK_LIB macro [2] already appends -ldl to LIBS, depending on
> the outcome of the check [2]. The global LIBS variable (similar to
> CFLAGS e.g.) is used for linking all binaries.
> 
> Linked (see, I learned my lesson :-p) is the patch [1] to address this.
> 
> The only downside of AC_CHECK_LIB is, that it assigns -ldl (and -lrt)
> to the global LIBS variable. This means that all binaries are linked
> with -ldl -lrt, as mentioned above, even if they don't need them (i.e.
> ommysql/ompsql plugin,  rklogd rfc3195d).
> 
> I can prepare a patch to address this issue correctly, if wanted.
> 
> Cheers,
> Michael
> 
> [1] http://debs.michaelbiebl.de/rsyslog/fix_dl_compilation.patch
> [2] http://www.gnu.org/software/automake/manual/autoconf/Libraries.html
> 

From rgerhards at hq.adiscon.com  Tue Dec  4 11:23:26 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 4 Dec 2007 11:23:26 +0100
Subject: [rsyslog] rsyslog 1.20.0 preview available
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308596@grfint2.intern.adiscon.com>

Hi all,

there is a preview of rsyslog 1.20.0 available at

http://download.rsyslog.com/rsyslog/rsyslog-1.20.0.tar.gz

It obviously does not contains Michael Biebl's upcoming autotools patch,
but should be fairly well. I would be most interested in feedback from
the field, especially on the new postgres module by sur5r (thanks
again!). I plan to release end of week if now show stoppers show up.

Please note that I have disabled GSS-API in a rough way: you can simply
not select it in configure. This is because GSS-API support does need
some more work or it could become an interop problem in the future. So
we decided to push it back until that is solved.

Feedback is highly appreciated. If I updated the tarball, I'll announce
that. Oh, yes, doc is not finished yet, so do not wonder...

Rainer

From sur5r at sur5r.net  Tue Dec  4 14:00:14 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Tue, 4 Dec 2007 14:00:14 +0100
Subject: [rsyslog] rsyslog 1.20.0 preview available
References: <577465F99B41C842AAFBE9ED71E70ABA308596@grfint2.intern.adiscon.com>
Message-ID: <20071204140014.7c2187b2@mp-atlantis3.informatik.uni-mannheim.de>

On Tue, 4 Dec 2007 11:23:26 +0100
"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:

> Feedback is highly appreciated. If I updated the tarball, I'll
> announce that. Oh, yes, doc is not finished yet, so do not wonder...

Just a small thing I already mentioned on the forum but unfortunately
got overlooked:

ommysql, line 236: p+=sizeof(":ommysql:");

This adds increments the pointer one byte too far (trailing \0) and
thus eats the first byte of the hostname part.

It's a bit unfortunate that strlen() doesn't count the \0 while
sizeof() does...

Regards,
sur5r


From rgerhards at hq.adiscon.com  Tue Dec  4 14:10:06 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 4 Dec 2007 14:10:06 +0100
Subject: [rsyslog] rsyslog 1.20.0 preview available
In-Reply-To: <20071204140014.7c2187b2@mp-atlantis3.informatik.uni-mannheim.de>
References: <577465F99B41C842AAFBE9ED71E70ABA308596@grfint2.intern.adiscon.com>
	<20071204140014.7c2187b2@mp-atlantis3.informatik.uni-mannheim.de>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30859F@grfint2.intern.adiscon.com>

Indeed, thanks for pointing it out. Its fixed now, in cvs and I also
re-released the tarball (just for completeness).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Tuesday, December 04, 2007 2:00 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslog 1.20.0 preview available
> 
> On Tue, 4 Dec 2007 11:23:26 +0100
> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:
> 
> > Feedback is highly appreciated. If I updated the tarball, I'll
> > announce that. Oh, yes, doc is not finished yet, so do not wonder...
> 
> Just a small thing I already mentioned on the forum but unfortunately
> got overlooked:
> 
> ommysql, line 236: p+=sizeof(":ommysql:");
> 
> This adds increments the pointer one byte too far (trailing \0) and
> thus eats the first byte of the hostname part.
> 
> It's a bit unfortunate that strlen() doesn't count the \0 while
> sizeof() does...
> 
> Regards,
> sur5r
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From stephen.carville at gmail.com  Fri Dec  7 22:09:18 2007
From: stephen.carville at gmail.com (Stephen Carville)
Date: Fri, 7 Dec 2007 13:09:18 -0800
Subject: [rsyslog] Last Message repeated N times
Message-ID: <2428c0380712071309w31f4fc20qb933dfef47f445ac@mail.gmail.com>

I am using rsyslog 1.19.10 on a central server to collect messages and
store them in a mysql database.  One thing I've noticed is that the
"last message repeated NNN times" entries are alway entered with
FromHost="last", SysLogTag="message", and Message="repeated NNN
times".  Can this be fixed?

I still use the regular sysklogd package on the remote machines to
forward the messages.

-- 
Stephen Carville

From rgerhards at hq.adiscon.com  Tue Dec 11 11:06:00 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Dec 2007 11:06:00 +0100
Subject: [rsyslog] Asking for help...
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30862F@grfint2.intern.adiscon.com>

Hi folks,

I have a design question with upcoming rsyslog changes and could not
even find an appropriate place to ask. I have blogged about it, so the
information is available to everyone:

http://rgerhards.blogspot.com/2007/12/design-problem.html

If you happen to know a good place to ask (or even know a solution), I
would really appreciate to hear about that!

Thanks
Rainer

From rgerhards at hq.adiscon.com  Tue Dec 11 12:01:07 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Dec 2007 12:01:07 +0100
Subject: [rsyslog] Last Message repeated N times
In-Reply-To: <2428c0380712071309w31f4fc20qb933dfef47f445ac@mail.gmail.com>
References: <2428c0380712071309w31f4fc20qb933dfef47f445ac@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308637@grfint2.intern.adiscon.com>

Hi Stephen,

I was totally swamped the past days, sorry for the missing response.
I'll check what causes this. As a side-note, I think that the message
reduction code does not work well with the database - as you can no
longer query the individual messages. Is this your intent?

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Stephen Carville
> Sent: Friday, December 07, 2007 10:09 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Last Message repeated N times
> 
> I am using rsyslog 1.19.10 on a central server to collect messages and
> store them in a mysql database.  One thing I've noticed is that the
> "last message repeated NNN times" entries are alway entered with
> FromHost="last", SysLogTag="message", and Message="repeated NNN
> times".  Can this be fixed?
> 
> I still use the regular sysklogd package on the remote machines to
> forward the messages.
> 
> --
> Stephen Carville
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Tue Dec 11 12:26:32 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Dec 2007 12:26:32 +0100
Subject: [rsyslog] Last Message repeated N times
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308637@grfint2.intern.adiscon.com>
References: <2428c0380712071309w31f4fc20qb933dfef47f445ac@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA308637@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30863D@grfint2.intern.adiscon.com>

I think I misunderstood your problem... The "last message repeated n
times" stems back from the reporting sysklogd's - right? Mhhh... that's
a problem. I need to do some hardcoded processing in the parser to cover
part of it. The problem is that sysklogd emits a totally non-standard
message in that case. Even worse, I think it does not even contain a
clue at the missing information. I could probably use the sender's IP
address as source in  that case, but the tag seems to be impossible to
obtain.

Can you post me a copy of the %rawmsg% property, so that I can have a
look at what exactly you are seing?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, December 11, 2007 12:01 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Last Message repeated N times
> 
> Hi Stephen,
> 
> I was totally swamped the past days, sorry for the missing response.
> I'll check what causes this. As a side-note, I think that the message
> reduction code does not work well with the database - as you can no
> longer query the individual messages. Is this your intent?
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Stephen Carville
> > Sent: Friday, December 07, 2007 10:09 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Last Message repeated N times
> >
> > I am using rsyslog 1.19.10 on a central server to collect messages
> and
> > store them in a mysql database.  One thing I've noticed is that the
> > "last message repeated NNN times" entries are alway entered with
> > FromHost="last", SysLogTag="message", and Message="repeated NNN
> > times".  Can this be fixed?
> >
> > I still use the regular sysklogd package on the remote machines to
> > forward the messages.
> >
> > --
> > Stephen Carville
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From sur5r at sur5r.net  Tue Dec 11 14:48:15 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Tue, 11 Dec 2007 14:48:15 +0100
Subject: [rsyslog] Asking for help...
References: <577465F99B41C842AAFBE9ED71E70ABA30862F@grfint2.intern.adiscon.com>
Message-ID: <20071211144815.6a2ef123@marathon.haufe>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote on Tue, 11 Dec 2007
11:06:00 +0100:

> http://rgerhards.blogspot.com/2007/12/design-problem.html

Maybe I'm a bit short-sighted here, but isn't this exactly the situation
for non-blocking sockets? Or will that raise other problems?

Regards,
sur5r
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHXpUj1YAhDic+adYRAnBZAKCWNXoWIKHhnhm/Pe9CMi1MT5MOWQCeJknc
AvFjtLOkeHTNz/mdpz3Gy2k=
=YZgt
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Tue Dec 11 14:55:21 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 11 Dec 2007 14:55:21 +0100
Subject: [rsyslog] Asking for help...
In-Reply-To: <20071211144815.6a2ef123@marathon.haufe>
References: <577465F99B41C842AAFBE9ED71E70ABA30862F@grfint2.intern.adiscon.com>
	<20071211144815.6a2ef123@marathon.haufe>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308646@grfint2.intern.adiscon.com>

Thanks for the feedback - is it OK if we discuss this in the forum? I
think not everyone here on the list is interested in that discussion...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Tuesday, December 11, 2007 2:48 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] Asking for help...
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote on Tue, 11 Dec 2007
> 11:06:00 +0100:
> 
> > http://rgerhards.blogspot.com/2007/12/design-problem.html
> 
> Maybe I'm a bit short-sighted here, but isn't this exactly the
> situation
> for non-blocking sockets? Or will that raise other problems?
> 
> Regards,
> sur5r
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> 
> iD8DBQFHXpUj1YAhDic+adYRAnBZAKCWNXoWIKHhnhm/Pe9CMi1MT5MOWQCeJknc
> AvFjtLOkeHTNz/mdpz3Gy2k=
> =YZgt
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Wed Dec 12 17:06:48 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Dec 2007 17:06:48 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>

Hi all,

I have just released rsyslog 1.20.1. It primarily contains bug fixes: a
memory leak in regard to PostgreSQL date formatting and a potential race
condition within repeated message processing have been fixed. Also,
there are new runtime options to harden ACL processing when DNS is not
ready while rsyslogd start up. There are also some minor changes and
clean-ups. Version 1.20.1 is recommended to anyone using previous
releases. Users of PostgreSQL are urged to update to this version.

Download:
   http://www.rsyslog.com/Downloads-req-getit-lid-66.phtml

Changelog:
   http://www.rsyslog.com/Article147.phtml

You may want to watch my blog for upcoming developments:
http://rgerhards.blogspot.com. There was recently a lot of activity and
more is expected.

As always, feedback is appreciated.
Rainer Gerhards

From stephen.carville at gmail.com  Wed Dec 12 18:38:15 2007
From: stephen.carville at gmail.com (Stephen Carville)
Date: Wed, 12 Dec 2007 09:38:15 -0800
Subject: [rsyslog] Last Message repeated N times
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308637@grfint2.intern.adiscon.com>
References: <2428c0380712071309w31f4fc20qb933dfef47f445ac@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA308637@grfint2.intern.adiscon.com>
Message-ID: <2428c0380712120938x72f55b9na4aea606374f0255@mail.gmail.com>

On Dec 11, 2007 3:01 AM, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
> Hi Stephen,
>
> I was totally swamped the past days, sorry for the missing response.
> I'll check what causes this. As a side-note, I think that the message
> reduction code does not work well with the database - as you can no
> longer query the individual messages. Is this your intent?

My original intent was to have a centralized copy of all the changes
that cfengine makes.  That works extremely well and saves me a
buttload of time over having to log by hand in a shared Excel
spreadsheet every time I update a hosts file or tweak httpd.conf or
tomcat's server.xml.  That and logging the messages from auditd were
the main goals.  I added /var/log/messages and /var/log/secure mainly
because I could and because it gives the Linux side of the shop
greater credibility with the SOX (expletive deleted).  Now that
messages and secure are in the database I'll have to run a log
analyzer periodically looking for indications of "stuff" that's not
right but that comes after about six other projects.

This isn't a show stopper and I could probably just drop the "last
messages repeated.." entries without any harm.  Tho I;m not sure how
jsut yet.  Out of 2.4M rows about 385K are of this type.

> Rainer
>
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Stephen Carville
> > Sent: Friday, December 07, 2007 10:09 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: [rsyslog] Last Message repeated N times
> >
> > I am using rsyslog 1.19.10 on a central server to collect messages and
> > store them in a mysql database.  One thing I've noticed is that the
> > "last message repeated NNN times" entries are alway entered with
> > FromHost="last", SysLogTag="message", and Message="repeated NNN
> > times".  Can this be fixed?
> >
> > I still use the regular sysklogd package on the remote machines to
> > forward the messages.
> >
> > --
> > Stephen Carville
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>

-- 
Stephen Carville

From stephen.carville at gmail.com  Wed Dec 12 19:02:09 2007
From: stephen.carville at gmail.com (Stephen Carville)
Date: Wed, 12 Dec 2007 10:02:09 -0800
Subject: [rsyslog] Last Message repeated N times
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30863D@grfint2.intern.adiscon.com>
References: <2428c0380712071309w31f4fc20qb933dfef47f445ac@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA308637@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA30863D@grfint2.intern.adiscon.com>
Message-ID: <2428c0380712121002l70f37260p869ab35b8acb7068@mail.gmail.com>

On Dec 11, 2007 3:26 AM, Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:
> I think I misunderstood your problem... The "last message repeated n
> times" stems back from the reporting sysklogd's - right?

Would ugrading to rsyslogd on the client machines help?  I'm getting
ready for an upgrade cycle where everything except a couple of boxes
running old hardware not suppoert by the 2.6 kernel will be either
Oracle ULN 4.X or Redhat/CentOS 5.X  I didn't see a spec file in the
tar.gz but I can write one myself if there is no good one available.

> Mhhh... that's
> a problem. I need to do some hardcoded processing in the parser to cover
> part of it. The problem is that sysklogd emits a totally non-standard
> message in that case. Even worse, I think it does not even contain a
> clue at the missing information. I could probably use the sender's IP
> address as source in  that case, but the tag seems to be impossible to
> obtain.
>
> Can you post me a copy of the %rawmsg% property, so that I can have a
> look at what exactly you are seing?
>
> Rainer
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Tuesday, December 11, 2007 12:01 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] Last Message repeated N times
> >
> > Hi Stephen,
> >
> > I was totally swamped the past days, sorry for the missing response.
> > I'll check what causes this. As a side-note, I think that the message
> > reduction code does not work well with the database - as you can no
> > longer query the individual messages. Is this your intent?
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Stephen Carville
> > > Sent: Friday, December 07, 2007 10:09 PM
> > > To: rsyslog at lists.adiscon.com
> > > Subject: [rsyslog] Last Message repeated N times
> > >
> > > I am using rsyslog 1.19.10 on a central server to collect messages
> > and
> > > store them in a mysql database.  One thing I've noticed is that the
> > > "last message repeated NNN times" entries are alway entered with
> > > FromHost="last", SysLogTag="message", and Message="repeated NNN
> > > times".  Can this be fixed?
> > >
> > > I still use the regular sysklogd package on the remote machines to
> > > forward the messages.
> > >
> > > --
> > > Stephen Carville
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>



-- 
Stephen Carville

From mbiebl at gmail.com  Wed Dec 12 19:11:55 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 12 Dec 2007 19:11:55 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>

2007/12/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> I have just released rsyslog 1.20.1. It primarily contains bug fixes: a

1.20.1 unfortunately fails to compile (unless you use --enable-debug):
gcc -g -O2 -W -Wall -Wformat-security -Wshadow -Wcast-align
-Wpointer-arith -Wmissing-format-attribute -g -o rsyslogd
rsyslogd-syslogd.o rsyslogd-pidfile.o rsyslogd-template.o
rsyslogd-outchannel.o rsyslogd-stringbuf.o rsyslogd-srUtils.o
rsyslogd-parse.o rsyslogd-sync.o rsyslogd-net.o rsyslogd-msg.o
rsyslogd-expr.o rsyslogd-omshell.o rsyslogd-omusrmsg.o
rsyslogd-omfwd.o rsyslogd-tcpsyslog.o rsyslogd-omfile.o
rsyslogd-omdiscard.o rsyslogd-modules.o rsyslogd-objomsr.o
rsyslogd-cfsysline.o rsyslogd-linkedlist.o rsyslogd-iminternal.o
rsyslogd-action.o rsyslogd-gss-misc.o -Wl,--export-dynamic  -lz
-lpthread -ldl -lrt
rsyslogd-syslogd.o: In function `domarkActions':
/home/michael/cvs/rsyslog/build/../syslogd.c:3431: undefined reference
to `lockObj'
/home/michael/cvs/rsyslog/build/../syslogd.c:3439: undefined reference
to `unlockObj'
rsyslogd-syslogd.o: In function `callAction':
/home/michael/cvs/rsyslog/build/../syslogd.c:2442: undefined reference
to `lockObj'
/home/michael/cvs/rsyslog/build/../syslogd.c:2508: undefined reference
to `unlockObj'
/home/michael/cvs/rsyslog/build/../syslogd.c:2508: undefined reference
to `unlockObj'
collect2: ld returned 1 exit status
make[2]: *** [rsyslogd] Fehler 1
make[2]: Leaving directory `/home/michael/cvs/rsyslog/build'
make[1]: *** [all-recursive] Fehler 1
make[1]: Leaving directory `/home/michael/cvs/rsyslog/build'
make: *** [all] Fehler 2


sync.h declares lockObj/unlockObj, but sync.c only has these symbols
if compiled without NDEBUG.

Cheers,
Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From infofarmer at FreeBSD.org  Wed Dec 12 19:16:36 2007
From: infofarmer at FreeBSD.org (Andrew Pantyukhin)
Date: Wed, 12 Dec 2007 21:16:36 +0300
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
	<aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
Message-ID: <20071212181635.GQ51627@amilo.cenkes.org>

On Wed, Dec 12, 2007 at 07:11:55PM +0100, Michael Biebl wrote:
> 2007/12/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Hi all,
> >
> > I have just released rsyslog 1.20.1. It primarily contains bug fixes: a
> 
> sync.h declares lockObj/unlockObj, but sync.c only has these symbols
> if compiled without NDEBUG.

Try including config.h in sync.h

From mbiebl at gmail.com  Wed Dec 12 19:30:34 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 12 Dec 2007 19:30:34 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <20071212181635.GQ51627@amilo.cenkes.org>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
	<aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
	<20071212181635.GQ51627@amilo.cenkes.org>
Message-ID: <aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com>

2007/12/12, Andrew Pantyukhin <infofarmer at freebsd.org>:
> On Wed, Dec 12, 2007 at 07:11:55PM +0100, Michael Biebl wrote:
> > 2007/12/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Hi all,
> > >
> > > I have just released rsyslog 1.20.1. It primarily contains bug fixes: a
> >
> > sync.h declares lockObj/unlockObj, but sync.c only has these symbols
> > if compiled without NDEBUG.
>
> Try including config.h in sync.h
>

No difference.

The other header files (using NDEBUG) don't include config.h either btw.
The rely on the c file, which includes them, to include config.h.

Cheers,
Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From mbiebl at gmail.com  Wed Dec 12 20:18:38 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 12 Dec 2007 20:18:38 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
	<aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
	<20071212181635.GQ51627@amilo.cenkes.org>
	<aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com>
Message-ID: <aa76cdac0712121118kea18ee7md111ed3558bb7a87@mail.gmail.com>

2007/12/12, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/12, Andrew Pantyukhin <infofarmer at freebsd.org>:
> > On Wed, Dec 12, 2007 at 07:11:55PM +0100, Michael Biebl wrote:
> > > 2007/12/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > > Hi all,
> > > >
> > > > I have just released rsyslog 1.20.1. It primarily contains bug fixes: a
> > >
> > > sync.h declares lockObj/unlockObj, but sync.c only has these symbols
> > > if compiled without NDEBUG.
> >
> > Try including config.h in sync.h
> >
>

Found the culprit. Patch is attached.
Imho calling the define NDEBUG makes the code unnecessarily hard to
read (#ifndef NDEBUG ..). I had to stare at the code for a while until
I noticed that it was negated one time to often.

Cheers,
Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From sur5r at sur5r.net  Wed Dec 12 20:35:38 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Wed, 12 Dec 2007 20:35:38 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
	<aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
	<20071212181635.GQ51627@amilo.cenkes.org>
	<aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com>
	<aa76cdac0712121118kea18ee7md111ed3558bb7a87@mail.gmail.com>
Message-ID: <20071212203538.6dbb00d1@marathon.haufe>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

"Michael Biebl" <mbiebl at gmail.com> wrote on Wed, 12 Dec 2007 20:18:38
+0100:

> Found the culprit. Patch is attached.

Mailman ate it again. Try attaching it with mimetype text/plain, seems
to work then.

Regards,
sur5r
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHYDgQ1YAhDic+adYRAtxQAKCspmrE+HfH12YVBzJcIBQ/GqNKXACfdSl3
NkY9aWuB+ftzvx2t4MtKPHw=
=BODR
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Wed Dec 12 21:04:09 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Dec 2007 21:04:09 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <20071212203538.6dbb00d1@marathon.haufe>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com><aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com><20071212181635.GQ51627@amilo.cenkes.org><aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com><aa76cdac0712121118kea18ee7md111ed3558bb7a87@mail.gmail.com>
	<20071212203538.6dbb00d1@marathon.haufe>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308665@grfint2.intern.adiscon.com>

Hold for a moment, the actual patch is a bit different . more soon.

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Wednesday, December 12, 2007 8:36 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] rsyslog 1.20.1 released
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> "Michael Biebl" <mbiebl at gmail.com> wrote on Wed, 12 Dec 2007 20:18:38
> +0100:
> 
> > Found the culprit. Patch is attached.
> 
> Mailman ate it again. Try attaching it with mimetype text/plain, seems
> to work then.
> 
> Regards,
> sur5r
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> 
> iD8DBQFHYDgQ1YAhDic+adYRAtxQAKCspmrE+HfH12YVBzJcIBQ/GqNKXACfdSl3
> NkY9aWuB+ftzvx2t4MtKPHw=
> =BODR
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Wed Dec 12 21:07:26 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Dec 2007 21:07:26 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308665@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com><aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com><20071212181635.GQ51627@amilo.cenkes.org><aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com><aa76cdac0712121118kea18ee7md111ed3558bb7a87@mail.gmail.com><20071212203538.6dbb00d1@marathon.haufe>
	<577465F99B41C842AAFBE9ED71E70ABA308665@grfint2.intern.adiscon.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308666@grfint2.intern.adiscon.com>

Re-released, info follows soon...

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Wednesday, December 12, 2007 9:04 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 1.20.1 released
> 
> Hold for a moment, the actual patch is a bit different . more soon.
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> > Sent: Wednesday, December 12, 2007 8:36 PM
> > To: rsyslog at lists.adiscon.com
> > Subject: Re: [rsyslog] rsyslog 1.20.1 released
> >
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > "Michael Biebl" <mbiebl at gmail.com> wrote on Wed, 12 Dec 2007
20:18:38
> > +0100:
> >
> > > Found the culprit. Patch is attached.
> >
> > Mailman ate it again. Try attaching it with mimetype text/plain,
> seems
> > to work then.
> >
> > Regards,
> > sur5r
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.6 (GNU/Linux)
> >
> > iD8DBQFHYDgQ1YAhDic+adYRAtxQAKCspmrE+HfH12YVBzJcIBQ/GqNKXACfdSl3
> > NkY9aWuB+ftzvx2t4MtKPHw=
> > =BODR
> > -----END PGP SIGNATURE-----
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From mbiebl at gmail.com  Wed Dec 12 21:31:23 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 12 Dec 2007 21:31:23 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308665@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
	<aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
	<20071212181635.GQ51627@amilo.cenkes.org>
	<aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com>
	<aa76cdac0712121118kea18ee7md111ed3558bb7a87@mail.gmail.com>
	<20071212203538.6dbb00d1@marathon.haufe>
	<577465F99B41C842AAFBE9ED71E70ABA308665@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712121231i62631494p4ea39e41136adac3@mail.gmail.com>

2007/12/12, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hold for a moment, the actual patch is a bit different . more soon.

Yeah, got it the wrong way around. Didn't read the comment in sync.h
carefully enough.
Thanks for you sharp eye!

As I said, imho it's easier to read and grasp if the define is called DEBUG.

Cheers,
Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Wed Dec 12 21:40:25 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 12 Dec 2007 21:40:25 +0100
Subject: [rsyslog] rsyslog 1.20.1 released
In-Reply-To: <aa76cdac0712121130x4c96b883t4ee91e3362fb83d6@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308661@grfint2.intern.adiscon.com>
	<aa76cdac0712121011v566469bbx381513b6e4dc6943@mail.gmail.com>
	<20071212181635.GQ51627@amilo.cenkes.org>
	<aa76cdac0712121030y49b70fc0r6379af2b361a5f06@mail.gmail.com>
	<aa76cdac0712121118kea18ee7md111ed3558bb7a87@mail.gmail.com>
	<aa76cdac0712121130x4c96b883t4ee91e3362fb83d6@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308667@grfint2.intern.adiscon.com>

Hi all,

Man, when did I really screw up the last time? And I thought I had run
all test...

Anyhow, thanks all for helping to solve this issue.

Michael got it right, it is the NDEBUG. But is not the .c file but the
header file that was wrong. The idea is that we have functions only in
debugging mode, because functions make it easier to debug. But they are
slower than macro expansions. So in release mode, the functions are
replace with the mutex calls directly (which, I think, a macros, too).
If I recall it correctly, I screwed up after I created that
optimization.

> > Found the culprit. Patch is attached.
> > Imho calling the define NDEBUG makes the code unnecessarily hard to
> > read (#ifndef NDEBUG ..). I had to stare at the code for a 
> while until
> > I noticed that it was negated one time to often.

The NDEBUG is an assert() inheritance. Assert() uses it to indicate a
release build. I've build on the same tool to avoid introducing another
one. But obviously, it has its drawbacks, too. I think I'll solve that
by adding a comment.

I hope this clarifies. Drop me a note if I got it wrong again or you
have a question ;)

The tarball that is now in the repository should solve the issue. 

Rainer

From mbiebl at gmail.com  Wed Dec 12 21:47:32 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 12 Dec 2007 21:47:32 +0100
Subject: [rsyslog] Problems with IncludeConfig
Message-ID: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>

Hi everyone,

the current IncludeConfig implementation has some issues, especially
for distributers:
All files in the directory as specified by $IncludeConfig are included.
This means, temp files like *~ *.bak etc that are left behind by
editors, or stuff like *.rpmsave or *.dpkg-old/*.dpkg-new.
This is obviously unfortunate.

How shall we solve this?
1.) I patched the Debian package to only include files ending with
*.conf. If we document that properly this is imho an acceptable
solution.
2.) Implement an internal blacklist of endings that are ignored (see above)
3.) Instead of hardcoding the file ending to *.conf (as in 1.), use
the same syntax as e.g. apache, which allows to specify shell globs:
example:
Include /etc/apache2/conf.d/*.conf

What do you think?

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From r.bhatia at ipax.at  Thu Dec 13 09:55:49 2007
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Thu, 13 Dec 2007 09:55:49 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
Message-ID: <4760F395.4030502@ipax.at>

Michael Biebl wrote:
> the current IncludeConfig implementation has some issues, especially
> for distributers:
> All files in the directory as specified by $IncludeConfig are included.
> This means, temp files like *~ *.bak etc that are left behind by
> editors, or stuff like *.rpmsave or *.dpkg-old/*.dpkg-new.
> This is obviously unfortunate.
> 
> How shall we solve this?
> 1.) I patched the Debian package to only include files ending with
> *.conf. If we document that properly this is imho an acceptable
> solution.
> 2.) Implement an internal blacklist of endings that are ignored (see above)
> 3.) Instead of hardcoding the file ending to *.conf (as in 1.), use
> the same syntax as e.g. apache, which allows to specify shell globs:
> example:
> Include /etc/apache2/conf.d/*.conf

i would opt for the third option - thou i'm not sure how much work
this creates ;)

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________

From rgerhards at hq.adiscon.com  Thu Dec 13 13:04:33 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 13 Dec 2007 13:04:33 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <4760F395.4030502@ipax.at>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>

Lol... the initial message seems not to have passed my spam filter... ;)

I, too, tend to option three. The problem, though is
backwards-compatibility... Any suggestions and concerns are appreciated.

I'll look at the code to see how much work that creates. I think the
complexity is quite acceptable. I am right now in a threading redesign,
so I'd like to push this away for a little while... Anyhow, please keep
comments floating - I'll listen and drop in any thoughts I have.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Raoul Bhatia [IPAX]
> Sent: Thursday, December 13, 2007 9:56 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
> 
> Michael Biebl wrote:
> > the current IncludeConfig implementation has some issues, especially
> > for distributers:
> > All files in the directory as specified by $IncludeConfig are
> included.
> > This means, temp files like *~ *.bak etc that are left behind by
> > editors, or stuff like *.rpmsave or *.dpkg-old/*.dpkg-new.
> > This is obviously unfortunate.
> >
> > How shall we solve this?
> > 1.) I patched the Debian package to only include files ending with
> > *.conf. If we document that properly this is imho an acceptable
> > solution.
> > 2.) Implement an internal blacklist of endings that are ignored (see
> above)
> > 3.) Instead of hardcoding the file ending to *.conf (as in 1.), use
> > the same syntax as e.g. apache, which allows to specify shell globs:
> > example:
> > Include /etc/apache2/conf.d/*.conf
> 
> i would opt for the third option - thou i'm not sure how much work
> this creates ;)
> 
> cheers,
> raoul
> --
> ____________________________________________________________________
> DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
> Technischer Leiter
> 
> IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
> Barawitzkagasse 10/2/2/11           email.            office at ipax.at
> 1190 Wien                           tel.               +43 1 3670030
> FN 277995t HG Wien                  fax.            +43 1 3670030 15
> ____________________________________________________________________
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Fri Dec 14 09:29:19 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Dec 2007 09:29:19 +0100
Subject: [rsyslog] IMPORTANT: Version Changes
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308674@grfint2.intern.adiscon.com>

Hi all,

some information first here on the mailing list before I announce it. As
you know, I am looking at the way threading is supposed to work in
future releases and, most importantly, looking at the inputs (like mark
message generation).

Around summer, I wrote that I will probably need to release new major
versions when we go into multithreading redesign. It looks like we have
reached this stage. I tried to keep a single code base that still
supports both single- and multi-threaded operations. I have looked into
this the past days and I need to say that it creates a lot of complexity
and hard to understand code.

For this reason, I think it is finally time to branch the code based and
release some new versions.

Soon, I will create a branch for the current 1.20.1 code base. That will
only receive bug fixes, but no new development (except, I guess, GSSAPI
which I about to be contributed by Red Hat). When we are confident the
last changes worked well and introduced no new bugs, there will be a
version 2.0.0 stable release based on that code base.

CVS head, however, will then be rsyslog version 3. It will receive the
new input module interface. It requires pthreads, because there is no
way input modules and many more of the new desired features can be
implemented without them. Consequently, I will remove all
single-threading code from it, resulting in an easier to understand code
base. Please note that I expect this code to change dramatically when it
is being modified to be more modular (much like it was when I introduced
modular outputs in summer). Please note that I will apply any non-bugfix
patches to this code base, only.

I have somewhat bad feeling of going ahead with implementing a more
sophisticated and more parallel multi-threading while we still have an
issue with the segfault. However, I think by now we did everything
imaginable to capture that rare bug. I have come to the conclusion that
the best chance to find it is go ahead and implement the more
sophisticated design. That will lead to a review, and rewrite, of much
of the code in question, uncovering this we didn't think about before.
The recently discovered race condition is an excellent sample.

One thing about the license: rsyslog 2 will stay with "GPL v2 and above"
license, but rsyslog V3 will be licensed under "GPL v3 and above". I
already wrote about that change. It is my firm believe that GPL v3
brings benefit to our freedom to use digital goods. I am a strong oppose
of digital restrictions management (DRM) and software patens and I do
not like the idea that rsyslog benefits anyone who encourages these
things. I hope for your understanding.

I will set stage now for these changes and will do a web announcement
soon. Please don't be surprised that rsyslog v3 will be available before
v2, you now know the reason.

Rainer

From mbiebl at gmail.com  Fri Dec 14 14:36:03 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 14 Dec 2007 14:36:03 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>

2007/12/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Lol... the initial message seems not to have passed my spam filter... ;)
>
> I, too, tend to option three. The problem, though is

Yeah, it's the most flexible one.

> backwards-compatibility... Any suggestions and concerns are appreciated.

This feature has been pretty new, and as such probably isn't widely used.
I would simply emphasize this change in the release notes and the Changelog.
The worst that can happen is, that a file is not included anymore.

>
> I'll look at the code to see how much work that creates. I think the
> complexity is quite acceptable. I am right now in a threading redesign,
> so I'd like to push this away for a little while... Anyhow, please keep
> comments floating - I'll listen and drop in any thoughts I have.

Should be fairly straightforward to implement using
http://www.gnu.org/software/libc/manual/html_node/Globbing.html#Globbing

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From mbiebl at gmail.com  Fri Dec 14 17:24:02 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 14 Dec 2007 17:24:02 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
	<aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
Message-ID: <aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>

2007/12/14, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Lol... the initial message seems not to have passed my spam filter... ;)
> >
> > I, too, tend to option three. The problem, though is
>
> Yeah, it's the most flexible one.
>
> > backwards-compatibility... Any suggestions and concerns are appreciated.
>
> This feature has been pretty new, and as such probably isn't widely used.
> I would simply emphasize this change in the release notes and the Changelog.
> The worst that can happen is, that a file is not included anymore.
>
> >
> > I'll look at the code to see how much work that creates. I think the
> > complexity is quite acceptable. I am right now in a threading redesign,
> > so I'd like to push this away for a little while... Anyhow, please keep
> > comments floating - I'll listen and drop in any thoughts I have.
>
> Should be fairly straightforward to implement using
> http://www.gnu.org/software/libc/manual/html_node/Globbing.html#Globbing
>

Small example using glob:

#include <glob.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char **argv)
{
        glob_t files;
        int ret = 0;
        int i = 0;
        char *path;

        if (argc > 1) {
                path = argv[1];
        } else {
                path = strdup("/etc/rsyslog.d/*.conf");
        }

        ret = glob(path, GLOB_ERR, NULL, &files);
        printf("All files matching %s:\n", path);
        for (i = 0; i < files.gl_pathc; i++) {
                printf("\t%s\n", files.gl_pathv[i]);
        }

        globfree(&files);
        printf("Finished\n");

        return EXIT_SUCCESS;
}

man 3 glob has more information.

If wanted, I can prepare a patch for rsyslog.

Cheers,
Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Fri Dec 14 17:29:38 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 14 Dec 2007 17:29:38 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com><4760F395.4030502@ipax.at><577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com><aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
	<aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>

Michael,

sorry for the sluggish reply. I am right in the middle of the initial
threading code (have a look at CVS ;)) and that blocks most of
everything else ;)

It would be great if you could supply a patch, I'd apply it as soon as I
got something together with the new code. I try to do the threading in
an as atomic code change as possible, but it will proabbyl busy me for a
while ;)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, December 14, 2007 5:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
> 
> 2007/12/14, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/13, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Lol... the initial message seems not to have passed my spam
> filter... ;)
> > >
> > > I, too, tend to option three. The problem, though is
> >
> > Yeah, it's the most flexible one.
> >
> > > backwards-compatibility... Any suggestions and concerns are
> appreciated.
> >
> > This feature has been pretty new, and as such probably isn't widely
> used.
> > I would simply emphasize this change in the release notes and the
> Changelog.
> > The worst that can happen is, that a file is not included anymore.
> >
> > >
> > > I'll look at the code to see how much work that creates. I think
> the
> > > complexity is quite acceptable. I am right now in a threading
> redesign,
> > > so I'd like to push this away for a little while... Anyhow, please
> keep
> > > comments floating - I'll listen and drop in any thoughts I have.
> >
> > Should be fairly straightforward to implement using
> >
>
http://www.gnu.org/software/libc/manual/html_node/Globbing.html#Globbin
> g
> >
> 
> Small example using glob:
> 
> #include <glob.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <string.h>
> 
> int main(int argc, char **argv)
> {
>         glob_t files;
>         int ret = 0;
>         int i = 0;
>         char *path;
> 
>         if (argc > 1) {
>                 path = argv[1];
>         } else {
>                 path = strdup("/etc/rsyslog.d/*.conf");
>         }
> 
>         ret = glob(path, GLOB_ERR, NULL, &files);
>         printf("All files matching %s:\n", path);
>         for (i = 0; i < files.gl_pathc; i++) {
>                 printf("\t%s\n", files.gl_pathv[i]);
>         }
> 
>         globfree(&files);
>         printf("Finished\n");
> 
>         return EXIT_SUCCESS;
> }
> 
> man 3 glob has more information.
> 
> If wanted, I can prepare a patch for rsyslog.
> 
> Cheers,
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From mbiebl at gmail.com  Sun Dec 16 23:38:14 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 16 Dec 2007 23:38:14 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
	<aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
	<aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com>

2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> sorry for the sluggish reply. I am right in the middle of the initial
> threading code (have a look at CVS ;)) and that blocks most of
> everything else ;)
>
> It would be great if you could supply a patch, I'd apply it as soon as I
> got something together with the new code. I try to do the threading in
> an as atomic code change as possible, but it will proabbyl busy me for a
> while ;)

Unfortunately I can't set the mimetype via gmail, so the patch is at

http://debs.michaelbiebl.de/rsyslog/rsyslog-glob.patch

I didn't check the return values for glob. I wanted I can implement
that and printout debugging messsages in that case.
The patch also doesn't contain any changes to the documentation (yet).
I wanted to get some feedback first.

Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From mbiebl at gmail.com  Mon Dec 17 05:09:59 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Mon, 17 Dec 2007 05:09:59 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
	<aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
	<aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>
	<aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com>
Message-ID: <aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>

2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > sorry for the sluggish reply. I am right in the middle of the initial
> > threading code (have a look at CVS ;)) and that blocks most of
> > everything else ;)
> >
> > It would be great if you could supply a patch, I'd apply it as soon as I
> > got something together with the new code. I try to do the threading in
> > an as atomic code change as possible, but it will proabbyl busy me for a
> > while ;)
>
> Unfortunately I can't set the mimetype via gmail, so the patch is at
>
> http://debs.michaelbiebl.de/rsyslog/rsyslog-glob.patch
>

I have an updated patch
 http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch
which is backward compatible.
It allows the following

include config files:
 $IncludeConfig /etc/rsyslog.d/foo.conf
include config directories:
 $IncludeConfig /etc/rsyslog.d/ (with or without trailing slash)
include via shell glob:
 $IncludeConfig /etc/rsyslog.d/*.conf

One remaining issue is, how to handle symlinks. I made the patch
consistent to the current behaviour of doIncludeDirectories to ignore
symlinks.

I know that the apache Include directive allows symlinks, and
supporting it could be handy.
So my question is, what do others think about it. Please let me know.

Cheers,
Michael

P.S: A personal wish: as the list is subscriber only, would it be
possible to allow attachements (other than text/plain, gmail is
obviously too stupid)?


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From mbiebl at gmail.com  Mon Dec 17 05:27:28 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Mon, 17 Dec 2007 05:27:28 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
	<aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
	<aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>
	<aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com>
	<aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>
Message-ID: <aa76cdac0712162027r85875d2rc7223f9c6a61a43@mail.gmail.com>

2007/12/17, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> I have an updated patch
>  http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch

Spotted a minor typo, so here goes v3
http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-3.patch

Michael

Btw. gmail uses
Content-Type: multipart/mixed
and then
Content-Type: text/plain for the mail
and
Content-Type: text/x-diff for the patch
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Mon Dec 17 08:04:46 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Dec 2007 08:04:46 +0100
Subject: [rsyslog] Tackeling the segfault...
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308683@grfint2.intern.adiscon.com>

Hi all,

thanks to mildew, I could create another fix that may solve the issue. I
will do a writeup of what happens, but that'll take a short while. In
the mean time, I would appreciate if those of you that experience the
segfault could try the new version. I have released a pre-1.20.2 tarball
at

http://download.rsyslog.com/rsyslog/rsyslog-1.20.2.tar.gz

As usual, this is not yet 1.20.2, so whenever it is re-released be sure
to update your installation. I would appreciate feedback on it very
much.

Many thanks,
Rainer

From mbiebl at gmail.com  Mon Dec 17 08:20:59 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Mon, 17 Dec 2007 08:20:59 +0100
Subject: [rsyslog] Tackeling the segfault...
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308683@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308683@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>

2007/12/17, Rainer Gerhards <rgerhards at hq.adiscon.com>:
>
> http://download.rsyslog.com/rsyslog/rsyslog-1.20.2.tar.gz
>
> As usual, this is not yet 1.20.2, so whenever it is re-released be sure
> to update your installation. I would appreciate feedback on it very
> much.

Hi Rainer,

it would be handy if you would post md5sums or sha1sums of the tarball.
This would make it easier to identify/distinguish the prereleases and
the final tarball release.

Cheers,
Michael

P.S: current CVS head doesn't compile with USE_NETZIP.

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Mon Dec 17 08:26:38 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Dec 2007 08:26:38 +0100
Subject: [rsyslog] Tackeling the segfault...
In-Reply-To: <aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308683@grfint2.intern.adiscon.com>
	<aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>
Message-ID: <1197876398.21389.3.camel@localhost.localdomain>


On Mon, 2007-12-17 at 08:20 +0100, Michael Biebl wrote:
> 2007/12/17, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >
> > http://download.rsyslog.com/rsyslog/rsyslog-1.20.2.tar.gz
> >
> > As usual, this is not yet 1.20.2, so whenever it is re-released be sure
> > to update your installation. I would appreciate feedback on it very
> > much.
> 
> Hi Rainer,
> 
> it would be handy if you would post md5sums or sha1sums of the tarball.
> This would make it easier to identify/distinguish the prereleases and
> the final tarball release.

good suggestion. For this release the md5sum is
45e6f14094c26f2dd3a3ff7a4d025381 

> 
> Cheers,
> Michael
> 
> P.S: current CVS head doesn't compile with USE_NETZIP.
Will look into that. I have to admit that CVS head may have some issues
the next few days. I thought about holding off commits until everything
is perfect, that would result in one gigantic diff once it is done -
which would make it hard to follow what I am doing. So I am now doing
commits whenever something seems to work reasonably well, but definitely
not prime time yet. The 3.0 branch is not ready now for being used in
practice. But, NETZIP should compile and I'll look into it. Thanks :)

From rgerhards at hq.adiscon.com  Mon Dec 17 08:27:34 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Dec 2007 08:27:34 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712162027r85875d2rc7223f9c6a61a43@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com><4760F395.4030502@ipax.at><577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com><aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com><aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com><aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com><aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>
	<aa76cdac0712162027r85875d2rc7223f9c6a61a43@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308685@grfint2.intern.adiscon.com>

Hi Michael,

I like this patch, especially it being backwards-compatible. And I also
tend to agree that symlinks would be nice to have.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Monday, December 17, 2007 5:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
> 
> 2007/12/17, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> > > 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > I have an updated patch
> >  http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch
> 
> Spotted a minor typo, so here goes v3
> http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-3.patch
> 
> Michael
> 
> Btw. gmail uses
> Content-Type: multipart/mixed
> and then
> Content-Type: text/plain for the mail
> and
> Content-Type: text/x-diff for the patch
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Mon Dec 17 08:48:01 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Dec 2007 08:48:01 +0100
Subject: [rsyslog] Tackeling the segfault...
In-Reply-To: <aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308683@grfint2.intern.adiscon.com>
	<aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308686@grfint2.intern.adiscon.com>

> P.S: current CVS head doesn't compile with USE_NETZIP.

Mmmhhh... I cannot reproduce this on my machine (using Fedora 8). Can
you supply me the error messages - I guess best via private mail because
this will get into too much detail for most on this list (Or we move it
to the dev corner in the forum, so that those interested can follow)...

Thanks,
Rainer

From r.bhatia at ipax.at  Mon Dec 17 10:47:50 2007
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Mon, 17 Dec 2007 10:47:50 +0100
Subject: [rsyslog] Tackeling the segfault...
In-Reply-To: <aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308683@grfint2.intern.adiscon.com>
	<aa76cdac0712162320l3b089b2eue6a3c4699ad0b277@mail.gmail.com>
Message-ID: <476645C6.2080909@ipax.at>

Michael Biebl wrote:
> 2007/12/17, Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> http://download.rsyslog.com/rsyslog/rsyslog-1.20.2.tar.gz
>>
>> As usual, this is not yet 1.20.2, so whenever it is re-released be sure
>> to update your installation. I would appreciate feedback on it very
>> much.
> 
> Hi Rainer,
> 
> it would be handy if you would post md5sums or sha1sums of the tarball.
> This would make it easier to identify/distinguish the prereleases and
> the final tarball release.

or simply use the well known -pre or -rc tags?

cheers,
raoul
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________

From rgerhards at hq.adiscon.com  Mon Dec 17 11:04:33 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Dec 2007 11:04:33 +0100
Subject: [rsyslog] backward compatibility...
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>

Hi all,

I would appreciate your feedback on backward compatibility. I have
started to create a loadable input module interface in rsyslog v3. My
first input module is the mark message handler (because it has almost no
code, so it was a good example of what needs to be done...). I have now
basically have finished an initial version. The interface needs to be
changed in the longer term, but I'll postpone that until I've done a bit
more work on the config interface. Now I am facing a problem with
backward compatibility:

In sysklogd/rsyslog pre-v3, the -m option controls how often mark
messages are written (with -m0 disabling this mechanism). In v3, the
$MarkMessagePeriod config file directive does the same. However, a
$modload immark is necessary to activate that functionality because
otherwise the module is not loaded (which is good if it is not needed).
When trying to remain backward-compatible, I have an issue when the -m
option is absent. If so, that means a mark message period of 20 minutes.
However, it is hard to detect what may be intended in v3 if this option
is not present. Does it mean the former default (running as a drop-in
replacement) or does it actually mean that no mark messages should
appear if no such directive is in rsyslog.conf?

Do you think it would be acceptable to not emit messages in this case,
slightly breaking backward compatibility? Or should I come up with a
solution. The later would probably require a specific option setting to
disable the -m option and its default (somehow telling "I am not running
in v2 compatibility mode").

I am not sure what is best...

Rainer 

From r.bhatia at ipax.at  Mon Dec 17 11:17:03 2007
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Mon, 17 Dec 2007 11:17:03 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
Message-ID: <47664C9F.5030301@ipax.at>

Rainer Gerhards wrote:
> Hi all,
> 
> I would appreciate your feedback on backward compatibility. I have
> started to create a loadable input module interface in rsyslog v3. My
> first input module is the mark message handler (because it has almost no
> code, so it was a good example of what needs to be done...). I have now
> basically have finished an initial version. The interface needs to be
> changed in the longer term, but I'll postpone that until I've done a bit
> more work on the config interface. Now I am facing a problem with
> backward compatibility:
> 
> In sysklogd/rsyslog pre-v3, the -m option controls how often mark
> messages are written (with -m0 disabling this mechanism). In v3, the
> $MarkMessagePeriod config file directive does the same. However, a
> $modload immark is necessary to activate that functionality because
> otherwise the module is not loaded (which is good if it is not needed).
> When trying to remain backward-compatible, I have an issue when the -m
> option is absent. If so, that means a mark message period of 20 minutes.
> However, it is hard to detect what may be intended in v3 if this option
> is not present. Does it mean the former default (running as a drop-in
> replacement) or does it actually mean that no mark messages should
> appear if no such directive is in rsyslog.conf?
> 
> Do you think it would be acceptable to not emit messages in this case,
> slightly breaking backward compatibility? Or should I come up with a
> solution. The later would probably require a specific option setting to
> disable the -m option and its default (somehow telling "I am not running
> in v2 compatibility mode").
> 
> I am not sure what is best...

i would use the opportunity and not hesitate to use a
make-everything-right-even-if-we-break-something approach.

upgrading seems to not be trivial, as one needs to go through the config
files anyway. so lets do things right and maybe issue some error/warning
message on obsolete options (like -m).

cheers,
raoul
ps: i hope i'm getting you right ;)
-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________

From rgerhards at hq.adiscon.com  Mon Dec 17 11:35:02 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 17 Dec 2007 11:35:02 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <47664C9F.5030301@ipax.at>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
	<47664C9F.5030301@ipax.at>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA308692@grfint2.intern.adiscon.com>

> i would use the opportunity and not hesitate to use a
> make-everything-right-even-if-we-break-something approach.
> 
> upgrading seems to not be trivial, as one needs to go through the
> config
> files anyway. so lets do things right and maybe issue some
> error/warning
> message on obsolete options (like -m).
> 
> cheers,
> raoul
> ps: i hope i'm getting you right ;)

Thanks - yes, this was the kind of feedback I am looking for. Sorry for
the quite complex description, I could not come up with a better one...

Rainer

From rgerhards at hq.adiscon.com  Tue Dec 18 17:21:26 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 18 Dec 2007 17:21:26 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
	<aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
	<aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>
	<aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com>
	<aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>
Message-ID: <1197994886.14826.2.camel@localhost.localdomain>

This patch now also made it to the repository. Applied to both v2
(previously 1.20.2) and v3.

Rainer

On Mon, 2007-12-17 at 05:09 +0100, Michael Biebl wrote:
> 2007/12/16, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/14, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Michael,
> > >
> > > sorry for the sluggish reply. I am right in the middle of the initial
> > > threading code (have a look at CVS ;)) and that blocks most of
> > > everything else ;)
> > >
> > > It would be great if you could supply a patch, I'd apply it as soon as I
> > > got something together with the new code. I try to do the threading in
> > > an as atomic code change as possible, but it will proabbyl busy me for a
> > > while ;)
> >
> > Unfortunately I can't set the mimetype via gmail, so the patch is at
> >
> > http://debs.michaelbiebl.de/rsyslog/rsyslog-glob.patch
> >
> 
> I have an updated patch
>  http://debs.michaelbiebl.de/rsyslog/rsyslog-glob-2.patch
> which is backward compatible.
> It allows the following
> 
> include config files:
>  $IncludeConfig /etc/rsyslog.d/foo.conf
> include config directories:
>  $IncludeConfig /etc/rsyslog.d/ (with or without trailing slash)
> include via shell glob:
>  $IncludeConfig /etc/rsyslog.d/*.conf
> 
> One remaining issue is, how to handle symlinks. I made the patch
> consistent to the current behaviour of doIncludeDirectories to ignore
> symlinks.
> 
> I know that the apache Include directive allows symlinks, and
> supporting it could be handy.
> So my question is, what do others think about it. Please let me know.
> 
> Cheers,
> Michael
> 
> P.S: A personal wish: as the list is subscriber only, would it be
> possible to allow attachements (other than text/plain, gmail is
> obviously too stupid)?
> 
> 

From sur5r at sur5r.net  Tue Dec 18 22:16:54 2007
From: sur5r at sur5r.net (Jakob Haufe)
Date: Tue, 18 Dec 2007 22:16:54 +0100
Subject: [rsyslog] backward compatibility...
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
Message-ID: <20071218221654.27ec59a2@marathon.haufe>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote on Mon, 17 Dec 2007
11:04:33 +0100:

> Do you think it would be acceptable to not emit messages in this case,
> slightly breaking backward compatibility? Or should I come up with a
> solution. The later would probably require a specific option setting
> to disable the -m option and its default (somehow telling "I am not
> running in v2 compatibility mode").

This makes me think about sysklogd compatibility. How about the
following: If there's no rsyslog.conf but a syslog.conf then rsyslogd
could go to v2/sysklogd-compatibility mode. If not, it should run in
native mode.
This should improve compatibility in "drop-in replacement for stock
sysklogd" setups.

Regards,
sur5r
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHaDjK1YAhDic+adYRArd6AJ9fXRZU1TnW4L/eKRSfcE3x/G/PMACfbDEF
GgU6vviKYQQofAeAJ9vEnDw=
=6u5b
-----END PGP SIGNATURE-----

From mic at npgx.com.au  Wed Dec 19 04:45:00 2007
From: mic at npgx.com.au (Michael Mansour)
Date: Wed, 19 Dec 2007 13:45:00 +1000
Subject: [rsyslog] backward compatibility...
In-Reply-To: <20071218221654.27ec59a2@marathon.haufe>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
	<20071218221654.27ec59a2@marathon.haufe>
Message-ID: <20071219033907.M7098@npgx.com.au>

Hi,

> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote on Mon, 17 Dec 2007
> 11:04:33 +0100:
> 
> > Do you think it would be acceptable to not emit messages in this case,
> > slightly breaking backward compatibility? Or should I come up with a
> > solution. The later would probably require a specific option setting
> > to disable the -m option and its default (somehow telling "I am not
> > running in v2 compatibility mode").
> 
> This makes me think about sysklogd compatibility. How about the
> following: If there's no rsyslog.conf but a syslog.conf then rsyslogd
> could go to v2/sysklogd-compatibility mode. If not, it should run in
> native mode.
> This should improve compatibility in "drop-in replacement for stock
> sysklogd" setups.

That's true, but I've also found developers are happy to cater for rsyslog
through updates to their software. For example, I use drbdlinks (software for
clusters) and when I introduced rsyslog into my environment I found drbdlinks
incompatible with it. 

I put a workaround in place and at the same time emailed the drbdlinks
developer, and he introduced rsyslog support into his code, provided an update
to his software and mentioned me in the suggestion.

Jamie Cameron of Webmin, Virtualmin and Virtualmin Pro is/has also
introduced/introducing support for rsyslog into his products, since some
Webmin functionality looks for the /etc/syslog.conf file to allow some
functionality in the apps and will/now looks for /etc/rsyslog.conf also.

I saw will/now since I have his confirmation it's on his Todo list from a few
months back, but I haven't tested any updates from him since we spoke about it
back then.

The moral of this story is, for any software you use, ask the developers to
add rsyslog support if things they develop only supports syslog.

Regards,

Michael.

> Regards,
> sur5r
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> 
> iD8DBQFHaDjK1YAhDic+adYRArd6AJ9fXRZU1TnW4L/eKRSfcE3x/G/PMACfbDEF
> GgU6vviKYQQofAeAJ9vEnDw=
> =6u5b
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
------- End of Original Message -------


From mbiebl at gmail.com  Wed Dec 19 05:51:57 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Dec 2007 05:51:57 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712182051m2ef76c2sb401e0bd45904382@mail.gmail.com>

2007/12/17, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> I would appreciate your feedback on backward compatibility. I have
> started to create a loadable input module interface in rsyslog v3. My
> first input module is the mark message handler (because it has almost no
> code, so it was a good example of what needs to be done...). I have now
> basically have finished an initial version. The interface needs to be
> changed in the longer term, but I'll postpone that until I've done a bit
> more work on the config interface. Now I am facing a problem with
> backward compatibility:
>
> In sysklogd/rsyslog pre-v3, the -m option controls how often mark
> messages are written (with -m0 disabling this mechanism). In v3, the

One thing I was wondering:
If you intend to shift all (even core) functionality into loadable
modules, how do do you handle things like --help or available command
line options like -m
Do you want to hardcode it  or will you provide an interface, where
rsyslog will query the module about its help message and available
options.

I'm also still a bit uncertain, if moving everything resp. core
functionality to modules is a good idea (for problems you already
mentioned).
Imho having all core functionality in a single binary is simply much
more robust and fool proof. For things like the SQL db output plugin,
the module interface is great, because it avoids to pull in large
library and package dependencies and allows to install them on a as
need basis. For other functionality I still need to recognize the
benefits.

Rainer, could you roughly sketch, how you envision to break rsyslog
into loadable modules in v3. Which kind of functionality would be
loadable as module, which functionality do you plan to keep in the
rsyslogd binary. A listing of all (planned) modules + the provided
functionality and requirements would really help.

Another thing: Say you move the regexp support into a separate module.
If a regexp is then used in rsyslog.conf, will you bail out with an
error, simply print a warning (which could go unnoticed and the poor
administrator doesn't know why his regexp doesn't know) or load
modules on demand.
For the latter you'd need some kind of interface to query the *.so
files for their supported functionality. I.e. the modules would export
a list of config directives it supports and rsyslog could upon startup
query each available module and create a map.
So, e.g. the ommysql module would export its support for the :ommysql:
config directive.
Whenever rsyslog finds such a config directive it could/would load the
modules on demand.
Same could be done for the command line parameters.
The imklog module would export, that it supports the -m command line
parameter. Whenever that commandline parameter is used, rsyslog would
know which module to load.

There are only rough ideas and there is certainly still much to
consider. But what do you think about the basic idea?

Cheers,
Michael

Cheers,
Michael




-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From mbiebl at gmail.com  Wed Dec 19 09:27:13 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Dec 2007 09:27:13 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <1197994886.14826.2.camel@localhost.localdomain>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com>
	<4760F395.4030502@ipax.at>
	<577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com>
	<aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com>
	<aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com>
	<aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com>
	<aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com>
	<1197994886.14826.2.camel@localhost.localdomain>
Message-ID: <aa76cdac0712190027k55824b7cn582acd6e7a1a6feb@mail.gmail.com>

2007/12/18, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> This patch now also made it to the repository. Applied to both v2
> (previously 1.20.2) and v3.

Cool, thanks.
Remains to update the documentation doc/rsconf1_includeconfig.html.
I'm usually not good a writing docs, but if you want I can provide a patch ;-)

Cheers,
Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Wed Dec 19 09:33:01 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Dec 2007 09:33:01 +0100
Subject: [rsyslog] Problems with IncludeConfig
In-Reply-To: <aa76cdac0712190027k55824b7cn582acd6e7a1a6feb@mail.gmail.com>
References: <aa76cdac0712121247u52502fcaxf9fd0a33940f996f@mail.gmail.com><4760F395.4030502@ipax.at><577465F99B41C842AAFBE9ED71E70ABA30866E@grfint2.intern.adiscon.com><aa76cdac0712140536x4777273pd96abd7d5bc944eb@mail.gmail.com><aa76cdac0712140824s22f43398qe93b26462ed527f6@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA30867C@grfint2.intern.adiscon.com><aa76cdac0712161438n69382660u9d2a082914d717c9@mail.gmail.com><aa76cdac0712162009q9fe3ecaq407df79296e4baff@mail.gmail.com><1197994886.14826.2.camel@localhost.localdomain>
	<aa76cdac0712190027k55824b7cn582acd6e7a1a6feb@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086A8@grfint2.intern.adiscon.com>

Oops... this time I forgot the doc (I always try to have at least a
minimal description there). I just added two lines, maybe you have a
look. I am a bit under pressure because I just released 1.21.1, but now
re-created the tarball with the somewhat enhanced doc. If you are up for
a better doc, just sent me a patch ;) It'll appear in the next release.

Release notification is due in a few minutes...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, December 19, 2007 9:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Problems with IncludeConfig
> 
> 2007/12/18, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > This patch now also made it to the repository. Applied to both v2
> > (previously 1.20.2) and v3.
> 
> Cool, thanks.
> Remains to update the documentation doc/rsconf1_includeconfig.html.
> I'm usually not good a writing docs, but if you want I can provide a
> patch ;-)
> 
> Cheers,
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From rgerhards at hq.adiscon.com  Wed Dec 19 09:42:35 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Dec 2007 09:42:35 +0100
Subject: [rsyslog] rsyslog 1.21.0 released
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086A9@grfint2.intern.adiscon.com>

Hi all,

as I mentioned a few minutes ago, I have just released version 1.21.0.
First question may be "why 1.21.0 and not 2.0.0"? The answer is that
varmojfekoj provided a patch that allowed me to finally enable GSS-API
support in rsyslog. While I really like this, I did not find it
appropriate to include such a major new feature in the first release of
a 2.0.0 *stable*. So I went back to the v1 releases. If I do not hear of
any serious problems, I'll re-release that version as 2.0.0.

What also brings us to the new features: GSS-API is obviously now there.
There is very limited documentation that goes with it. I'll focus on
that in the v3 releases, with docs being propagated down to v2 as
possible. If anyone would like to contribute doc, I'd obviously be very
happy ;) I currently do not intend to spent too much time with it,
because I do not like to distract myself from v3 until I managed to get
at least the most important things up and running.

V1.21.0 also contains Michael Biebl's $IncludeConfig enhancement, many
cleanups as well as the fix that hopefully addresses the segfault. So
far, I have neither heard a success or failure report. For success, it
obviously is too early, rsyslogd often needed a few days to experience
the fault. I still hope for the best...

A quick note on v3: things are going pretty well. I'll post an answer to
Michael Biebl's note on what will become modular a bit later, it needs
some time. GSS-API was the last major new feature I'll accept for v2.
Everything else will go in v3. I expect to release v2 by the end of the
week and the initial v3 maybe a week later. I will make a preview
tarball of v3 available soon so that those interested can have a look at
it (and, of course, CVS head already has it).

Download links and changelog can be found on the usual places on
www.rsyslog.com.

As always, feedback is appreciated.

Rainer Gerhards 

From mbiebl at gmail.com  Wed Dec 19 09:50:44 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Dec 2007 09:50:44 +0100
Subject: [rsyslog] rsyslog 1.21.0 released
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA3086A9@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA3086A9@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712190050u5ec134fbp6675ab637602ae89@mail.gmail.com>

2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Download links and changelog can be found on the usual places on
> www.rsyslog.com.

Download link seems to be missing.

Cheers,
Michael

P.S: what about posting md5sums for release tarballs in the annoucment
email and on www.rsyslog.com?

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From mbiebl at gmail.com  Wed Dec 19 09:52:04 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Dec 2007 09:52:04 +0100
Subject: [rsyslog] rsyslog 1.21.0 released
In-Reply-To: <aa76cdac0712190050u5ec134fbp6675ab637602ae89@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA3086A9@grfint2.intern.adiscon.com>
	<aa76cdac0712190050u5ec134fbp6675ab637602ae89@mail.gmail.com>
Message-ID: <aa76cdac0712190052g1ec7035bhf68270abd32bf45b@mail.gmail.com>

2007/12/19, Michael Biebl <mbiebl at gmail.com>:
> 2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Download links and changelog can be found on the usual places on
> > www.rsyslog.com.
>
> Download link seems to be missing.

Ah, seems to be there now.

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Wed Dec 19 10:13:18 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Dec 2007 10:13:18 +0100
Subject: [rsyslog] rsyslog 1.21.0 released
In-Reply-To: <aa76cdac0712190052g1ec7035bhf68270abd32bf45b@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA3086A9@grfint2.intern.adiscon.com><aa76cdac0712190050u5ec134fbp6675ab637602ae89@mail.gmail.com>
	<aa76cdac0712190052g1ec7035bhf68270abd32bf45b@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086AB@grfint2.intern.adiscon.com>

I guess it was a cache issue, I always start by creating the download,
then changelog entry, etc... The md5sum is part of the download
description. I thought it is OK to post it there, only. It also somewhat
saved me when I applied the last-minute doc patch and re-released. With
the md5sum in the mail, it would have created some confusion. What do
you say?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, December 19, 2007 9:52 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 1.21.0 released
> 
> 2007/12/19, Michael Biebl <mbiebl at gmail.com>:
> > 2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > Download links and changelog can be found on the usual places on
> > > www.rsyslog.com.
> >
> > Download link seems to be missing.
> 
> Ah, seems to be there now.
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From mbiebl at gmail.com  Wed Dec 19 10:21:29 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Wed, 19 Dec 2007 10:21:29 +0100
Subject: [rsyslog] rsyslog 1.21.0 released
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA3086AB@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA3086A9@grfint2.intern.adiscon.com>
	<aa76cdac0712190050u5ec134fbp6675ab637602ae89@mail.gmail.com>
	<aa76cdac0712190052g1ec7035bhf68270abd32bf45b@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA3086AB@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712190121icbf2cb7v70c672a68bba9329@mail.gmail.com>

2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> I guess it was a cache issue, I always start by creating the download,
> then changelog entry, etc... The md5sum is part of the download
> description. I thought it is OK to post it there, only. It also somewhat
> saved me when I applied the last-minute doc patch and re-released. With
> the md5sum in the mail, it would have created some confusion. What do
> you say?

Agreed. Posting the md5sum on the web page should be fine.

Cheers,
Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Wed Dec 19 12:37:09 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 19 Dec 2007 12:37:09 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <aa76cdac0712182051m2ef76c2sb401e0bd45904382@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
	<aa76cdac0712182051m2ef76c2sb401e0bd45904382@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086B1@grfint2.intern.adiscon.com>

Michael and all,

I took me a while to craft a response to your excellent question. I have
done this as a blog post so that it is easier to reference it in the
future.

I suggest that everyone interested in the v3 design has a look at it,
because it describes the way I am heading. If someone doesn't like that,
it is now time to speak up - in a few weeks it will probably be
impossible to change routes.

http://rgerhards.blogspot.com/2007/12/modules-core-functionality-and-rsy
slog.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Wednesday, December 19, 2007 5:52 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] backward compatibility...
> 
> 2007/12/17, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Hi all,
> >
> > I would appreciate your feedback on backward compatibility. I have
> > started to create a loadable input module interface in rsyslog v3.
My
> > first input module is the mark message handler (because it has
almost
> no
> > code, so it was a good example of what needs to be done...). I have
> now
> > basically have finished an initial version. The interface needs to
be
> > changed in the longer term, but I'll postpone that until I've done a
> bit
> > more work on the config interface. Now I am facing a problem with
> > backward compatibility:
> >
> > In sysklogd/rsyslog pre-v3, the -m option controls how often mark
> > messages are written (with -m0 disabling this mechanism). In v3, the
> 
> One thing I was wondering:
> If you intend to shift all (even core) functionality into loadable
> modules, how do do you handle things like --help or available command
> line options like -m
> Do you want to hardcode it  or will you provide an interface, where
> rsyslog will query the module about its help message and available
> options.
> 
> I'm also still a bit uncertain, if moving everything resp. core
> functionality to modules is a good idea (for problems you already
> mentioned).
> Imho having all core functionality in a single binary is simply much
> more robust and fool proof. For things like the SQL db output plugin,
> the module interface is great, because it avoids to pull in large
> library and package dependencies and allows to install them on a as
> need basis. For other functionality I still need to recognize the
> benefits.
> 
> Rainer, could you roughly sketch, how you envision to break rsyslog
> into loadable modules in v3. Which kind of functionality would be
> loadable as module, which functionality do you plan to keep in the
> rsyslogd binary. A listing of all (planned) modules + the provided
> functionality and requirements would really help.
> 
> Another thing: Say you move the regexp support into a separate module.
> If a regexp is then used in rsyslog.conf, will you bail out with an
> error, simply print a warning (which could go unnoticed and the poor
> administrator doesn't know why his regexp doesn't know) or load
> modules on demand.
> For the latter you'd need some kind of interface to query the *.so
> files for their supported functionality. I.e. the modules would export
> a list of config directives it supports and rsyslog could upon startup
> query each available module and create a map.
> So, e.g. the ommysql module would export its support for the :ommysql:
> config directive.
> Whenever rsyslog finds such a config directive it could/would load the
> modules on demand.
> Same could be done for the command line parameters.
> The imklog module would export, that it supports the -m command line
> parameter. Whenever that commandline parameter is used, rsyslog would
> know which module to load.
> 
> There are only rough ideas and there is certainly still much to
> consider. But what do you think about the basic idea?
> 
> Cheers,
> Michael
> 
> Cheers,
> Michael
> 
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From mbiebl at gmail.com  Thu Dec 20 10:08:56 2007
From: mbiebl at gmail.com (Michael Biebl)
Date: Thu, 20 Dec 2007 10:08:56 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA3086B1@grfint2.intern.adiscon.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
	<aa76cdac0712182051m2ef76c2sb401e0bd45904382@mail.gmail.com>
	<577465F99B41C842AAFBE9ED71E70ABA3086B1@grfint2.intern.adiscon.com>
Message-ID: <aa76cdac0712200108q7e9eb058g2e179c2fe00f65d0@mail.gmail.com>

2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael and all,
>
> I took me a while to craft a response to your excellent question. I have
> done this as a blog post so that it is easier to reference it in the
> future.
>
> I suggest that everyone interested in the v3 design has a look at it,
> because it describes the way I am heading. If someone doesn't like that,
> it is now time to speak up - in a few weeks it will probably be
> impossible to change routes.

Based on your recent blog post here are some thoughts of mine. Please
keep in mind, that being (Debian) package maintainer, so I speak from
a distributors pov.

1.) external dependencies
Having loadable modules for stuff like MySQL/PostgreSQL output support
is great (for a package maintainer)
This means I can package a basic rsyslog package with minimal
dependencies (glibc) and people can chose to install
rsyslog-(mysql,pgsql) based on their needs.
Here in this case, loadable modules have a real benefit.
The only remaining external dependencies currently in rsyslog are libz
(NETZIP support) and gssapi-krb (KERBEROS support). If those
functionality would be put into a loadable module, I'd support that.
2.) memory usage
I don't think this is a real issue, even for embedded systems. See
point 1. If we manage to put external dependencies into loadable
modules, that would be sufficient imho.
Modularising everything won't have that much of an memory benefit
imho. Given that a standard rsyslogd is about 1M RSS, this doesn't
imply the need for more modularization.
3.) Security
You mentioned, that you try to improve security through modules.
Usually, having loadable module support is considered a security risk.
One messed up $IncludeConfig directive (or manipulated through a
malicious attacker), and you load potentially hazardous modules.
Loadable modules support introduces a much bigger attack vector.

I'm not a SELinux guy. But I'd be interested if loadable modules could
cause trouble in putting rsyslog in it's own security domain. Maybe
the fedora guys can comment on this.

4.) (code) modularization
Writing modular code doesn't strictly imply that the modules have to
be loadable *.so files.
You can still write modular code, with a strict API etc. and organize
it e.g. via sub directories.
5.) Performance penalty through loadable modules.
I could be wrong on this point, but given that you have pointers on
functions, when you use modules (dlsym), there is an additional
indirection on each function call. This could have a performance
impact for core functionality. This is just speculation and should be
tested/evaluated beforehand. After all, rsyslog tries to meet
high-performance needs, too.
6.) Inconvenience
This is just a gut feeling, but having to load all sorts of moduls
first, before rsyslog does anything, could prove cumbersome. As
administrator you'd have to know, which modules to load, to get
rsyslog to do what you want. This means additional effort (reading
docs) and inconvenience.
7.) Robustness
Having a single binary can prove to be a live safer. E.g. you could
simply copy the rsyslogd binary from another machine if there is
something wrong with your local copy.
As you were talking about embedded systems, having the ability to
compile a static binary including all functionality is a definite
plus. There might be platforms out there which don't support dlopen().

I'm still missing the complete picture, too. It's still a bit too vague for me.

You were talking about input, output and filter modules. Rainer, could
cou try to complete this list somehow, maybe draw/sketch a flow
diagram, marking the modules.

input: local, kernel and network, ...
filter: regexp, ...
output: mysql, pgsql, file, forward...
authorization?


I hope, this all doesn't sound too negative. But before going all
modular, all these issues should be considered imho.

Hopefully these comments will help.

Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

From rgerhards at hq.adiscon.com  Thu Dec 20 10:24:49 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Dec 2007 10:24:49 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <aa76cdac0712200108q7e9eb058g2e179c2fe00f65d0@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com><aa76cdac0712182051m2ef76c2sb401e0bd45904382@mail.gmail.com><577465F99B41C842AAFBE9ED71E70ABA3086B1@grfint2.intern.adiscon.com>
	<aa76cdac0712200108q7e9eb058g2e179c2fe00f65d0@mail.gmail.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086B9@grfint2.intern.adiscon.com>

Michael,

thanks for the good points. I'll provide a more in-depth reply later,
but noticed that I missed one important point in the blog post.

Loadable modules are also (at least partly) about platform independence.
Let me elaborate this with an example:

I have yet to fully port rsyslog to solaris. The reason I stopped is
that Solaris uses a mechanism different from Linux to handle local
logging (e.g. via logger). I didn't like to mess up even more with the
code that pulls different log sources.

With the loadable input plugins, there is a very clean solution. If you
run on Linux, just load imLocaLogLinux and if you happen to run under
Solaris load imLocalLogSolaris instead. No conditional compilation, no
complex code. Plus, anyone interested to provide an input implementation
for whatever weird system is free to do so.

So far to what I forgot to mention...

A quick reply: I think we agree that modular code is good to have. Our
main difference seems to be if it should be dynamically or statically
linked together. I am right now modularizing the inputs, have done two
so far and going to the local log socket next. That will also bring some
practical experience, which it already did. So I'll do a round of
modularization and then look at the result to fine-tune it. If it will
be loadable or not will not really be a major concern at that stage. I
just wanted to provide reasoning why I develop concurrently to this very
good discussion.

Again, a more in-depth reply a bit later.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Thursday, December 20, 2007 10:09 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] backward compatibility...
> 
> 2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael and all,
> >
> > I took me a while to craft a response to your excellent question. I
> have
> > done this as a blog post so that it is easier to reference it in the
> > future.
> >
> > I suggest that everyone interested in the v3 design has a look at
it,
> > because it describes the way I am heading. If someone doesn't like
> that,
> > it is now time to speak up - in a few weeks it will probably be
> > impossible to change routes.
> 
> Based on your recent blog post here are some thoughts of mine. Please
> keep in mind, that being (Debian) package maintainer, so I speak from
> a distributors pov.
> 
> 1.) external dependencies
> Having loadable modules for stuff like MySQL/PostgreSQL output support
> is great (for a package maintainer)
> This means I can package a basic rsyslog package with minimal
> dependencies (glibc) and people can chose to install
> rsyslog-(mysql,pgsql) based on their needs.
> Here in this case, loadable modules have a real benefit.
> The only remaining external dependencies currently in rsyslog are libz
> (NETZIP support) and gssapi-krb (KERBEROS support). If those
> functionality would be put into a loadable module, I'd support that.
> 2.) memory usage
> I don't think this is a real issue, even for embedded systems. See
> point 1. If we manage to put external dependencies into loadable
> modules, that would be sufficient imho.
> Modularising everything won't have that much of an memory benefit
> imho. Given that a standard rsyslogd is about 1M RSS, this doesn't
> imply the need for more modularization.
> 3.) Security
> You mentioned, that you try to improve security through modules.
> Usually, having loadable module support is considered a security risk.
> One messed up $IncludeConfig directive (or manipulated through a
> malicious attacker), and you load potentially hazardous modules.
> Loadable modules support introduces a much bigger attack vector.
> 
> I'm not a SELinux guy. But I'd be interested if loadable modules could
> cause trouble in putting rsyslog in it's own security domain. Maybe
> the fedora guys can comment on this.
> 
> 4.) (code) modularization
> Writing modular code doesn't strictly imply that the modules have to
> be loadable *.so files.
> You can still write modular code, with a strict API etc. and organize
> it e.g. via sub directories.
> 5.) Performance penalty through loadable modules.
> I could be wrong on this point, but given that you have pointers on
> functions, when you use modules (dlsym), there is an additional
> indirection on each function call. This could have a performance
> impact for core functionality. This is just speculation and should be
> tested/evaluated beforehand. After all, rsyslog tries to meet
> high-performance needs, too.
> 6.) Inconvenience
> This is just a gut feeling, but having to load all sorts of moduls
> first, before rsyslog does anything, could prove cumbersome. As
> administrator you'd have to know, which modules to load, to get
> rsyslog to do what you want. This means additional effort (reading
> docs) and inconvenience.
> 7.) Robustness
> Having a single binary can prove to be a live safer. E.g. you could
> simply copy the rsyslogd binary from another machine if there is
> something wrong with your local copy.
> As you were talking about embedded systems, having the ability to
> compile a static binary including all functionality is a definite
> plus. There might be platforms out there which don't support dlopen().
> 
> I'm still missing the complete picture, too. It's still a bit too
vague
> for me.
> 
> You were talking about input, output and filter modules. Rainer, could
> cou try to complete this list somehow, maybe draw/sketch a flow
> diagram, marking the modules.
> 
> input: local, kernel and network, ...
> filter: regexp, ...
> output: mysql, pgsql, file, forward...
> authorization?
> 
> 
> I hope, this all doesn't sound too negative. But before going all
> modular, all these issues should be considered imho.
> 
> Hopefully these comments will help.
> 
> Michael
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From pvrabec at redhat.com  Thu Dec 20 10:47:56 2007
From: pvrabec at redhat.com (Peter Vrabec)
Date: Thu, 20 Dec 2007 10:47:56 +0100
Subject: [rsyslog] backward compatibility...
In-Reply-To: <aa76cdac0712200108q7e9eb058g2e179c2fe00f65d0@mail.gmail.com>
References: <577465F99B41C842AAFBE9ED71E70ABA308691@grfint2.intern.adiscon.com>
	<577465F99B41C842AAFBE9ED71E70ABA3086B1@grfint2.intern.adiscon.com>
	<aa76cdac0712200108q7e9eb058g2e179c2fe00f65d0@mail.gmail.com>
Message-ID: <200712201047.56857.pvrabec@redhat.com>

just a quick response,

On Thursday 20 December 2007 10:08:56 am Michael Biebl wrote:
> 2007/12/19, Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael and all,
> >
> > I took me a while to craft a response to your excellent question. I have
> > done this as a blog post so that it is easier to reference it in the
> > future.
> >
> > I suggest that everyone interested in the v3 design has a look at it,
> > because it describes the way I am heading. If someone doesn't like that,
> > it is now time to speak up - in a few weeks it will probably be
> > impossible to change routes.
>
> Based on your recent blog post here are some thoughts of mine. Please
> keep in mind, that being (Debian) package maintainer, so I speak from
> a distributors pov.
>

> 3.) Security
> You mentioned, that you try to improve security through modules.
> Usually, having loadable module support is considered a security risk.
> One messed up $IncludeConfig directive (or manipulated through a
> malicious attacker), and you load potentially hazardous modules.
> Loadable modules support introduces a much bigger attack vector.
>
> I'm not a SELinux guy. But I'd be interested if loadable modules could
> cause trouble in putting rsyslog in it's own security domain. Maybe
> the fedora guys can comment on this.
I'm not SELinux guy either, but I don't see any problem here. Modules and 
binary will be in same domain.


From rgerhards at hq.adiscon.com  Thu Dec 20 15:56:33 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Dec 2007 15:56:33 +0100
Subject: [rsyslog] Another compatibility question...
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086BF@grfint2.intern.adiscon.com>

Hi all,

I'd appreciate your feedback on this question.

Historically, syslog messages were not only received on port 514, but
were also send from that port. This is an artifact from the fact that
early syslogd used a single port both for sending and receiving. So far,
I have maintained that and done so with some pain. With v3, I would like
to get rid of this legacy (because it introduces inter-module
dependencies and also makes multiple listeners really complicated).

So I wonder if anyone over here finds this "feature" (syslog messages
are sent from port 514 by default) important. What pops up my mind are
firewall rules. Is there anything else broken if we change that?

Feedback is highly appreciated,
Rainer

From bakers at web-ster.com  Thu Dec 20 18:04:33 2007
From: bakers at web-ster.com (Scott Baker)
Date: Thu, 20 Dec 2007 09:04:33 -0800
Subject: [rsyslog] Syslog logging taking too much disk write?
Message-ID: <476AA0A1.5080300@web-ster.com>

I have a semi busy syslog server which logs about 40 or 50 lines per
second. I noticed using dstat that it's taking a TON of disk write.
In the megaBYTES per second range. This seems extremely high to me
so I wrote a script to test it.

http://www.perturb.org/code/log_lines.pl

First download dstat, gkrellm, or iostat and look at what your
normal disk write is per second. On my underused desktop it hovers
around zero or maybe 25KB/s per second.

If I run my script and tell it to log 10 lines to syslog per second

perl log_lines.pl 10

the write usage spikes up to ~160KB/s. We're only writing about 740
bytes of data per second so why is the disk write so high?

If I tell my script to instead log to a local file in /tmp at 10
lines per second

perl log_lines.pl 10 -t

The write usage spikes to ~120KB/s about once every 5 seconds (as
the buffer fills up). I'm curious why syslog writing is so much more
intense? It seems to me on a really busy syslog server that you
could seriously smoke your HDs with all the writing it's doing. Am I
missing something?

-- 
Scott Baker - Canby Telcom
RHCE - System Administrator - 503.266.8253

From rgerhards at hq.adiscon.com  Thu Dec 20 18:39:40 2007
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 20 Dec 2007 18:39:40 +0100
Subject: [rsyslog] Syslog logging taking too much disk write?
In-Reply-To: <476AA0A1.5080300@web-ster.com>
References: <476AA0A1.5080300@web-ster.com>
Message-ID: <577465F99B41C842AAFBE9ED71E70ABA3086C1@grfint2.intern.adiscon.com>

Could you post your rsyslog.conf please. Without seeing it, it sounds
like you sync the disc with every write. Unfortunately, this is the
legacy default. You can turn it off with placing a dash before the
selector line:

e.g. *.* /var/log/whatever

syncs after each write, while

*.* -/var/log/whatever

uses buffering - which should be fairly safe with decent hardware and a
UPS...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Scott Baker
> Sent: Thursday, December 20, 2007 6:05 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Syslog logging taking too much disk write?
> 
> I have a semi busy syslog server which logs about 40 or 50 lines per
> second. I noticed using dstat that it's taking a TON of disk write.
> In the megaBYTES per second range. This seems extremely high to me
> so I wrote a script to test it.
> 
> http://www.perturb.org/code/log_lines.pl
> 
> First download dstat, gkrellm, or iostat and look at what your
> normal disk write is per second. On my underused desktop it hovers
> around zero or maybe 25KB/s per second.
> 
> If I run my script and tell it to log 10 lines to syslog per second
> 
> perl log_lines.pl 10
> 
> the write usage spikes up to ~160KB/s. We're only writing about 740
> bytes of data per second so why is the disk write so high?
> 
> If I tell my script to instead log to a local file in /tmp at 10
> lines per second
> 
> perl log_lines.pl 10 -t
> 
> The write usage spikes to ~120KB/s about once every 5 seconds (as
> the buffer fills up). I'm curious why syslog writing is so much more
> intense? It seems to me on a really busy syslog server that you
> could seriously smoke your HDs with all the writing it's doing. Am I
> missing something?
> 
> --
> Scott Baker - Canby Telcom
> RHCE - System Administrator - 503.266.8253
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog

From bakers at web-ster.com  Thu Dec 20 19:09:06 2007
From: bakers at web-ster.com (Scott Baker)
Date: Thu, 20 Dec 2007 10:09:06 -0800
Subject: [rsyslog] Syslog logging taking too much disk write?
In-Reply-To: <577465F99B41C842AAFBE9ED71E70ABA3086C1@grfint2.intern.adiscon.com>
References: <476AA0A1.5080300@web-ster.com>
	<577465F99B41C842AAFBE9ED71E70ABA3086C1@grfint2.intern.adiscon.com>
Message-ID: <476AAFC2.207@web-ster.com>

Rainer Gerhards wrote:
>