From gmanea-ma at lists.mszet.de  Sat Jan  2 11:43:51 2010
From: gmanea-ma at lists.mszet.de (Michael =?UTF-8?B?U3RyYXXDnw==?=)
Date: Sat, 2 Jan 2010 11:43:51 +0100
Subject: [rsyslog] rsyslog config-test croaks
Message-ID: <20100102114351.00f8f080@merkur.home.mszet.de>

Hello

I'm Michael Strau? and i am tying to use rsyslog.

I'm using debian Lenny width rsyslog 4.4.2 from backports.org.
I changed the debian standard configuration and adapted it for my needs.

It works in my opinion fully correctly. So I am impossible to
comprehend this error:

# rsyslogd -c4 -N1
rsyslogd: version 4.4.2, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: the last error occured in /etc/rsyslog.conf, line 25
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

These are the criticized Lines:

24: $ModLoad imrelp
25: $InputRELPServerRun 2514
26:

I can't detect a failure.

Best regards
Michael

-- 




From mbiebl at gmail.com  Sat Jan  2 15:41:05 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sat, 2 Jan 2010 15:41:05 +0100
Subject: [rsyslog] rsyslog config-test croaks
In-Reply-To: <20100102114351.00f8f080@merkur.home.mszet.de>
References: <20100102114351.00f8f080@merkur.home.mszet.de>
Message-ID: <aa76cdac1001020641r2be6200cnbfa29607a100e963@mail.gmail.com>

2010/1/2 Michael Strau? <gmanea-ma at lists.mszet.de>:
> Hello
>
> I'm Michael Strau? and i am tying to use rsyslog.
>
> I'm using debian Lenny width rsyslog 4.4.2 from backports.org.
> I changed the debian standard configuration and adapted it for my needs.
>
> It works in my opinion fully correctly. So I am impossible to
> comprehend this error:
>
> # rsyslogd -c4 -N1
> rsyslogd: version 4.4.2, config validation run (level 1), master config /etc/rsyslog.conf
> rsyslogd: the last error occured in /etc/rsyslog.conf, line 25
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> These are the criticized Lines:
>
> 24: $ModLoad imrelp
> 25: $InputRELPServerRun 2514
> 26:
>
> I can't detect a failure.

Have you installed the rsyslog-relp package which contains the imrelp module?

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From gmanea-ma at lists.mszet.de  Sat Jan  2 21:20:26 2010
From: gmanea-ma at lists.mszet.de (Michael =?UTF-8?B?U3RyYXXDnw==?=)
Date: Sat, 2 Jan 2010 21:20:26 +0100
Subject: [rsyslog] rsyslog config-test croaks
References: <20100102114351.00f8f080@merkur.home.mszet.de>
	<aa76cdac1001020641r2be6200cnbfa29607a100e963@mail.gmail.com>
Message-ID: <20100102212026.65b0fd56@merkur.home.mszet.de>

On Sat, 2 Jan 2010 15:41:05 +0100, Michael Biebl <mbiebl at gmail.com> wrote:

> 
> Have you installed the rsyslog-relp package which contains the imrelp module?
> 

Yes, and also it receives the messages from the client correctly.
This is merely a cosmetic problem.

Regards,
Michael

-- 




From ktm at rice.edu  Tue Jan  5 20:53:49 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Tue, 5 Jan 2010 13:53:49 -0600
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
Message-ID: <20100105195349.GO18110@it.is.rice.edu>

I am running rsyslog version 4.2.0 on a Redhat 5 machine
and noticed slow logins to the box. The strace on the login
sshd shows the following:

9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
9937       0.000019 connect(4, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be restarted)
5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 9844
5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [], SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted system call)
5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished ...>
9937       8.001608 <... sendto resumed> ) = 90
9937       0.000028 close(4)            = 0
9937       0.000039 read(6, "\0\0\5\36", 4) = 4
9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
9937       0.000104 close(6)            = 0
9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9938
9937       0.000032 alarm(0)            = 102
9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
...

The problem seems to be caused by writing to /dev/log which should
be being managed by the rsyslog program. I see a similar problem
reported earlier on the forum:

rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)

This was for version 3.18.4 but the symptom sounded very similar.
I restarted the rsyslog process and the login times returned to normal.
Let me know if there is something further I can do to help you debug
this matter.

Regards,
Ken


From david at lang.hm  Tue Jan  5 22:12:43 2010
From: david at lang.hm (david at lang.hm)
Date: Tue, 5 Jan 2010 13:12:43 -0800 (PST)
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
In-Reply-To: <20100105195349.GO18110@it.is.rice.edu>
References: <20100105195349.GO18110@it.is.rice.edu>
Message-ID: <alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>

this sounds like rsyslog is failing to send the logs out to the RELP 
server, and so is building up a large queue. restarting rsyslog would 
clear the queued up log messages and make it fast again.

David Lang


  On Tue, 5 Jan 2010, Kenneth Marshall wrote:

> Date: Tue, 5 Jan 2010 13:53:49 -0600
> From: Kenneth Marshall <ktm at rice.edu>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Cc: sandmant at rice.edu
> Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
> 
> I am running rsyslog version 4.2.0 on a Redhat 5 machine
> and noticed slow logins to the box. The strace on the login
> sshd shows the following:
>
> 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> 9937       0.000019 connect(4, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be restarted)
> 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 9844
> 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [], SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted system call)
> 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished ...>
> 9937       8.001608 <... sendto resumed> ) = 90
> 9937       0.000028 close(4)            = 0
> 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> 9937       0.000104 close(6)            = 0
> 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9938
> 9937       0.000032 alarm(0)            = 102
> 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> ...
>
> The problem seems to be caused by writing to /dev/log which should
> be being managed by the rsyslog program. I see a similar problem
> reported earlier on the forum:
>
> rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
>
> This was for version 3.18.4 but the symptom sounded very similar.
> I restarted the rsyslog process and the login times returned to normal.
> Let me know if there is something further I can do to help you debug
> this matter.
>
> Regards,
> Ken
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From kenneho.ndu at gmail.com  Wed Jan  6 15:57:25 2010
From: kenneho.ndu at gmail.com (Kenneth Holter)
Date: Wed, 6 Jan 2010 15:57:25 +0100
Subject: [rsyslog] rsyslog+stunnel works only when running "rsyslogd"
	fromthe shell
In-Reply-To: <3A240503F9F2194780469F072D9A70541162FF2B@m342.silverspringnet.com>
References: <c25f25140912230012s5570db5r9c1959ab0f2d5b5e@mail.gmail.com>
	<3A240503F9F2194780469F072D9A70541162FF2B@m342.silverspringnet.com>
Message-ID: <c25f25141001060657p475097b8o44be587c2a1a1fb3@mail.gmail.com>

Yeah, it's really old, but for now I'll have to stick with it. :(


I added the "-d" option to rsyslog daemon, and came across this:

1098717504: Called fprintlog, logging to builtin-fwd 127.0.0.1:61514/tcp
1098717504: create tcp connection failed, reason Permission denied
1098717504: no working socket could be obtained
1098717504: error forwarding via tcp, suspending

Seems like the reason why it doesn't work is that it fails to create the TCP
session from itself (i.e. rsyslog) to the stunnel port. I've sent this
information to Red Hat support, but if anyone here have an ideas as to
what's causing this please do let me know.


- Kenneth

On Wed, Dec 23, 2009 at 9:59 PM, Siddhartha Jain
<sjain at silverspringnet.com>wrote:

> Kenneth,
>
> Not sure why RedHat/CentOS continue to bundle rsyslog 2.0.6. This
> version is ancient. Since 2.x, rsyslog has gone through 2.x, 4.x and now
> the current, 5.x.
>
> I would highly recommend rolling your own RPM from recent 5.x or 4.x
> code.
>
> - Siddhartha
>
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Kenneth Holter
> > Sent: Wednesday, December 23, 2009 12:13 AM
> > To: rsyslog at lists.adiscon.com
>  > Subject: [rsyslog] rsyslog+stunnel works only when running "rsyslogd"
> > fromthe shell
> >
> > Hi.
> >
> >
> > I'm running rsyslog v2.0.6 provided with my RHEL 5 installation. For
> > some
> > time now I've had rsyslog issues with some of my RHEL 5 servers, and
> > I've
> > not been able to figure out the problems, and would like to hear from
> > others
> > that may have experienced the same problem. I've been in contact with
> > Red
> > Hat support, but they've not been able to reproduce this problem, so
> > we'be
> > not succeeded in resolving the issue.
> >
> > First, let me describe my setup: My RHEL 5 servers have set up a TLS
> > tunnel
> > (using stunnel) between themselves and the log host. This works
> > perfectly.
> > I've configured rsyslog to forward messages to this tunnel by adding a
> > "
> > *.*  @@127.0.0.1:61514 " line to the bottom of /etc/rsyslog.conf file.
> > The
> > stunnel is listening on port 61514.
> >
> > On almost all my servers, this works as planned. But for some reason,
> a
> > few
> > servers are having problems forwarding messages to their stunnel
> > connection.
> > By running "tcpdump -i lo" I can see that these servers are not
> > transmitting
> > anything on the loopback interface, and are thus not forwarding
> > anything to
> > the stunnel port. One of my theories was that the line above simply
> > wasn't
> > picked up by rsyslog daemon. So I stopped the daemon, ran "rsyslogd
> -d"
> > to
> > view the debug output, and everthing works fine.
> >
> > For some reason, when I run rsyslog like this (i.e by issuing
> > "rsyslogd" in
> > the command prompt) instead of issuing "/etc/init.d/rsyslog start",
> > everything work fine. I'm really puzzled as to why this is so. Does
> > anyone
> > know why this is so? I have the exact same setup one all my servers,
> > but one
> > a small number of them have this problem.
> >
> >
> > Best regards,
> > Kenneth Holter
>  > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From marc.schiffbauer at mightycare.de  Wed Jan  6 16:14:59 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 16:14:59 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
Message-ID: <201001061615.00121.marc.schiffbauer@mightycare.de>

Hi all,

which encoding should be chosen for the database when using postgres?

My rsyslog version is 4.4.3.

Which client_encoding does rsyslog use in ompgsql?


I currently have set UTF-8 on the database. It worked for a while until some 
special message arrived at the server where postgres denies the INSERT:

2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for 
encoding "UTF8": 0xd220
2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if the 
byte sequence does not match the encoding expected by the server, which is 
controlled by "client_encoding".

Now rsyslog is not able to log anything... it is currently spooling to disk 
because it "hangs" at this message not being accepted by postgres.

Any hints?
TIA
-Marc



From marc.schiffbauer at mightycare.de  Wed Jan  6 16:48:02 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 16:48:02 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061615.00121.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
Message-ID: <201001061648.02984.marc.schiffbauer@mightycare.de>

Hi all again,

replying to myself because I think I found the solution:

With an db encoding of SQL_ASCII the postgres server will not do any character 
conversion which seems to be the right thing for syslog messages where the 
encoding cannot be determined reliably.

Maybe this is an important piece for the rsyslog documentation as well.

Now everthing is working again.

To convert my existing database I switch to user postgres and used "pg_dump -C 
syslog > syslog.sql" to dump the database. Then added a "DROP DATABASE syslog" 
before the "CREATE DATABASE", changed any encodings from "UTF-8" to 
"SQL_ASCII" (client_encoding and in the CREATE DATABASE statement) and then 
loaded the data again with "psql < syslog.sql".

-Marc



Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> Hi all,
> 
> which encoding should be chosen for the database when using postgres?
> 
> My rsyslog version is 4.4.3.
> 
> Which client_encoding does rsyslog use in ompgsql?
> 
> 
> I currently have set UTF-8 on the database. It worked for a while until
>  some special message arrived at the server where postgres denies the
>  INSERT:
> 
> 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> encoding "UTF8": 0xd220
> 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
>  the byte sequence does not match the encoding expected by the server,
>  which is controlled by "client_encoding".
> 
> Now rsyslog is not able to log anything... it is currently spooling to disk
> because it "hangs" at this message not being accepted by postgres.
> 
> Any hints?
> TIA
> -Marc
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Wed Jan  6 16:53:52 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Wed, 6 Jan 2010 09:53:52 -0600
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061648.02984.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
Message-ID: <20100106155352.GU18110@it.is.rice.edu>

Would it be possible to send the poorly behaving loggers to
a different port to allow it to be cleaned up properly? Using
SQL_ASCII does allow truly anything into the database, which
means that all the output pieces need to process it appropriately
too.

Regards,
Ken

On Wed, Jan 06, 2010 at 04:48:02PM +0100, Marc Schiffbauer wrote:
> Hi all again,
> 
> replying to myself because I think I found the solution:
> 
> With an db encoding of SQL_ASCII the postgres server will not do any character 
> conversion which seems to be the right thing for syslog messages where the 
> encoding cannot be determined reliably.
> 
> Maybe this is an important piece for the rsyslog documentation as well.
> 
> Now everthing is working again.
> 
> To convert my existing database I switch to user postgres and used "pg_dump -C 
> syslog > syslog.sql" to dump the database. Then added a "DROP DATABASE syslog" 
> before the "CREATE DATABASE", changed any encodings from "UTF-8" to 
> "SQL_ASCII" (client_encoding and in the CREATE DATABASE statement) and then 
> loaded the data again with "psql < syslog.sql".
> 
> -Marc
> 
> 
> 
> Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> > Hi all,
> > 
> > which encoding should be chosen for the database when using postgres?
> > 
> > My rsyslog version is 4.4.3.
> > 
> > Which client_encoding does rsyslog use in ompgsql?
> > 
> > 
> > I currently have set UTF-8 on the database. It worked for a while until
> >  some special message arrived at the server where postgres denies the
> >  INSERT:
> > 
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
> >  the byte sequence does not match the encoding expected by the server,
> >  which is controlled by "client_encoding".
> > 
> > Now rsyslog is not able to log anything... it is currently spooling to disk
> > because it "hangs" at this message not being accepted by postgres.
> > 
> > Any hints?
> > TIA
> > -Marc
> > 
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From marc.schiffbauer at mightycare.de  Wed Jan  6 17:32:43 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 17:32:43 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <20100106155352.GU18110@it.is.rice.edu>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
	<20100106155352.GU18110@it.is.rice.edu>
Message-ID: <201001061732.44115.marc.schiffbauer@mightycare.de>

Am Mittwoch, 6. Januar 2010 16:53:52 schrieb Kenneth Marshall:
> Would it be possible to send the poorly behaving loggers to
> a different port to allow it to be cleaned up properly? 

No, not in that case I am afraid.

An option in rsyslog that would allow it to skip/trash/log-to-a-file those bad 
messages would be a nice thing.

> Using
> SQL_ASCII does allow truly anything into the database, which
> means that all the output pieces need to process it appropriately
> too.

Yes but this is working nicely here with phplogcon.

-Marc

> 
> Regards,
> Ken
> 
> On Wed, Jan 06, 2010 at 04:48:02PM +0100, Marc Schiffbauer wrote:
> > Hi all again,
> >
> > replying to myself because I think I found the solution:
> >
> > With an db encoding of SQL_ASCII the postgres server will not do any
> > character conversion which seems to be the right thing for syslog
> > messages where the encoding cannot be determined reliably.
> >
> > Maybe this is an important piece for the rsyslog documentation as well.
> >
> > Now everthing is working again.
> >
> > To convert my existing database I switch to user postgres and used
> > "pg_dump -C syslog > syslog.sql" to dump the database. Then added a "DROP
> > DATABASE syslog" before the "CREATE DATABASE", changed any encodings from
> > "UTF-8" to "SQL_ASCII" (client_encoding and in the CREATE DATABASE
> > statement) and then loaded the data again with "psql < syslog.sql".
> >
> > -Marc
> >
> > Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> > > Hi all,
> > >
> > > which encoding should be chosen for the database when using postgres?
> > >
> > > My rsyslog version is 4.4.3.
> > >
> > > Which client_encoding does rsyslog use in ompgsql?
> > >
> > >
> > > I currently have set UTF-8 on the database. It worked for a while until
> > >  some special message arrived at the server where postgres denies the
> > >  INSERT:
> > >
> > > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > > encoding "UTF8": 0xd220
> > > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen
> > > if the byte sequence does not match the encoding expected by the
> > > server, which is controlled by "client_encoding".
> > >
> > > Now rsyslog is not able to log anything... it is currently spooling to
> > > disk because it "hangs" at this message not being accepted by postgres.
> > >
> > > Any hints?
> > > TIA
> > > -Marc
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Wed Jan  6 17:40:16 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Wed, 6 Jan 2010 10:40:16 -0600
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061732.44115.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
	<20100106155352.GU18110@it.is.rice.edu>
	<201001061732.44115.marc.schiffbauer@mightycare.de>
Message-ID: <20100106164016.GV18110@it.is.rice.edu>

On Wed, Jan 06, 2010 at 05:32:43PM +0100, Marc Schiffbauer wrote:
> Am Mittwoch, 6. Januar 2010 16:53:52 schrieb Kenneth Marshall:
> > Would it be possible to send the poorly behaving loggers to
> > a different port to allow it to be cleaned up properly? 
> 
> No, not in that case I am afraid.
> 
> An option in rsyslog that would allow it to skip/trash/log-to-a-file those bad 
> messages would be a nice thing.
> 
> > Using
> > SQL_ASCII does allow truly anything into the database, which
> > means that all the output pieces need to process it appropriately
> > too.
> 
> Yes but this is working nicely here with phplogcon.
> 
> -Marc
> 
I was more concerned about possible compromizes caused by the
ability to insert pretty arbitrary binary data into the system.
If we have this problem in the future, I will investigate other
options further. It might be possible to have the driver also
store them in a bad record table using such an option.

Cheers,
Ken


From a.smith at ukgrid.net  Thu Jan  7 15:53:36 2010
From: a.smith at ukgrid.net (Andy Smith)
Date: Thu, 07 Jan 2010 14:53:36 +0000
Subject: [rsyslog] help with config syntax
Message-ID: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>

Hi,

   Im having trouble getting the config setup how I need it. On a mail  
server I have a lot of data being written to the main messages file,  
thats because I have mail daemons writting data with a "notice"  
severity that is configured to be written to messages (so this is  
expected). How can I prevent just mail.notice going to the messages  
file while keeping all other *.notice stuff going there? I tried  
adding !mail.notice to the config for the messages file but this didnt  
seem to work...
Here is my config:

*.err;kern.warning;auth.notice;mail.crit;local7.none             
/dev/console;Tra
ditionalFormatWithPRI
mail.info;mail.notice                                    
-/var/log/maillog;Tradit
ionalFormatWithPRI
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local7.none       
  -/var/log/messages;TraditionalFormatWithPRI
security.*                                      /var/log/security
auth.info;authpriv.info                         /var/log/auth.log
lpr.info                                        /var/log/lpd-errs
ftp.info                                        /var/log/xferlog
cron.*                                          /var/log/cron
*.=debug;local7.none                                    /var/log/debug.log
*.emerg                                         *

thanks Andy.


From danson at rackspace.com  Thu Jan  7 19:43:54 2010
From: danson at rackspace.com (Daniel Anson)
Date: Thu, 7 Jan 2010 12:43:54 -0600
Subject: [rsyslog] RHEL5 rsyslog 4 rpms
Message-ID: <7616_1262890053_o07IlMci013462_8DFDF421C24C4B4883F75F4E81EF785627D32BEBD6@DFW1MXM01.RACKSPACE.CORP>

If anyone is interested, an RPM engineer I know has packaged RHEL5 rsyslog4 rpms.  These are available for public download and testing @ http://dl.iuscommunity.org/pub/ius  Any comments can be emailed directly to him at ius-coredev at lists.launchpad.net

rpms are regularly packaged by him so let him know what you think.  I believe you just have to add the yum repo.

--Daniel M. Anson
--Linux Systems Engineer



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From david at lang.hm  Fri Jan  8 17:41:33 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 8 Jan 2010 08:41:33 -0800 (PST)
Subject: [rsyslog] help with config syntax
In-Reply-To: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>
References: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>
Message-ID: <alpine.DEB.2.00.1001080840040.3341@asgard.lang.hm>

On Thu, 7 Jan 2010, Andy Smith wrote:

> Hi,
>
>   Im having trouble getting the config setup how I need it. On a mail
> server I have a lot of data being written to the main messages file,
> thats because I have mail daemons writting data with a "notice"
> severity that is configured to be written to messages (so this is
> expected). How can I prevent just mail.notice going to the messages
> file while keeping all other *.notice stuff going there? I tried
> adding !mail.notice to the config for the messages file but this didnt
> seem to work...
> Here is my config:
>
> *.err;kern.warning;auth.notice;mail.crit;local7.none  /dev/console;TraditionalFormatWithPRI
> mail.info;mail.notice -/var/log/maillog;TraditionalFormatWithPRI

at this point you can tell it to drop the message by adding the line

& ~

this tells it to use the same matchine rules as the line above, and drop 
the message (don't process it in any further rules)

David Lang

> *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local7.none
>  -/var/log/messages;TraditionalFormatWithPRI
> security.*                                      /var/log/security
> auth.info;authpriv.info                         /var/log/auth.log
> lpr.info                                        /var/log/lpd-errs
> ftp.info                                        /var/log/xferlog
> cron.*                                          /var/log/cron
> *.=debug;local7.none                                    /var/log/debug.log
> *.emerg                                         *
>
> thanks Andy.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Mon Jan 11 12:15:55 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jan 2010 12:15:55 +0100
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
References: <20100105195349.GO18110@it.is.rice.edu>
	<alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>

I think there is a patch (or a recommendation) regarding RELP in my mail
backlog. If I got it right, RELP does not necessarily detect a broken
connection, and thus no recovery action is initiated. I'll try to get to this
ASAP, but I am now the second day in office and there is still a pile of
things I need to look into ...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, January 05, 2010 10:13 PM
> To: rsyslog-users
> Cc: sandmant at rice.edu
> Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> 
> this sounds like rsyslog is failing to send the logs out to the RELP
> server, and so is building up a large queue. restarting rsyslog would
> clear the queued up log messages and make it fast again.
> 
> David Lang
> 
> 
>   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> 
> > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > From: Kenneth Marshall <ktm at rice.edu>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog at lists.adiscon.com
> > Cc: sandmant at rice.edu
> > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> >
> > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > and noticed slow logins to the box. The strace on the login
> > sshd shows the following:
> >
> > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> path="/dev/log"...}, 110) = 0
> > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be
> restarted)
> > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> 0}], WNOHANG, NULL) = 9844
> > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [],
> SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> system call)
> > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> ...>
> > 9937       8.001608 <... sendto resumed> ) = 90
> > 9937       0.000028 close(4)            = 0
> > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> > 9937       0.000104 close(6)            = 0
> > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> 0}], 0, NULL) = 9938
> > 9937       0.000032 alarm(0)            = 102
> > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [],
> SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > ...
> >
> > The problem seems to be caused by writing to /dev/log which should
> > be being managed by the rsyslog program. I see a similar problem
> > reported earlier on the forum:
> >
> > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> >
> > This was for version 3.18.4 but the symptom sounded very similar.
> > I restarted the rsyslog process and the login times returned to
> normal.
> > Let me know if there is something further I can do to help you debug
> > this matter.
> >
> > Regards,
> > Ken
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ktm at rice.edu  Mon Jan 11 14:52:19 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Mon, 11 Jan 2010 07:52:19 -0600
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5	machine
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
References: <20100105195349.GO18110@it.is.rice.edu>
	<alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
Message-ID: <20100111135218.GM1895@it.is.rice.edu>

It does seem to act like the RELP problem, but my use is only
with a regular TCP connection using @@logmachine. It had the
same symptom and restarting rsyslog cleared it up.

Regards,
Ken

On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote:
> I think there is a patch (or a recommendation) regarding RELP in my mail
> backlog. If I got it right, RELP does not necessarily detect a broken
> connection, and thus no recovery action is initiated. I'll try to get to this
> ASAP, but I am now the second day in office and there is still a pile of
> things I need to look into ...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Tuesday, January 05, 2010 10:13 PM
> > To: rsyslog-users
> > Cc: sandmant at rice.edu
> > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > machine
> > 
> > this sounds like rsyslog is failing to send the logs out to the RELP
> > server, and so is building up a large queue. restarting rsyslog would
> > clear the queued up log messages and make it fast again.
> > 
> > David Lang
> > 
> > 
> >   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> > 
> > > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > > From: Kenneth Marshall <ktm at rice.edu>
> > > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > > To: rsyslog at lists.adiscon.com
> > > Cc: sandmant at rice.edu
> > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > machine
> > >
> > > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > > and noticed slow logins to the box. The strace on the login
> > > sshd shows the following:
> > >
> > > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> > path="/dev/log"...}, 110) = 0
> > > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be
> > restarted)
> > > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > 0}], WNOHANG, NULL) = 9844
> > > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [],
> > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> > system call)
> > > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> > ...>
> > > 9937       8.001608 <... sendto resumed> ) = 90
> > > 9937       0.000028 close(4)            = 0
> > > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> > > 9937       0.000104 close(6)            = 0
> > > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > 0}], 0, NULL) = 9938
> > > 9937       0.000032 alarm(0)            = 102
> > > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [],
> > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > > ...
> > >
> > > The problem seems to be caused by writing to /dev/log which should
> > > be being managed by the rsyslog program. I see a similar problem
> > > reported earlier on the forum:
> > >
> > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> > >
> > > This was for version 3.18.4 but the symptom sounded very similar.
> > > I restarted the rsyslog process and the login times returned to
> > normal.
> > > Let me know if there is something further I can do to help you debug
> > > this matter.
> > >
> > > Regards,
> > > Ken
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From rgerhards at hq.adiscon.com  Mon Jan 11 16:39:01 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jan 2010 16:39:01 +0100
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5	machine
References: <20100105195349.GO18110@it.is.rice.edu><alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
	<20100111135218.GM1895@it.is.rice.edu>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036A0@GRFEXC.intern.adiscon.com>

A "problem" I am aware of is that a died peer (or connection dropped an
interim firewall) is not detected as broken, because no messages are
exchanged any longer. An often-used solution is KEEPALIVE, but this can also
take some time to timeout (and may have bad effects on slow connection or
those with outages of interim systems). I know that I wanted to implement the
capability to activate KEEPALIVE, but I am not sure if I found time to
actually do it. Will let you know once I can check that.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Kenneth Marshall
> Sent: Monday, January 11, 2010 2:52 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> 
> It does seem to act like the RELP problem, but my use is only
> with a regular TCP connection using @@logmachine. It had the
> same symptom and restarting rsyslog cleared it up.
> 
> Regards,
> Ken
> 
> On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote:
> > I think there is a patch (or a recommendation) regarding RELP in my
> mail
> > backlog. If I got it right, RELP does not necessarily detect a broken
> > connection, and thus no recovery action is initiated. I'll try to get
> to this
> > ASAP, but I am now the second day in office and there is still a pile
> of
> > things I need to look into ...
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > > Sent: Tuesday, January 05, 2010 10:13 PM
> > > To: rsyslog-users
> > > Cc: sandmant at rice.edu
> > > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a
> RHEL5
> > > machine
> > >
> > > this sounds like rsyslog is failing to send the logs out to the
> RELP
> > > server, and so is building up a large queue. restarting rsyslog
> would
> > > clear the queued up log messages and make it fast again.
> > >
> > > David Lang
> > >
> > >
> > >   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> > >
> > > > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > > > From: Kenneth Marshall <ktm at rice.edu>
> > > > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > > > To: rsyslog at lists.adiscon.com
> > > > Cc: sandmant at rice.edu
> > > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > > machine
> > > >
> > > > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > > > and noticed slow logins to the box. The strace on the login
> > > > sshd shows the following:
> > > >
> > > > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > > > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > > > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> > > path="/dev/log"...}, 110) = 0
> > > > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]:
> "...,
> > > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > > > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]:
> "...,
> > > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > > > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To
> be
> > > restarted)
> > > > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > > 0}], WNOHANG, NULL) = 9844
> > > > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > > > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740,
> [],
> > > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > > > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> > > system call)
> > > > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> > > ...>
> > > > 9937       8.001608 <... sendto resumed> ) = 90
> > > > 9937       0.000028 close(4)            = 0
> > > > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > > > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> > > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) =
> 1310
> > > > 9937       0.000104 close(6)            = 0
> > > > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> > > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > > > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > > > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s)
> ==
> > > 0}], 0, NULL) = 9938
> > > > 9937       0.000032 alarm(0)            = 102
> > > > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0,
> [],
> > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > > > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > > > ...
> > > >
> > > > The problem seems to be caused by writing to /dev/log which
> should
> > > > be being managed by the rsyslog program. I see a similar problem
> > > > reported earlier on the forum:
> > > >
> > > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> > > >
> > > > This was for version 3.18.4 but the symptom sounded very similar.
> > > > I restarted the rsyslog process and the login times returned to
> > > normal.
> > > > Let me know if there is something further I can do to help you
> debug
> > > > this matter.
> > > >
> > > > Regards,
> > > > Ken
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dirk.schulz at kinzesberg.de  Mon Jan 11 18:23:18 2010
From: dirk.schulz at kinzesberg.de (Dirk H. Schulz)
Date: Mon, 11 Jan 2010 18:23:18 +0100
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
Message-ID: <4B4B5E86.9050409@kinzesberg.de>

Hi folks,

I am running two central logservers using rsyslog that several dozen 
servers report to (mostly also rsyslog).

The  central logservers are writing everything into a  database and 
additionally into local logfiles.

I would like to change configuration in a way that only local messages 
are written to local logfiles, and all messages (local and received from 
remote servers) into the database.

Is this possible with Rsyslog? I have searched the documentation, but 
did not find anything helpful.

Any hint or help is appreciated.

Dirk



From david at lang.hm  Mon Jan 11 19:42:20 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jan 2010 10:42:20 -0800 (PST)
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
In-Reply-To: <4B4B5E86.9050409@kinzesberg.de>
References: <4B4B5E86.9050409@kinzesberg.de>
Message-ID: <alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>

On Mon, 11 Jan 2010, Dirk H. Schulz wrote:

> Hi folks,
>
> I am running two central logservers using rsyslog that several dozen
> servers report to (mostly also rsyslog).
>
> The  central logservers are writing everything into a  database and
> additionally into local logfiles.
>
> I would like to change configuration in a way that only local messages
> are written to local logfiles, and all messages (local and received from
> remote servers) into the database.

yes, I do something similar to this on my systems.

All logs except local logs get written to local files, all local logs get 
sent over the network (at which point they then get picked up as remote 
logs), and all logs (local or remote) get sent to a remote system.

:fromhost, !isequal, "127.0.0.1"        /var/log/messages;TraditionalFormat
:fromhost, isequal, "127.0.0.1"         @192.168.1.8;TraditionalForwardFormat
*.*             @192.168.1.2





From paul.ruiz at gmail.com  Mon Jan 11 21:46:10 2010
From: paul.ruiz at gmail.com (Paul Ruiz)
Date: Mon, 11 Jan 2010 12:46:10 -0800
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
In-Reply-To: <alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>
References: <4B4B5E86.9050409@kinzesberg.de>
	<alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>
Message-ID: <f210e0b1001111246v56fe45acnf00c241f319148e4@mail.gmail.com>

I do this by running 2 rsyslog processes, one for local logs just like
all other installations and one that does only log collection.  The
log collection one has it's own init, config and pid file.  This way I
can rely on packaged config for local logging being identical in
production and a secondary package for log collection that only
includes the conf and init script depending on the standard rsyslog
package.

/usr/sbin/rsyslogd -c4 -f /etc/rsyslog-collector.conf -i
/var/run/rsyslogd-collector.pid


On Mon, Jan 11, 2010 at 10:42 AM,  <david at lang.hm> wrote:
> On Mon, 11 Jan 2010, Dirk H. Schulz wrote:
>
>> Hi folks,
>>
>> I am running two central logservers using rsyslog that several dozen
>> servers report to (mostly also rsyslog).
>>
>> The ?central logservers are writing everything into a ?database and
>> additionally into local logfiles.
>>
>> I would like to change configuration in a way that only local messages
>> are written to local logfiles, and all messages (local and received from
>> remote servers) into the database.
>
> yes, I do something similar to this on my systems.
>
> All logs except local logs get written to local files, all local logs get
> sent over the network (at which point they then get picked up as remote
> logs), and all logs (local or remote) get sent to a remote system.
>
> :fromhost, !isequal, "127.0.0.1" ? ? ? ?/var/log/messages;TraditionalFormat
> :fromhost, isequal, "127.0.0.1" ? ? ? ? @192.168.1.8;TraditionalForwardFormat
> *.* ? ? ? ? ? ? @192.168.1.2
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From sepperlot at googlemail.com  Tue Jan 12 16:16:30 2010
From: sepperlot at googlemail.com (Sepperlot)
Date: Tue, 12 Jan 2010 16:16:30 +0100
Subject: [rsyslog] Only log from network devices to database
Message-ID: <4B4C924E.200@googlemail.com>

Hello.

I'm trying to log messages from various network devices to rsyslog and
write them into a database.
Therefore I use a setup as described in
http://www.rsyslog.com/doc-rsyslog_mysql.html

My (simple) rsyslog.conf contains the following:

$ModLoad imudp
$UDPServerAddress x.x.x.x
$UDPServerRun 1514 # standard port is used by syslog-ng

$ModLoad ommysql
*.*             :ommysql:localhost,DBNAME,DBUSER,DBPASS


This writes all arriving log messages to the database and I can watch
them with phplogcon. Up to here everything is ok and works.

Now I only want to log messages from specific network devices identified
by ip address but I'm totaly lost when it comes to combine filter
conditions and actions. I've tried

:fromhost-ip, isequal "IP.IP.IP.IP" \
:ommysql:localhost,DBNAME,DBUSER,DBPASS

*.* :fromhost-ip, isequal "IP.IP.IP.IP" \
:ommysql:localhost,DBNAME,DBUSER,DBPASS

but obvious this is BS ;)
Goal is to log only network devices and maybe later log different
devices to different databases.

The backslash is added by me only in this mail. The commands are all in
one line.

Any help is appreciated.

Best regards
Sebastian


From rgerhards at hq.adiscon.com  Tue Jan 12 17:37:09 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jan 2010 17:37:09 +0100
Subject: [rsyslog] Only log from network devices to database
References: <4B4C924E.200@googlemail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036B7@GRFEXC.intern.adiscon.com>

The config does not look obviously wrong to me (but I am bad at catching
errors...). A good suggestion is to write a debug log, it will tell you in
detail what happened during the filter evaluation.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Sepperlot
> Sent: Tuesday, January 12, 2010 4:17 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Only log from network devices to database
> 
> Hello.
> 
> I'm trying to log messages from various network devices to rsyslog and
> write them into a database.
> Therefore I use a setup as described in
> http://www.rsyslog.com/doc-rsyslog_mysql.html
> 
> My (simple) rsyslog.conf contains the following:
> 
> $ModLoad imudp
> $UDPServerAddress x.x.x.x
> $UDPServerRun 1514 # standard port is used by syslog-ng
> 
> $ModLoad ommysql
> *.*             :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> 
> This writes all arriving log messages to the database and I can watch
> them with phplogcon. Up to here everything is ok and works.
> 
> Now I only want to log messages from specific network devices
> identified
> by ip address but I'm totaly lost when it comes to combine filter
> conditions and actions. I've tried
> 
> :fromhost-ip, isequal "IP.IP.IP.IP" \
> :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> *.* :fromhost-ip, isequal "IP.IP.IP.IP" \
> :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> but obvious this is BS ;)
> Goal is to log only network devices and maybe later log different
> devices to different databases.
> 
> The backslash is added by me only in this mail. The commands are all in
> one line.
> 
> Any help is appreciated.
> 
> Best regards
> Sebastian
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From xkubina at fi.muni.cz  Wed Jan 13 12:16:06 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 12:16:06 +0100
Subject: [rsyslog] How to add new configuration option
Message-ID: <4B4DAB76.7070201@fi.muni.cz>

Hi,

I would appreciate any help with adding support for a new configuration 
directive. I have done some
code and I need now something like:
$AddClientCN [on/off].
I have read the sources to find out how rsyslog processes conf file. 
There is some linked list with
known commands. I think that it is enough to add new item to this list 
but I don't know how.
Is this my idea right?

Thanks for any help.

Regards,

Tomas


From rgerhards at hq.adiscon.com  Wed Jan 13 12:17:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 12:17:59 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>

Hi Tomas,

it's probably the simplest if you post your code so that I can give you the
relevant hints.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Wednesday, January 13, 2010 12:16 PM
> To: rsyslog-users
> Subject: [rsyslog] How to add new configuration option
> 
> Hi,
> 
> I would appreciate any help with adding support for a new configuration
> directive. I have done some
> code and I need now something like:
> $AddClientCN [on/off].
> I have read the sources to find out how rsyslog processes conf file.
> There is some linked list with
> known commands. I think that it is enough to add new item to this list
> but I don't know how.
> Is this my idea right?
> 
> Thanks for any help.
> 
> Regards,
> 
> Tomas
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 13 12:43:11 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 12:43:11 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>

Hi all,

I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
contains a number of bug fixes, some of them important for some environments.
As usual for a beta, it does not contain anything else but fixes. The full
lest can be seen in the change log.

Please note that it is my intent do replace the current (instable ;))
v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
get a few thumbs up, I may be able to accelerate promoting it to stable.

An update for the current master branch will happen soon.

ChangeLog:
http://www.rsyslog.com/Article435.phtml

Download:
http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml

I hope this release is useful.

Rainer 



From xkubina at fi.muni.cz  Wed Jan 13 14:02:31 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 14:02:31 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
Message-ID: <4B4DC467.5000903@fi.muni.cz>

Rainer Gerhards wrote:
> Hi Tomas,
>
> it's probably the simplest if you post your code so that I can give you the
> relevant hints.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
>> Sent: Wednesday, January 13, 2010 12:16 PM
>> To: rsyslog-users
>> Subject: [rsyslog] How to add new configuration option
>>
>> Hi,
>>
>> I would appreciate any help with adding support for a new configuration
>> directive. I have done some
>> code and I need now something like:
>> $AddClientCN [on/off].
>> I have read the sources to find out how rsyslog processes conf file.
>> There is some linked list with
>> known commands. I think that it is enough to add new item to this list
>> but I don't know how.
>> Is this my idea right?
>>
>> Thanks for any help.
>>
>> Regards,
>>
>> Tomas
>>     
>   
Hi Rainer,

the modified files are attached. The alternative code is marked by #if 
statement.
I had to try to do this modification because the project, I am 
interested in, needs
to verify client's authentication. I realize that the patch is something 
like a hack,
because the rsyslog's architecture doesn't provide this feature (adding 
client CN to
syslog message) and it is not proper solution, but for our needs it is 
enough.
BTW I use this templete:
template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME% 
%syslogtag%%msg%\n";

I have done a similar code for adding client principal for imgssapi.

Thanks for help.

Regards,

Tomas



From r.bhatia at ipax.at  Wed Jan 13 14:17:04 2010
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Jan 2010 14:17:04 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <4B4DC467.5000903@fi.muni.cz>
References: <4B4DAB76.7070201@fi.muni.cz>	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
	<4B4DC467.5000903@fi.muni.cz>
Message-ID: <4B4DC7D0.30300@ipax.at>

-ENOATTACHMENT

the mailinglist strips off this stuff :)

cheers,

On 01/13/2010 02:02 PM, Tomas Kubina wrote:
> the modified files are attached. The alternative code is marked by #if
> statement.
> I had to try to do this modification because the project, I am
> interested in, needs
> to verify client's authentication. I realize that the patch is something
> like a hack,
> because the rsyslog's architecture doesn't provide this feature (adding
> client CN to
> syslog message) and it is not proper solution, but for our needs it is
> enough.
> BTW I use this templete:
> template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
> %syslogtag%%msg%\n";
> 
> I have done a similar code for adding client principal for imgssapi.


-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From xkubina at fi.muni.cz  Wed Jan 13 14:48:33 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 14:48:33 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
Message-ID: <4B4DCF31.6090105@fi.muni.cz>

Rainer Gerhards wrote:
> Hi Tomas,
>
> it's probably the simplest if you post your code so that I can give you the
> relevant hints.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
>> Sent: Wednesday, January 13, 2010 12:16 PM
>> To: rsyslog-users
>> Subject: [rsyslog] How to add new configuration option
>>
>> Hi,
>>
>> I would appreciate any help with adding support for a new configuration
>> directive. I have done some
>> code and I need now something like:
>> $AddClientCN [on/off].
>> I have read the sources to find out how rsyslog processes conf file.
>> There is some linked list with
>> known commands. I think that it is enough to add new item to this list
>> but I don't know how.
>> Is this my idea right?
>>
>> Thanks for any help.
>>
>> Regards,
>>
>> Tomas
>>     
>   
Hi Rainer,

the modified files are attached. The alternative code is marked by #if
statement. I had to try to do this modification because the project,
I am interested in, needs to verify client's authentication. I realize 
that the patch is something  like a hack, because the rsyslog's 
architecture doesn't provide this feature (adding client CN to
syslog message) and it is not proper solution, but for our needs it is
enough.
BTW I use this templete:
template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
%syslogtag%%msg%\n";

I have done a similar code for adding client principal for imgssapi.

Thanks for help.

Regards,

Tomas

FILES:

nsd_gtls.c

static rsRetVal
AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew)
{
	DEFiRet;
	int gnuRet;
	nsd_gtls_t *pNew = NULL;
	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;

         ISOBJ_TYPE_assert((pThis), nsd_gtls);
	CHKiRet(nsd_gtlsConstruct(&pNew)); // TODO: prevent construct/destruct!
	CHKiRet(nsd_ptcp.Destruct(&pNew->pTcp));
	CHKiRet(nsd_ptcp.AcceptConnReq(pThis->pTcp, &pNew->pTcp));
	
	if(pThis->iMode == 0) {
		/* we are in non-TLS mode, so we are done */
		*ppNew = (nsd_t*) pNew;
		FINALIZE;
	}

	/* if we reach this point, we are in TLS mode */
	CHKiRet(gtlsInitSession(pNew));
	gtlsSetTransportPtr(pNew, ((nsd_ptcp_t*) (pNew->pTcp))->sock);
	pNew->authMode = pThis->authMode;
	pNew->pPermPeers = pThis->pPermPeers;

	/* we now do the handshake. This is a bit complicated, because we are
	 * on non-blocking sockets. Usually, the handshake will not complete
	 * immediately, so that we need to retry it some time later.
	 */
	gnuRet = gnutls_handshake(pNew->sess);
	if(gnuRet == GNUTLS_E_AGAIN || gnuRet == GNUTLS_E_INTERRUPTED) {
		pNew->rtryCall = gtlsRtry_handshake;
		dbgprintf("GnuTLS handshake does not complete immediately - setting to 
retry (this is OK and normal)\n");
	} else if(gnuRet == 0) {
		/* we got a handshake, now check authorization */
		CHKiRet(gtlsChkPeerAuth(pNew));
	} else {
		ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR);
	}

	pNew->iMode = 1; /* this session is now in TLS mode! */
#if 1
         pNew->clientCNValid = 0;
#endif
	*ppNew = (nsd_t*) pNew;

finalize_it:
	if(iRet != RS_RET_OK) {
		if(pNew != NULL)
			nsd_gtlsDestruct(&pNew);
	}
	RETiRet;
}

static rsRetVal
Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
{
	DEFiRet;
	ssize_t iBytesCopy; /* how many bytes are to be copied to the client 
buffer? */
	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
	ISOBJ_TYPE_assert(pThis, nsd_gtls);
#if 1
         cstr_t *pstrCN = NULL;
         const gnutls_datum *cert_list;
	unsigned int cert_list_size = 0;
         gnutls_x509_crt cert;
         int len = 0;
         char *buf_temp;
#endif
	if(pThis->bAbortConn)
		ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);

	if(pThis->iMode == 0) {
		CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
		FINALIZE;
	}

	/* --- in TLS mode now --- */

	/* Buffer logic applies only if we are in TLS mode. Here we
	 * assume that we will switch from plain to TLS, but never back. This
	 * assumption may be unsafe, but it is the model for the time being and I
	 * do not see any valid reason why we should switch back to plain TCP after
	 * we were in TLS mode. However, in that case we may lose something that
	 * is already in the receive buffer ... risk accepted. -- rgerhards, 
2008-06-23
	 */

	if(pThis->pszRcvBuf == NULL) {
		/* we have no buffer, so we need to malloc one */
		CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
		pThis->lenRcvBuf = -1;
	}

	/* now check if we have something in our buffer. If so, we satisfy
	 * the request from buffer contents.
	 */
	if(pThis->lenRcvBuf == -1) { /* no data present, must read */
		CHKiRet(gtlsRecordRecv(pThis));
	}

	if(pThis->lenRcvBuf == 0) { /* EOS */
		*pLenBuf = 0;
		/* in this case, we also need to free the receive buffer, if we
		 * allocated one. -- rgerhards, 2008-12-03
		 */
		if(pThis->pszRcvBuf != NULL) {
			free(pThis->pszRcvBuf);
			pThis->pszRcvBuf = NULL;
		}
		ABORT_FINALIZE(RS_RET_CLOSED);
	}

	/* if we reach this point, data is present in the buffer and must be 
copied */
	iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
	if(iBytesCopy > *pLenBuf) {
		iBytesCopy = *pLenBuf;
	} else {
		pThis->lenRcvBuf = -1; /* buffer will be emptied below */
	}
#if 1
         if (pThis->clientCNValid != 1)
         {
           cert_list = gnutls_certificate_get_peers(pThis->sess, 
&cert_list_size);

	  if(cert_list_size > 0)
           {
	    // we only print information about the first certificate
	    gnutls_x509_crt_init(&cert);
	    gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER);

             CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));

             len = snprintf(NULL, 0, "CN:%s ", (char*)cstrGetSzStr(pstrCN));
             if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char))) )
               return -1;

             snprintf(pThis->clientCN, len + 1, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
             pThis->clientCN[len] = '\0';
             pThis->clientCNLen = len + 1;

             pThis->clientCNValid = 1;
           }
         }

         iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ? 
iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;

         buf_temp = (char*)malloc(iBytesCopy);

         if (buf_temp)
         {
           memset(buf_temp, 0, iBytesCopy);
           strncpy(buf_temp, pThis->clientCN, pThis->clientCNLen);
           strncat(buf_temp, pThis->pszRcvBuf, pThis->lenRcvBuf);
           buf_temp[iBytesCopy] ='\0';
         }

         memset(pBuf, 0, *pLenBuf);
	memcpy(pBuf, buf_temp, iBytesCopy);

         if (buf_temp)
           free(buf_temp);
#else
         memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, iBytesCopy);
#endif	
         pThis->ptrRcvBuf += iBytesCopy;
	*pLenBuf = iBytesCopy;

finalize_it:
	dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf 
%d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
	RETiRet;
}

tcps_sess.c

static rsRetVal
Close(tcps_sess_t *pThis)
{
	DEFiRet;

	ISOBJ_TYPE_assert(pThis, tcps_sess);
	netstrm.Destruct(&pThis->pStrm);
	if(pThis->fromHost != NULL) {
		prop.Destruct(&pThis->fromHost);
	}
	if(pThis->fromHostIP != NULL)
		prop.Destruct(&pThis->fromHostIP);
#if 1
        if(pThis->clientPrincipal != NULL)
		free(pThis->clientPrincipal);
#endif
	RETiRet;
}

tcps_sess.h

/* the tcps_sess object */
struct tcps_sess_s {
	BEGINobjInstance;	/* Data to implement generic object - MUST be the 
first data element! */
	tcpsrv_t *pSrv;	/* pointer back to my server (e.g. for callbacks) */
	tcpLstnPortList_t *pLstnInfo;	/* pointer back to listener info */
	netstrm_t *pStrm;
	int iMsg;		 /* index of next char to store in msg */
	int bAtStrtOfFram;	/* are we at the very beginning of a new frame? */
	enum {
		eAtStrtFram,
		eInOctetCnt,
		eInMsg
	} inputState;		/* our current state */
	int iOctetsRemain;	/* Number of Octets remaining in message */
	TCPFRAMINGMODE eFraming;
	uchar *pMsg;		/* message (fragment) received */
	prop_t *fromHost;	/* host name we received messages from */
	prop_t *fromHostIP;
	void *pUsr;		/* a user-pointer */
#if 1
         char *clientPrincipal;    /* client principal */
         int clientPrincipalLen;
#endif
	rsRetVal (*DoSubmitMessage)(tcps_sess_t*, uchar*, int); /* submit 
message callback */
};




From rgerhards at hq.adiscon.com  Wed Jan 13 15:27:31 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 15:27:31 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
	<4B4DCF31.6090105@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>

Thanks for the code. Unfortunately, adding the config switch to it is not
quite easy in that case (good I asked for the actual code). I'd say that you
best do it similar to the other config directives, like the authentication
mode. They actual directives are in the upper level code (imtcp/omfwd).
There, they are shuffled over to the instance data, which goes along with
each of the configured listeners/sender. Then, when a new network stream is
created, the params are passed down to the generic stream interface and there
passed down to the selected stream driver, which finally stores and acts on
them. It's clumpsy and quite some work, but that is what is needed for the
old config system. You probably need to add around 50 to 100 lines of code
altogether to the various files. It's not complex, but easy to forget
something. Best start by a directive (like $..AuthMode), see how it is
handled (and passed down) in imtcp and work your way down the stack ;)

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Wednesday, January 13, 2010 2:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to add new configuration option
> 
> Rainer Gerhards wrote:
> > Hi Tomas,
> >
> > it's probably the simplest if you post your code so that I 
> can give you the
> > relevant hints.
> >
> > Rainer
> >
> >   
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> >> Sent: Wednesday, January 13, 2010 12:16 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] How to add new configuration option
> >>
> >> Hi,
> >>
> >> I would appreciate any help with adding support for a new 
> configuration
> >> directive. I have done some
> >> code and I need now something like:
> >> $AddClientCN [on/off].
> >> I have read the sources to find out how rsyslog processes 
> conf file.
> >> There is some linked list with
> >> known commands. I think that it is enough to add new item 
> to this list
> >> but I don't know how.
> >> Is this my idea right?
> >>
> >> Thanks for any help.
> >>
> >> Regards,
> >>
> >> Tomas
> >>     
> >   
> Hi Rainer,
> 
> the modified files are attached. The alternative code is marked by #if
> statement. I had to try to do this modification because the project,
> I am interested in, needs to verify client's authentication. 
> I realize 
> that the patch is something  like a hack, because the rsyslog's 
> architecture doesn't provide this feature (adding client CN to
> syslog message) and it is not proper solution, but for our needs it is
> enough.
> BTW I use this templete:
> template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
> %syslogtag%%msg%\n";
> 
> I have done a similar code for adding client principal for imgssapi.
> 
> Thanks for help.
> 
> Regards,
> 
> Tomas
> 
> FILES:
> 
> nsd_gtls.c
> 
> static rsRetVal
> AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew)
> {
> 	DEFiRet;
> 	int gnuRet;
> 	nsd_gtls_t *pNew = NULL;
> 	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 
>          ISOBJ_TYPE_assert((pThis), nsd_gtls);
> 	CHKiRet(nsd_gtlsConstruct(&pNew)); // TODO: prevent 
> construct/destruct!
> 	CHKiRet(nsd_ptcp.Destruct(&pNew->pTcp));
> 	CHKiRet(nsd_ptcp.AcceptConnReq(pThis->pTcp, &pNew->pTcp));
> 	
> 	if(pThis->iMode == 0) {
> 		/* we are in non-TLS mode, so we are done */
> 		*ppNew = (nsd_t*) pNew;
> 		FINALIZE;
> 	}
> 
> 	/* if we reach this point, we are in TLS mode */
> 	CHKiRet(gtlsInitSession(pNew));
> 	gtlsSetTransportPtr(pNew, ((nsd_ptcp_t*) (pNew->pTcp))->sock);
> 	pNew->authMode = pThis->authMode;
> 	pNew->pPermPeers = pThis->pPermPeers;
> 
> 	/* we now do the handshake. This is a bit complicated, 
> because we are
> 	 * on non-blocking sockets. Usually, the handshake will 
> not complete
> 	 * immediately, so that we need to retry it some time later.
> 	 */
> 	gnuRet = gnutls_handshake(pNew->sess);
> 	if(gnuRet == GNUTLS_E_AGAIN || gnuRet == GNUTLS_E_INTERRUPTED) {
> 		pNew->rtryCall = gtlsRtry_handshake;
> 		dbgprintf("GnuTLS handshake does not complete 
> immediately - setting to 
> retry (this is OK and normal)\n");
> 	} else if(gnuRet == 0) {
> 		/* we got a handshake, now check authorization */
> 		CHKiRet(gtlsChkPeerAuth(pNew));
> 	} else {
> 		ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR);
> 	}
> 
> 	pNew->iMode = 1; /* this session is now in TLS mode! */
> #if 1
>          pNew->clientCNValid = 0;
> #endif
> 	*ppNew = (nsd_t*) pNew;
> 
> finalize_it:
> 	if(iRet != RS_RET_OK) {
> 		if(pNew != NULL)
> 			nsd_gtlsDestruct(&pNew);
> 	}
> 	RETiRet;
> }
> 
> static rsRetVal
> Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
> {
> 	DEFiRet;
> 	ssize_t iBytesCopy; /* how many bytes are to be copied 
> to the client 
> buffer? */
> 	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 	ISOBJ_TYPE_assert(pThis, nsd_gtls);
> #if 1
>          cstr_t *pstrCN = NULL;
>          const gnutls_datum *cert_list;
> 	unsigned int cert_list_size = 0;
>          gnutls_x509_crt cert;
>          int len = 0;
>          char *buf_temp;
> #endif
> 	if(pThis->bAbortConn)
> 		ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);
> 
> 	if(pThis->iMode == 0) {
> 		CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
> 		FINALIZE;
> 	}
> 
> 	/* --- in TLS mode now --- */
> 
> 	/* Buffer logic applies only if we are in TLS mode. Here we
> 	 * assume that we will switch from plain to TLS, but 
> never back. This
> 	 * assumption may be unsafe, but it is the model for 
> the time being and I
> 	 * do not see any valid reason why we should switch 
> back to plain TCP after
> 	 * we were in TLS mode. However, in that case we may 
> lose something that
> 	 * is already in the receive buffer ... risk accepted. 
> -- rgerhards, 
> 2008-06-23
> 	 */
> 
> 	if(pThis->pszRcvBuf == NULL) {
> 		/* we have no buffer, so we need to malloc one */
> 		CHKmalloc(pThis->pszRcvBuf = 
> MALLOC(NSD_GTLS_MAX_RCVBUF));
> 		pThis->lenRcvBuf = -1;
> 	}
> 
> 	/* now check if we have something in our buffer. If so, 
> we satisfy
> 	 * the request from buffer contents.
> 	 */
> 	if(pThis->lenRcvBuf == -1) { /* no data present, must read */
> 		CHKiRet(gtlsRecordRecv(pThis));
> 	}
> 
> 	if(pThis->lenRcvBuf == 0) { /* EOS */
> 		*pLenBuf = 0;
> 		/* in this case, we also need to free the 
> receive buffer, if we
> 		 * allocated one. -- rgerhards, 2008-12-03
> 		 */
> 		if(pThis->pszRcvBuf != NULL) {
> 			free(pThis->pszRcvBuf);
> 			pThis->pszRcvBuf = NULL;
> 		}
> 		ABORT_FINALIZE(RS_RET_CLOSED);
> 	}
> 
> 	/* if we reach this point, data is present in the 
> buffer and must be 
> copied */
> 	iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
> 	if(iBytesCopy > *pLenBuf) {
> 		iBytesCopy = *pLenBuf;
> 	} else {
> 		pThis->lenRcvBuf = -1; /* buffer will be 
> emptied below */
> 	}
> #if 1
>          if (pThis->clientCNValid != 1)
>          {
>            cert_list = gnutls_certificate_get_peers(pThis->sess, 
> &cert_list_size);
> 
> 	  if(cert_list_size > 0)
>            {
> 	    // we only print information about the first certificate
> 	    gnutls_x509_crt_init(&cert);
> 	    gnutls_x509_crt_import(cert, &cert_list[0], 
> GNUTLS_X509_FMT_DER);
> 
>              CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));
> 
>              len = snprintf(NULL, 0, "CN:%s ", 
> (char*)cstrGetSzStr(pstrCN));
>              if ( !(pThis->clientCN = malloc((len + 
> 1)*sizeof(char))) )
>                return -1;
> 
>              snprintf(pThis->clientCN, len + 1, "CN:%s ", 
> (char*)cstrGetSzStr(pstrCN));
>              pThis->clientCN[len] = '\0';
>              pThis->clientCNLen = len + 1;
> 
>              pThis->clientCNValid = 1;
>            }
>          }
> 
>          iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < 
> *pLenBuf ? 
> iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;
> 
>          buf_temp = (char*)malloc(iBytesCopy);
> 
>          if (buf_temp)
>          {
>            memset(buf_temp, 0, iBytesCopy);
>            strncpy(buf_temp, pThis->clientCN, pThis->clientCNLen);
>            strncat(buf_temp, pThis->pszRcvBuf, pThis->lenRcvBuf);
>            buf_temp[iBytesCopy] ='\0';
>          }
> 
>          memset(pBuf, 0, *pLenBuf);
> 	memcpy(pBuf, buf_temp, iBytesCopy);
> 
>          if (buf_temp)
>            free(buf_temp);
> #else
>          memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, 
> iBytesCopy);
> #endif	
>          pThis->ptrRcvBuf += iBytesCopy;
> 	*pLenBuf = iBytesCopy;
> 
> finalize_it:
> 	dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf 
> %d, ptrRcvBuf 
> %d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
> 	RETiRet;
> }
> 
> tcps_sess.c
> 
> static rsRetVal
> Close(tcps_sess_t *pThis)
> {
> 	DEFiRet;
> 
> 	ISOBJ_TYPE_assert(pThis, tcps_sess);
> 	netstrm.Destruct(&pThis->pStrm);
> 	if(pThis->fromHost != NULL) {
> 		prop.Destruct(&pThis->fromHost);
> 	}
> 	if(pThis->fromHostIP != NULL)
> 		prop.Destruct(&pThis->fromHostIP);
> #if 1
>         if(pThis->clientPrincipal != NULL)
> 		free(pThis->clientPrincipal);
> #endif
> 	RETiRet;
> }
> 
> tcps_sess.h
> 
> /* the tcps_sess object */
> struct tcps_sess_s {
> 	BEGINobjInstance;	/* Data to implement generic 
> object - MUST be the 
> first data element! */
> 	tcpsrv_t *pSrv;	/* pointer back to my server (e.g. for 
> callbacks) */
> 	tcpLstnPortList_t *pLstnInfo;	/* pointer back to 
> listener info */
> 	netstrm_t *pStrm;
> 	int iMsg;		 /* index of next char to store 
> in msg */
> 	int bAtStrtOfFram;	/* are we at the very beginning 
> of a new frame? */
> 	enum {
> 		eAtStrtFram,
> 		eInOctetCnt,
> 		eInMsg
> 	} inputState;		/* our current state */
> 	int iOctetsRemain;	/* Number of Octets remaining 
> in message */
> 	TCPFRAMINGMODE eFraming;
> 	uchar *pMsg;		/* message (fragment) received */
> 	prop_t *fromHost;	/* host name we received 
> messages from */
> 	prop_t *fromHostIP;
> 	void *pUsr;		/* a user-pointer */
> #if 1
>          char *clientPrincipal;    /* client principal */
>          int clientPrincipalLen;
> #endif
> 	rsRetVal (*DoSubmitMessage)(tcps_sess_t*, uchar*, int); 
> /* submit 
> message callback */
> };
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Thu Jan 14 09:37:08 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 00:37:08 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001140036380.15983@asgard.lang.hm>

I am not seeing a tag for 5.3.6 in git. Am I missing something?

David Lang

On Wed, 13 Jan 2010, Rainer Gerhards wrote:

> Hi all,
>
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
>
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> An update for the current master branch will happen soon.
>
> ChangeLog:
> http://www.rsyslog.com/Article435.phtml
>
> Download:
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
>
> I hope this release is useful.
>
> Rainer
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Thu Jan 14 10:35:16 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jan 2010 10:35:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001140036380.15983@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036C6@GRFEXC.intern.adiscon.com>

Oh, thanks - I added the tag, but forgot to push it (looks like the vacation
was too good ;)).

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 14, 2010 9:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> I am not seeing a tag for 5.3.6 in git. Am I missing something?
> 
> David Lang
> 
> On Wed, 13 Jan 2010, Rainer Gerhards wrote:
> 
> > Hi all,
> >
> > I have just released rsyslog 5.3.6, a new v5-beta. Note 
> that this version
> > contains a number of bug fixes, some of them important for 
> some environments.
> > As usual for a beta, it does not contain anything else but 
> fixes. The full
> > lest can be seen in the change log.
> >
> > Please note that it is my intent do replace the current 
> (instable ;))
> > v5-stable by this beta soon. So I would appreciate feedback 
> on 5.3.6 - if I
> > get a few thumbs up, I may be able to accelerate promoting 
> it to stable.
> >
> > An update for the current master branch will happen soon.
> >
> > ChangeLog:
> > http://www.rsyslog.com/Article435.phtml
> >
> > Download:
> > 
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> >
> > I hope this release is useful.
> >
> > Rainer
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Thu Jan 14 14:58:51 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Thu, 14 Jan 2010 07:58:51 -0600
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <20100114135851.GF1895@it.is.rice.edu>

Hi Rainer,

I have been running 5.3.6 with a PostgreSQL 8.4 backend and
it has not exhibited the problems that I saw in 5.3.5 that
caused me to roll back to 4.4.2. Thank you for the fixes to
the PostgreSQL transaction interface. I will be doing some
more testing of the new functionality but it looks good.

Regards,
Ken

On Wed, Jan 13, 2010 at 12:43:11PM +0100, Rainer Gerhards wrote:
> Hi all,
> 
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
> 
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.
> 
> An update for the current master branch will happen soon.
> 
> ChangeLog:
> http://www.rsyslog.com/Article435.phtml
> 
> Download:
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> 
> I hope this release is useful.
> 
> Rainer 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From rgerhards at hq.adiscon.com  Thu Jan 14 16:05:12 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jan 2010 16:05:12 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<20100114135851.GF1895@it.is.rice.edu>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036CC@GRFEXC.intern.adiscon.com>

Hi Ken,

Thanks for the feedback, much appreciated. Please let my know anything more
of interest that you may find out.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
> Kenneth Marshall
> Sent: Thursday, January 14, 2010 2:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Hi Rainer,
> 
> I have been running 5.3.6 with a PostgreSQL 8.4 backend and
> it has not exhibited the problems that I saw in 5.3.5 that
> caused me to roll back to 4.4.2. Thank you for the fixes to
> the PostgreSQL transaction interface. I will be doing some
> more testing of the new functionality but it looks good.
> 
> Regards,
> Ken
> 
> On Wed, Jan 13, 2010 at 12:43:11PM +0100, Rainer Gerhards wrote:
> > Hi all,
> > 
> > I have just released rsyslog 5.3.6, a new v5-beta. Note 
> that this version
> > contains a number of bug fixes, some of them important for 
> some environments.
> > As usual for a beta, it does not contain anything else but 
> fixes. The full
> > lest can be seen in the change log.
> > 
> > Please note that it is my intent do replace the current 
> (instable ;))
> > v5-stable by this beta soon. So I would appreciate feedback 
> on 5.3.6 - if I
> > get a few thumbs up, I may be able to accelerate promoting 
> it to stable.
> > 
> > An update for the current master branch will happen soon.
> > 
> > ChangeLog:
> > http://www.rsyslog.com/Article435.phtml
> > 
> > Download:
> > 
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> > 
> > I hope this release is useful.
> > 
> > Rainer 
> > 
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ryan.b.lynch at gmail.com  Thu Jan 14 16:16:43 2010
From: ryan.b.lynch at gmail.com (Ryan Lynch)
Date: Thu, 14 Jan 2010 10:16:43 -0500
Subject: [rsyslog] MySQL output module: General questions.
Message-ID: <115906d11001140716o3c45a659ndddb7a5851fc7d35@mail.gmail.com>

Hi,

I was hoping that someone with experience using the MySQL output
module, or maybe someone familiar with the source, could help me
understand a few details about the module.

1) Can ommysql use SSL connections to the database server? If not, are
there any future plans to add SSL support?

2) Do failover destinations ('$ActionExecOnlyWhenPreviousIsSuspended',
http://wiki.rsyslog.com/index.php/FailoverSyslogServer) work correctly
with ommysql? If so, how and when do connection failures
register--does the failover happen when the MySQL client fails to
execute an INSERT statement, or when the TCP socket dies, or what?

3) Does ommysql support periodically re-connection to the database server?

4) Is the retry limit for ommysql's INSERT process configurable? The
HOWTO (http://www.rsyslog.com/doc-rsyslog_mysql.html), in the section
'On Reliability...', says "If rsyslogd is unable to store a message,
it performs one retry." I assume this means the retry limit is
hard-coded--is that right?

5) How efficient is ommysql in comparison to omtcp or omrelp? I
imagine there's more overhead for the MySQL protocol, but I don't know
whether are other considerations, too. I would love to hear what
levels of load people experience, with ommysql in production, and what
kind of log volumes they handle.

Thanks!

Ryan B. Lynch
ryan.b.lynch at gmail.com


From mbiebl at gmail.com  Fri Jan 15 07:48:46 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 07:48:46 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>

2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
>
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.

So I gave 5.3.6 a try and stumbled over some rather important regressions.

Compilation and installation went fine and rsyslog started up without
an error message.
I got one log message in the syslog
Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.

But silence afterwards.

When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice

Running rsyslogd -d -c4 I got
pluto:~# rsyslogd -d -c4
rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
rsyslogd: the last error occured in
/etc/rsyslog.d/network-manager.conf, line 2:"~"
rsyslogd: warning: selector line without actions will be discarded
rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
rsyslogd: the last error occured in
/etc/rsyslog.d/network-manager.conf, line 4:"~"
rsyslogd: warning: selector line without actions will be discarded
rsyslogd: the last error occured in /etc/rsyslog.conf, line
46:"$IncludeConfig /etc/rsyslog.d/*.conf"
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

The file in question (which worked fine with 4.4.2) contains
=====
:programname, contains, "NetworkManager" /var/log/NetworkManager.log
~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
~
=====


several issues here:
first of all, the above statements no longer work.
second, rsyslog can't be killed anymore with a single SIGTERM
third, it shouldn't just silently fail. I neither got an error message
on stdout/stderr, nor in the log file.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 07:52:17 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 07:52:17 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
Message-ID: <aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Hi all,
>>
>> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
>> contains a number of bug fixes, some of them important for some environments.
>> As usual for a beta, it does not contain anything else but fixes. The full
>> lest can be seen in the change log.
>>
>> Please note that it is my intent do replace the current (instable ;))
>> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
>> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> So I gave 5.3.6 a try and stumbled over some rather important regressions.
>
> Compilation and installation went fine and rsyslog started up without
> an error message.
> I got one log message in the syslog
> Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.
>
> But silence afterwards.
>
> When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice
>
> Running rsyslogd -d -c4 I got
> pluto:~# rsyslogd -d -c4
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 2:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 4:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 46:"$IncludeConfig /etc/rsyslog.d/*.conf"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> The file in question (which worked fine with 4.4.2) contains
> =====
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> =====
>
>
> several issues here:
> first of all, the above statements no longer work.
> second, rsyslog can't be killed anymore with a single SIGTERM
> third, it shouldn't just silently fail. I neither got an error message
> on stdout/stderr, nor in the log file.

fwiw, changing -c4 to -c5 and removing network-manager.conf didn't help.

rsyslog still logs nothing and rsyslog -d is suspicously silent (i.e.
no output).

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 07:53:07 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 22:53:07 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142252150.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Hi all,
>>
>> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
>> contains a number of bug fixes, some of them important for some environments.
>> As usual for a beta, it does not contain anything else but fixes. The full
>> lest can be seen in the change log.
>>
>> Please note that it is my intent do replace the current (instable ;))
>> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
>> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> So I gave 5.3.6 a try and stumbled over some rather important regressions.
>
> Compilation and installation went fine and rsyslog started up without
> an error message.
> I got one log message in the syslog
> Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.
>
> But silence afterwards.
>
> When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice
>
> Running rsyslogd -d -c4 I got

it may not matter, but I think you need to do -c5 with 5.x

David Lang

> pluto:~# rsyslogd -d -c4
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 2:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 4:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 46:"$IncludeConfig /etc/rsyslog.d/*.conf"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> The file in question (which worked fine with 4.4.2) contains
> =====
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> =====
>
>
> several issues here:
> first of all, the above statements no longer work.
> second, rsyslog can't be killed anymore with a single SIGTERM
> third, it shouldn't just silently fail. I neither got an error message
> on stdout/stderr, nor in the log file.
>
> Cheers,
> Michael
>
>
>


From pgollucci at p6m7g8.com  Fri Jan 15 07:59:45 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Fri, 15 Jan 2010 06:59:45 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
Message-ID: <4B501261.1070901@p6m7g8.com>

Michael Biebl wrote:
>> =====
>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> ~
>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> ~
>> =====

1) rsyslogd always pukes on itself if the config file doesn't parse.
    not new.

2) Its documented that selectors syntax is not a stable API / config 
file syntax, though, the maintainer should have noted it in UPDATING.
I would have hit this tomorrow myself.

http://www.rsyslog.com/doc-rsyslog_conf_filter.html
Expression-Based Filters
Expression based filters allow filtering on arbitrary complex 
expressions, which can include boolean, arithmetic and string 
operations. Expression filters will evolve into a full configuration 
scripting language. Unfortunately, their syntax will slightly change 
during that process. So if you use them now, you need to be prepared to 
change your configuration files some time later. However, we try to 
implement the scripting facility as soon as possible (also in respect to 
stage work needed). So the window of exposure is probably not too long.

-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From mbiebl at gmail.com  Fri Jan 15 08:15:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:15:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <4B501261.1070901@p6m7g8.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
Message-ID: <aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>

2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
> Michael Biebl wrote:
>>> =====
>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>> ~
>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>> ~
>>> =====
>
> 1) rsyslogd always pukes on itself if the config file doesn't parse.
> ? ?not new.
>
> 2) Its documented that selectors syntax is not a stable API / config
> file syntax, though, the maintainer should have noted it in UPDATING.
> I would have hit this tomorrow myself.
>

You forgot the part, where I said that I remove those lines and
rsyslog still doesn't log anything.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 08:19:09 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:19:09 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
Message-ID: <aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>> Michael Biebl wrote:
>>>> =====
>>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>>> ~
>>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>>> ~
>>>> =====
>>
>> 1) rsyslogd always pukes on itself if the config file doesn't parse.
>> ? ?not new.
>>
>> 2) Its documented that selectors syntax is not a stable API / config
>> file syntax, though, the maintainer should have noted it in UPDATING.
>> I would have hit this tomorrow myself.
>>
>
> You forgot the part, where I said that I remove those lines and
> rsyslog still doesn't log anything.

And the fact that if it fails to parse, it should complain loudly and
not fail silently.

Anyway, let's see what Rainer has to say.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:23:51 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:23:51 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>>> Michael Biebl wrote:
>>>>> =====
>>>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>>>> ~
>>>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>>>> ~
>>>>> =====
>>>
>>> 1) rsyslogd always pukes on itself if the config file doesn't parse.
>>> ? ?not new.
>>>
>>> 2) Its documented that selectors syntax is not a stable API / config
>>> file syntax, though, the maintainer should have noted it in UPDATING.
>>> I would have hit this tomorrow myself.
>>>
>>
>> You forgot the part, where I said that I remove those lines and
>> rsyslog still doesn't log anything.
>
> And the fact that if it fails to parse, it should complain loudly and
> not fail silently.

unfortunantly in my experiance it doesn't complain loudly :-(

in V5 there is a new option to tell it to exit if it can't read the 
config.

I suspect that the actual config error is significantly earlier in the 
config. One thing that I frequently tripped over when switching back and 
forth was th HUPisRestart option. In V5 that's not a valid option anymore 
and needs to be removed.

can you post your full config?

David Lang

> Anyway, let's see what Rainer has to say.
>
> Michael
>
>
>

From mbiebl at gmail.com  Fri Jan 15 08:27:16 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:27:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> in V5 there is a new option to tell it to exit if it can't read the config.
>
> I suspect that the actual config error is significantly earlier in the
> config. One thing that I frequently tripped over when switching back and
> forth was th HUPisRestart option. In V5 that's not a valid option anymore
> and needs to be removed.
>
> can you post your full config?

Here is the rsyslog.conf (default Debian install)
http://paste.debian.net/56723/
and the included network-manager.conf file
http://paste.debian.net/56724/

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:34:03 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:34:03 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>> in V5 there is a new option to tell it to exit if it can't read the config.
>>
>> I suspect that the actual config error is significantly earlier in the
>> config. One thing that I frequently tripped over when switching back and
>> forth was th HUPisRestart option. In V5 that's not a valid option anymore
>> and needs to be removed.
>>
>> can you post your full config?
>
> Here is the rsyslog.conf (default Debian install)
> http://paste.debian.net/56723/
> and the included network-manager.conf file
> http://paste.debian.net/56724/

I think I just realized the problem

you have

:programname, contains, "NetworkManager" /var/log/NetworkManager.log
~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
~

when you should have

:programname, contains, "NetworkManager" /var/log/NetworkManager.log
& ~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
& ~

give that a shot.

David Lang


From pgollucci at p6m7g8.com  Fri Jan 15 08:37:41 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Fri, 15 Jan 2010 07:37:41 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
Message-ID: <4B501B45.2080006@p6m7g8.com>

david at lang.hm wrote:
  > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> 
> when you should have
> 
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> & ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> & ~
Sounds reasonable.  I typically do --

# MySQL
:programname, contains, "mysql"  ?by_prog
& :omrelp:cl.tld:2514
& ~

# REST
*.* :omrelp:cl.tld:2514



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From mbiebl at gmail.com  Fri Jan 15 08:42:10 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:42:10 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>

2010/1/15  <david at lang.hm>:

> I think I just realized the problem
>
> you have
>
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
>
> when you should have
>
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> & ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> & ~
>
> give that a shot.

The error message goes away but rsyslog still logs nothing.

Interesting fact is, that the above syntax worked fine with 4.4.2



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:50:19 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:50:19 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>
>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> & ~
>>
>> give that a shot.
>
> The error message goes away but rsyslog still logs nothing.
>
> Interesting fact is, that the above syntax worked fine with 4.4.2

You can wait for Rainer to weigh in, but if you want to test more I would 
start by commenting out everything you can and see if it works, then 
putting more stuff back until it fails.

I have noticed that V5 tends to be a bit more sensitive to invalid lines 
than v4 was, v4 seemed to just ignore what it couldn't understand and 
continue, v5 just goes nuts (very similar to what you re reporting)

you may also try adding '$AboortOnUncleanConfig yes' to the config. I 
found that gave me an error in some cases where it just wouldn't do what I 
expected without it.

David Lang


From mbiebl at gmail.com  Fri Jan 15 08:55:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:55:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>

2010/1/15  <david at lang.hm>:
>
> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
> found that gave me an error in some cases where it just wouldn't do what I
> expected without it.

Oh, the irony :-)

rsyslogd: Option value must be on or off, but is 'yes'
rsyslogd: the last error occured in /etc/rsyslog.conf, line
11:"$AbortOnUncleanConfig yes"
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]


Unfortunately, no further clues. Will try the undocument-everything
approach now.
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:01:49 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:01:49 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>>
>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>> found that gave me an error in some cases where it just wouldn't do what I
>> expected without it.
>
> Oh, the irony :-)
>
> rsyslogd: Option value must be on or off, but is 'yes'
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 11:"$AbortOnUncleanConfig yes"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
>
> Unfortunately, no further clues. Will try the undocument-everything
> approach now.

anything different if you use 'on' instead of 'yes'?

David Lang


From mbiebl at gmail.com  Fri Jan 15 09:07:08 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:07:08 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> 2010/1/15 ?<david at lang.hm>:
>>>
>>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>>> found that gave me an error in some cases where it just wouldn't do what I
>>> expected without it.
>>
>> Oh, the irony :-)
>>
>> rsyslogd: Option value must be on or off, but is 'yes'
>> rsyslogd: the last error occured in /etc/rsyslog.conf, line
>> 11:"$AbortOnUncleanConfig yes"
>> rsyslogd: CONFIG ERROR: could not interpret master config file
>> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>
>>
>> Unfortunately, no further clues. Will try the undocument-everything
>> approach now.
>
> anything different if you use 'on' instead of 'yes'?

Tried that of course. There is no relevant error message.

Further testing revealed:
-d no longer gives me the debug messages on stdout.
I had to run kill -USR1 $(cat /var/run/rsyslogd.pid) to get a verbose output.

With a tiny rsyslog.conf like
$ModLoad imuxsock
*.* /var/log/debug-rsyslog

I finally get log messages again.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 09:08:23 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:08:23 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
Message-ID: <aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/15 ?<david at lang.hm>:
>> On Fri, 15 Jan 2010, Michael Biebl wrote:
>>
>>> 2010/1/15 ?<david at lang.hm>:
>>>>
>>>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>>>> found that gave me an error in some cases where it just wouldn't do what I
>>>> expected without it.
>>>
>>> Oh, the irony :-)
>>>
>>> rsyslogd: Option value must be on or off, but is 'yes'
>>> rsyslogd: the last error occured in /etc/rsyslog.conf, line
>>> 11:"$AbortOnUncleanConfig yes"
>>> rsyslogd: CONFIG ERROR: could not interpret master config file
>>> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>>
>>>
>>> Unfortunately, no further clues. Will try the undocument-everything
>>> approach now.
>>
>> anything different if you use 'on' instead of 'yes'?
>
> Tried that of course. There is no relevant error message.
>
> Further testing revealed:
> -d no longer gives me the debug messages on stdout.
> I had to run kill -USR1 $(cat /var/run/rsyslogd.pid) to get a verbose output.
>
> With a tiny rsyslog.conf like
> $ModLoad imuxsock
> *.* /var/log/debug-rsyslog
>
> I finally get log messages again.

BTW, I'm actually surprised that you don't encounter those problems yourself.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:11:08 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:11:08 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
	<aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> BTW, I'm actually surprised that you don't encounter those problems yourself.

I'm running 5.3.5 still, I haven't had time to build a new version 
(hopefully tomorrow)

David Lang


From mrdemeanour at jackpot.uk.net  Fri Jan 15 09:19:54 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 08:19:54 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <4B50252A.1000106@jackpot.uk.net>

david at lang.hm wrote:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
> 
>> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
>>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>>>> Michael Biebl wrote:
>>>>>> ===== :programname, contains, "NetworkManager"
>>>>>> /var/log/NetworkManager.log ~ :programname, contains,
>>>>>> "wpa_supplicant" /var/log/NetworkManager.log ~ =====
>>>> 
>>>> 1) rsyslogd always pukes on itself if the config file doesn't
>>>> parse. not new.
>>>> 
>>>> 2) Its documented that selectors syntax is not a stable API /
>>>> config file syntax, though, the maintainer should have noted it
>>>> in UPDATING. I would have hit this tomorrow myself.
>>>> 
>>> 
>>> You forgot the part, where I said that I remove those lines and 
>>> rsyslog still doesn't log anything.
>> 
>> And the fact that if it fails to parse, it should complain loudly
>> and not fail silently.
> 
> unfortunantly in my experiance it doesn't complain loudly :-(
> 
> in V5 there is a new option to tell it to exit if it can't read the
> config.

Regarding failure to parse the config:

If you have a config entry of this form:

*.*	-/var/log/syslog	# Send everything else to syslog

(i.e. with a trailing comment appended using hash), it doesn't work (on
4.5.6, at least - I've observed this with other versions, but I don't
have a list). The config line is silently ignored.

The manpage says:
     "Lines  starting with a hash mark ('#') and empty lines are
      ignored."

That's fair enough; but it doesn't mention that lines containing a
trailing comment will also be ignored (silently).

Incorrect config lines should elicit a complaint in rsyslogd's output.

-- 
Jack.


From mbiebl at gmail.com  Fri Jan 15 09:44:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:44:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <4B50252A.1000106@jackpot.uk.net>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
Message-ID: <aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>

Apparently it is that line in my config file, that make rsyslog unhappy:

daemon.*;mail.*;\
       news.err;\
       *.=debug;*.=info;\
       *.=notice;*.=warn       |/dev/xconsole


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:48:35 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:48:35 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> Apparently it is that line in my config file, that make rsyslog unhappy:
>
> daemon.*;mail.*;\
>       news.err;\
>       *.=debug;*.=info;\
>       *.=notice;*.=warn       |/dev/xconsole

my ubuntu laptop doesn't have /dev/xconsole

also, I thought that | was used to execute a program and send the 
logmessage to stdin on that program

David Lang


From mbiebl at gmail.com  Fri Jan 15 09:59:13 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:59:13 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
	<alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001150059rd187152w95901c02dd37b6d9@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> Apparently it is that line in my config file, that make rsyslog unhappy:
>>
>> daemon.*;mail.*;\
>> ? ? ? news.err;\
>> ? ? ? *.=debug;*.=info;\
>> ? ? ? *.=notice;*.=warn ? ? ? |/dev/xconsole
>
> my ubuntu laptop doesn't have /dev/xconsole

That's most likely because of the switch to native upstart jobs. The
old SysV init script had an explicit
mknod -m 640 /dev/xconsole p
line. The new upstart job apparently not anymore. That is arguably a
bug in the upstart job.
(I've CC Michael Vogt, as he is responsible for rsyslog during the lucid cycle)

> also, I thought that | was used to execute a program and send the
> logmessage to stdin on that program

We might argue about the usefulness of this, but it's mostly for
historical reasons. The original syslog.conf had this entry for ages.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:25:38 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:25:38 +0100
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6 (v5-beta)
	released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>

Hi folks,

jumping right in the middle and looking at one issue at the other ;)

Please note that nothing is silently ignored. Whenever rsyslog encounters a
problem, a message is generated. HOWEVER, almost nobody ever looks at the
messages emitted from the syslog facility and so the error messages are
"lost". See also:

http://blog.gerhards.net/2009/11/rsyslog-internal-messages.html

For this, the $AbortOnUnleanConfig directive has been introduced, which will
prevent rsyslog from starting if there is any problem. As the doc for that
directive

http://www.rsyslog.com/doc-rsconf1_abortonuncleanconfig.html

says, enabling it can have harsh consequences. There is a reason that rsyslog
by default does not abort - but rather emit an error message - and continue
to function for that part of the config that is OK. This usually is much
better than aborting.

Please note that this is a long-term issue. For example, see this blog post:

http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.html

Since I have written this post, rsyslog now has a config check action and
also emits error messages (if not disabled) to stderr during startup. I have
to admit I have no further clue how I can make sure people actually look at
the error messages... (it's quite frustrating for me). Any suggestions are
very welcome.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 9:20 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> david at lang.hm wrote:
> > On Fri, 15 Jan 2010, Michael Biebl wrote:
> >
> >> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> >>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
> >>>> Michael Biebl wrote:
> >>>>>> ===== :programname, contains, "NetworkManager"
> >>>>>> /var/log/NetworkManager.log ~ :programname, contains,
> >>>>>> "wpa_supplicant" /var/log/NetworkManager.log ~ =====
> >>>>
> >>>> 1) rsyslogd always pukes on itself if the config file doesn't
> >>>> parse. not new.
> >>>>
> >>>> 2) Its documented that selectors syntax is not a stable API /
> >>>> config file syntax, though, the maintainer should have noted it
> >>>> in UPDATING. I would have hit this tomorrow myself.
> >>>>
> >>>
> >>> You forgot the part, where I said that I remove those lines and
> >>> rsyslog still doesn't log anything.
> >>
> >> And the fact that if it fails to parse, it should complain loudly
> >> and not fail silently.
> >
> > unfortunantly in my experiance it doesn't complain loudly :-(
> >
> > in V5 there is a new option to tell it to exit if it can't read the
> > config.
> 
> Regarding failure to parse the config:
> 
> If you have a config entry of this form:
> 
> *.*	-/var/log/syslog	# Send everything else to syslog
> 
> (i.e. with a trailing comment appended using hash), it doesn't work (on
> 4.5.6, at least - I've observed this with other versions, but I don't
> have a list). The config line is silently ignored.
> 
> The manpage says:
>      "Lines  starting with a hash mark ('#') and empty lines are
>       ignored."
> 
> That's fair enough; but it doesn't mention that lines containing a
> trailing comment will also be ignored (silently).
> 
> Incorrect config lines should elicit a complaint in rsyslogd's output.
> 
> --
> Jack.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 14:28:03 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:28:03 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<alpine.DEB.2.00.1001142252150.4649@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D0@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 15, 2010 7:53 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> > Running rsyslogd -d -c4 I got
> 
> it may not matter, but I think you need to do -c5 with 5.x

No - if you specify -c4, it will start up with the v4 defaults, if you do
-c5, it will start up with the v5 defaults. Nothing else. That's what -c is
for (just the defaults). Note that currently -c4 and -c5, I think, are
aquivalent.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 14:30:14 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:30:14 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D1@GRFEXC.intern.adiscon.com>

> config. One thing that I frequently tripped over when switching back
> and
> forth was th HUPisRestart option. In V5 that's not a valid option
> anymore
> and needs to be removed.

Just FYI: if $HUPisRestart is present in a v5 config, it will generate an
error message, but that's it. No harsh effects (except, of course, if you set
rsyslog to abort on error ;)).




From rgerhards at hq.adiscon.com  Fri Jan 15 14:32:43 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:32:43 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 8:42 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15  <david at lang.hm>:
> 
> > I think I just realized the problem
> >
> > you have
> >
> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> > ~
> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> > ~
> >
> > when you should have
> >
> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> > & ~
> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> > & ~
> >
> > give that a shot.
> 
> The error message goes away but rsyslog still logs nothing.
> 
> Interesting fact is, that the above syntax worked fine with 4.4.2

I don't think so, you probably ignored (did not record?) the error message.
The tilde character is an action, and an action needs to be placed after a
filter. So a tilde character just on its own in a single line is definitely a
syntax error. The engine would not know what to do with such a line.

If it generated no error in v4.4.2, *that* was a bug (will verify later).

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 14:36:54 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:36:54 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D3@GRFEXC.intern.adiscon.com>

ah, that's interesting. The code for pipes (and file output in general) has
been considerably changed, and there was a problem with pipes. I assume that
/dev/xconsole exists? If so, it may fill up and block further processing.

Just to verify, could you try the latest version from the master branch?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 9:45 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Apparently it is that line in my config file, that make rsyslog
> unhappy:
> 
> daemon.*;mail.*;\
>        news.err;\
>        *.=debug;*.=info;\
>        *.=notice;*.=warn       |/dev/xconsole
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 14:39:11 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:39:11 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 8:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15  <david at lang.hm>:
> > in V5 there is a new option to tell it to exit if it can't read the
> config.
> >
> > I suspect that the actual config error is significantly earlier in
> the
> > config. One thing that I frequently tripped over when switching back
> and
> > forth was th HUPisRestart option. In V5 that's not a valid option
> anymore
> > and needs to be removed.
> >
> > can you post your full config?
> 
> Here is the rsyslog.conf (default Debian install)
> http://paste.debian.net/56723/

If I am not mistaken, the default Debian config discards rsyslog error
messages - at least I have not spotted any rule that records syslog.err
messages anywhere...

Rainer


From mbiebl at gmail.com  Fri Jan 15 14:45:20 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 14:45:20 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> Sent: Friday, January 15, 2010 8:42 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> 2010/1/15 ?<david at lang.hm>:
>>
>> > I think I just realized the problem
>> >
>> > you have
>> >
>> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> > ~
>> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> > ~
>> >
>> > when you should have
>> >
>> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> > & ~
>> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> > & ~
>> >
>> > give that a shot.
>>
>> The error message goes away but rsyslog still logs nothing.
>>
>> Interesting fact is, that the above syntax worked fine with 4.4.2
>
> I don't think so, you probably ignored (did not record?) the error message.
> The tilde character is an action, and an action needs to be placed after a
> filter. So a tilde character just on its own in a single line is definitely a
> syntax error. The engine would not know what to do with such a line.
>
> If it generated no error in v4.4.2, *that* was a bug (will verify later).

It definitely worked with 4.4.2, i.e. the
NetworkManager/wpa_supplicant messages were discarded.

Will have to check if rsyslog wrote any error message in the syslog.

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:47:48 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:47:48 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D6@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 2:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> Sent: Friday, January 15, 2010 8:42 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> 2010/1/15 ?<david at lang.hm>:
> >>
> >> > I think I just realized the problem
> >> >
> >> > you have
> >> >
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > ~
> >> > :programname, contains, "wpa_supplicant"
> /var/log/NetworkManager.log
> >> > ~
> >> >
> >> > when you should have
> >> >
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > & ~
> >> > :programname, contains, "wpa_supplicant"
> /var/log/NetworkManager.log
> >> > & ~
> >> >
> >> > give that a shot.
> >>
> >> The error message goes away but rsyslog still logs nothing.
> >>
> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >
> > I don't think so, you probably ignored (did not record?) the error
> message.
> > The tilde character is an action, and an action needs to be placed
> after a
> > filter. So a tilde character just on its own in a single line is
> definitely a
> > syntax error. The engine would not know what to do with such a line.
> >
> > If it generated no error in v4.4.2, *that* was a bug (will verify
> later).
> 
> It definitely worked with 4.4.2, i.e. the
> NetworkManager/wpa_supplicant messages were discarded.
> 
> Will have to check if rsyslog wrote any error message in the syslog.

OK, thanks, will see where the bug in v4 is. I am right now setting up a new
Debian test env, it's probaly easiest to find the issues using the same
platform as you :)

Rainer


From mbiebl at gmail.com  Fri Jan 15 14:43:09 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 14:43:09 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150543i47e9a0b2o7ebdc0e0a47dc22d@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> Here is the rsyslog.conf (default Debian install)
>> http://paste.debian.net/56723/
>
> If I am not mistaken, the default Debian config discards rsyslog error
> messages - at least I have not spotted any rule that records syslog.err
> messages anywhere...

*.*;auth,authpriv.none		-/var/log/syslog

should catch syslog errors, right?


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:52:56 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:52:56 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150543i47e9a0b2o7ebdc0e0a47dc22d@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D7@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 2:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>
> >> Here is the rsyslog.conf (default Debian install)
> >> http://paste.debian.net/56723/
> >
> > If I am not mistaken, the default Debian config discards rsyslog
> error
> > messages - at least I have not spotted any rule that records
> syslog.err
> > messages anywhere...
> 
> *.*;auth,authpriv.none		-/var/log/syslog
> 
> should catch syslog errors, right?

Oops, I overlooked "*.*". And, indeed, it should catch them.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 15:23:34 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 15:23:34 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>

Michael,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > ~
> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >
> > I don't think so, you probably ignored (did not record?) the error
> message.
> > The tilde character is an action, and an action needs to be placed
> after a
> > filter. So a tilde character just on its own in a single line is
> definitely a
> > syntax error. The engine would not know what to do with such a line.
> >
> > If it generated no error in v4.4.2, *that* was a bug (will verify
> later).
> 
> It definitely worked with 4.4.2, i.e. the
> NetworkManager/wpa_supplicant messages were discarded.

I used a Debian 5 I had available here, ran apt-get update/upgrade and
compiled rsyslog 4.4.2 from scratch. Then I entered the first line into the
config and restarted rsyslog.

After doing so, I had the relevant errors in /var/log/syslog.

Two observations: 

a) the commands were flagged as invalid by 4.4.2
b) error messages are logged (at least up to 4.4.2)

Note that I had the statements directly in my main config. Can you verify you
get the error messages, too, when you have them directly in the main config?

I'll now see if v5 does not emit the messages...

Rainer




From rgerhards at hq.adiscon.com  Fri Jan 15 15:33:10 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 15:33:10 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D9@GRFEXC.intern.adiscon.com>

Michael, 

I could reproduce the original bug report, now a bugzilla entry:

http://bugzilla.adiscon.com/show_bug.cgi?id=169

I guess you don't see any entries in /var/log/syslog simply because rsyslog
hangs and so is unable to process any further message. I suggest you
subscribe to the bug in bugzilla.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 3:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Michael,
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > >> > :programname, contains, "NetworkManager"
> > /var/log/NetworkManager.log
> > >> > ~
> > >> Interesting fact is, that the above syntax worked fine with 4.4.2
> > >
> > > I don't think so, you probably ignored (did not record?) the error
> > message.
> > > The tilde character is an action, and an action needs to be placed
> > after a
> > > filter. So a tilde character just on its own in a single line is
> > definitely a
> > > syntax error. The engine would not know what to do with such a
> line.
> > >
> > > If it generated no error in v4.4.2, *that* was a bug (will verify
> > later).
> >
> > It definitely worked with 4.4.2, i.e. the
> > NetworkManager/wpa_supplicant messages were discarded.
> 
> I used a Debian 5 I had available here, ran apt-get update/upgrade and
> compiled rsyslog 4.4.2 from scratch. Then I entered the first line into
> the
> config and restarted rsyslog.
> 
> After doing so, I had the relevant errors in /var/log/syslog.
> 
> Two observations:
> 
> a) the commands were flagged as invalid by 4.4.2
> b) error messages are logged (at least up to 4.4.2)
> 
> Note that I had the statements directly in my main config. Can you
> verify you
> get the error messages, too, when you have them directly in the main
> config?
> 
> I'll now see if v5 does not emit the messages...
> 
> Rainer
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Fri Jan 15 16:02:30 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 16:02:30 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> >> > :programname, contains, "NetworkManager"
>> /var/log/NetworkManager.log
>> >> > ~
>> >> Interesting fact is, that the above syntax worked fine with 4.4.2
>> >
>> > I don't think so, you probably ignored (did not record?) the error
>> message.
>> > The tilde character is an action, and an action needs to be placed
>> after a
>> > filter. So a tilde character just on its own in a single line is
>> definitely a
>> > syntax error. The engine would not know what to do with such a line.
>> >
>> > If it generated no error in v4.4.2, *that* was a bug (will verify
>> later).
>>
>> It definitely worked with 4.4.2, i.e. the
>> NetworkManager/wpa_supplicant messages were discarded.
>
> I used a Debian 5 I had available here, ran apt-get update/upgrade and
> compiled rsyslog 4.4.2 from scratch. Then I entered the first line into the
> config and restarted rsyslog.
>
> After doing so, I had the relevant errors in /var/log/syslog.
>
> Two observations:
>
> a) the commands were flagged as invalid by 4.4.2
> b) error messages are logged (at least up to 4.4.2)

Yeah, false alarm from my side, sorry.

4.4.2 writes an error message about using incorrect syntax and the log
messages are not dropped when using a simple "~". Everything as it
should be :-)
So this was all a red herring.

The real problem, as you already noticed, the non-working pipe which
causes 5.3.6 to hang and not process any further message.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 16:37:15 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 16:37:15 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>

> Yeah, false alarm from my side, sorry.

No problem - much better a false alarm here and there than no alarm at all.
Thankfully, we could avoid propagating the pipe error into v4-stable, which I
consider very useful :)

Rainer

> 
> 4.4.2 writes an error message about using incorrect syntax and the log
> messages are not dropped when using a simple "~". Everything as it
> should be :-)
> So this was all a red herring.
> 
> The real problem, as you already noticed, the non-working pipe which
> causes 5.3.6 to hang and not process any further message.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Fri Jan 15 16:43:14 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 16:43:14 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150743p70182910t144d10a068c8358b@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Yeah, false alarm from my side, sorry.
>
> No problem - much better a false alarm here and there than no alarm at all.
> Thankfully, we could avoid propagating the pipe error into v4-stable, which I
> consider very useful :)

BTW, can you reproduce the problem, that -d no longer produces a
verbose output with 5.3.6?

Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 16:45:02 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 16:45:02 +0100
Subject: [rsyslog] -d doesn't work - RE:  rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150743p70182910t144d10a068c8358b@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DB@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 4:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> Yeah, false alarm from my side, sorry.
> >
> > No problem - much better a false alarm here and there than no alarm
> at all.
> > Thankfully, we could avoid propagating the pipe error into v4-stable,
> which I
> > consider very useful :)
> 
> BTW, can you reproduce the problem, that -d no longer produces a
> verbose output with 5.3.6?

Will look into that after the fix. I remember I had this issue and I think it
is fixed. Maybe I forgot to merge some change into v5-beta. This most
probably is a result of the improved runtime debugging support. Sorry I
forgot to comment on this one.

Rainer


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:04:58 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:04:58 +0000
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
Message-ID: <4B50922A.8090900@jackpot.uk.net>

Rainer Gerhards wrote:
> Hi folks,
> 
> jumping right in the middle and looking at one issue at the other ;)
> 
> Please note that nothing is silently ignored. Whenever rsyslog encounters a
> problem, a message is generated. HOWEVER, almost nobody ever looks at the
> messages emitted from the syslog facility and so the error messages are
> "lost". See also:

Rainer,

Consider please this single action line from a simple config:
   *.*                             /var/log/syslog

If that is modified as follows:
   *.*                             /var/log/syslog   # Comment goes here

then (1) no message goes to stdout; (2) nothing gets logged to
/var/log/syslog, because the action line specifying that action is
faulty. The service starts; but given that the defective action line is
the only one in the config, it might as well have failed to start,
because no log output will ever be produced. In particular, messages for 
the syslog facility will not be sent anywhere.

I call that "silent"; as far as I can see, there is absolutely no
message anywhere indicating that the service had any problems with the
config.

# rsyslogd -v
rsyslogd 4.5.6, compiled with:
         FEATURE_REGEXP:                         Yes
         FEATURE_LARGEFILE:                      Yes
         FEATURE_NETZIP (message compression):   Yes
         GSSAPI Kerberos 5 support:              No
         FEATURE_DEBUG (debug build, slow code): No
         Atomic operations supported:            Yes
         Runtime Instrumentation (slow code):    No

I think config parsing problems should be output unconditionally to
stdout; but what do I know :-) Anyway, relying on the logging service to
tell you about a problem with the logging service seems - umm -
over-confident.
-- 
Jack.



From rgerhards at hq.adiscon.com  Fri Jan 15 17:08:36 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:08:36 +0100
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
	(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
	<4B50922A.8090900@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 5:05 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
> (v5-beta) released
> 
> Rainer Gerhards wrote:
> > Hi folks,
> >
> > jumping right in the middle and looking at one issue at the other ;)
> >
> > Please note that nothing is silently ignored. Whenever rsyslog
> encounters a
> > problem, a message is generated. HOWEVER, almost nobody ever looks at
> the
> > messages emitted from the syslog facility and so the error messages
> are
> > "lost". See also:
> 
> Rainer,
> 
> Consider please this single action line from a simple config:
>    *.*                             /var/log/syslog
> 
> If that is modified as follows:
>    *.*                             /var/log/syslog   # Comment goes
> here
> 
> then (1) no message goes to stdout; (2) nothing gets logged to
> /var/log/syslog, because the action line specifying that action is
> faulty. The service starts; but given that the defective action line is
> the only one in the config, it might as well have failed to start,
> because no log output will ever be produced. In particular, messages
> for
> the syslog facility will not be sent anywhere.
> 
> I call that "silent"; as far as I can see, there is absolutely no
> message anywhere indicating that the service had any problems with the
> config.

That's a problem with the current config syntax. Interestingly hard to fix.

> 
> # rsyslogd -v
> rsyslogd 4.5.6, compiled with:
>          FEATURE_REGEXP:                         Yes
>          FEATURE_LARGEFILE:                      Yes
>          FEATURE_NETZIP (message compression):   Yes
>          GSSAPI Kerberos 5 support:              No
>          FEATURE_DEBUG (debug build, slow code): No
>          Atomic operations supported:            Yes
>          Runtime Instrumentation (slow code):    No
> 
> I think config parsing problems should be output unconditionally to
> stdout; but what do I know :-) Anyway, relying on the logging service
> to
> tell you about a problem with the logging service seems - umm -
> over-confident.

Well, that's the meat of it. So what shall I do? I am asking this question
for roughly 20 months now, and so far obviously did not get a good answer,
nor do I have one. As I wrote, we can already output error messages to
stderr. Would it really help to add another option to send them to stdout as
well? 

All suggestions on how to handle error notifications are *very* welcome.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 17:18:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:18:24 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>

Michael,

Fix now in git, links at the bug tracker:

http://bugzilla.adiscon.com/show_bug.cgi?id=169

Please let me know if it works for you (the patch is a bit trickier than it
looks, so confirmations would be good).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 4:03 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> >> > :programname, contains, "NetworkManager"
> >> /var/log/NetworkManager.log
> >> >> > ~
> >> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >> >
> >> > I don't think so, you probably ignored (did not record?) the error
> >> message.
> >> > The tilde character is an action, and an action needs to be placed
> >> after a
> >> > filter. So a tilde character just on its own in a single line is
> >> definitely a
> >> > syntax error. The engine would not know what to do with such a
> line.
> >> >
> >> > If it generated no error in v4.4.2, *that* was a bug (will verify
> >> later).
> >>
> >> It definitely worked with 4.4.2, i.e. the
> >> NetworkManager/wpa_supplicant messages were discarded.
> >
> > I used a Debian 5 I had available here, ran apt-get update/upgrade
> and
> > compiled rsyslog 4.4.2 from scratch. Then I entered the first line
> into the
> > config and restarted rsyslog.
> >
> > After doing so, I had the relevant errors in /var/log/syslog.
> >
> > Two observations:
> >
> > a) the commands were flagged as invalid by 4.4.2
> > b) error messages are logged (at least up to 4.4.2)
> 
> Yeah, false alarm from my side, sorry.
> 
> 4.4.2 writes an error message about using incorrect syntax and the log
> messages are not dropped when using a simple "~". Everything as it
> should be :-)
> So this was all a red herring.
> 
> The real problem, as you already noticed, the non-working pipe which
> causes 5.3.6 to hang and not process any further message.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:23:10 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:23:10 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
	<4B50922A.8090900@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>

 -----Original Message-----
> If that is modified as follows:
>    *.*                             /var/log/syslog   # Comment goes
> here
> 

It's even worse: data is written to the 
file "/var/log/syslog   # Comment goes here"!

Looks like I need to find a solution at least for omfile. Thanks for bringing
this issue up in this context ;)

Rainer


> then (1) no message goes to stdout; (2) nothing gets logged to
> /var/log/syslog, because the action line specifying that action is
> faulty. The service starts; but given that the defective action line is
> the only one in the config, it might as well have failed to start,
> because no log output will ever be produced. In particular, messages
> for
> the syslog facility will not be sent anywhere.
> 
> I call that "silent"; as far as I can see, there is absolutely no
> message anywhere indicating that the service had any problems with the
> config.
> 
> # rsyslogd -v
> rsyslogd 4.5.6, compiled with:
>          FEATURE_REGEXP:                         Yes
>          FEATURE_LARGEFILE:                      Yes
>          FEATURE_NETZIP (message compression):   Yes
>          GSSAPI Kerberos 5 support:              No
>          FEATURE_DEBUG (debug build, slow code): No
>          Atomic operations supported:            Yes
>          Runtime Instrumentation (slow code):    No
> 
> I think config parsing problems should be output unconditionally to
> stdout; but what do I know :-) Anyway, relying on the logging service
> to
> tell you about a problem with the logging service seems - umm -
> over-confident.
> --
> Jack.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:27:00 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:27:00 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com><4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DF@GRFEXC.intern.adiscon.com>

bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=170

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 5:23 PM
> To: rsyslog-users
> Subject: [rsyslog] comments in file actions
> 
>  -----Original Message-----
> > If that is modified as follows:
> >    *.*                             /var/log/syslog   # Comment goes
> > here
> >
> 
> It's even worse: data is written to the
> file "/var/log/syslog   # Comment goes here"!
> 
> Looks like I need to find a solution at least for omfile. Thanks for
> bringing
> this issue up in this context ;)
> 
> Rainer
> 
> 
> > then (1) no message goes to stdout; (2) nothing gets logged to
> > /var/log/syslog, because the action line specifying that action is
> > faulty. The service starts; but given that the defective action line
> is
> > the only one in the config, it might as well have failed to start,
> > because no log output will ever be produced. In particular, messages
> > for
> > the syslog facility will not be sent anywhere.
> >
> > I call that "silent"; as far as I can see, there is absolutely no
> > message anywhere indicating that the service had any problems with
> the
> > config.
> >
> > # rsyslogd -v
> > rsyslogd 4.5.6, compiled with:
> >          FEATURE_REGEXP:                         Yes
> >          FEATURE_LARGEFILE:                      Yes
> >          FEATURE_NETZIP (message compression):   Yes
> >          GSSAPI Kerberos 5 support:              No
> >          FEATURE_DEBUG (debug build, slow code): No
> >          Atomic operations supported:            Yes
> >          Runtime Instrumentation (slow code):    No
> >
> > I think config parsing problems should be output unconditionally to
> > stdout; but what do I know :-) Anyway, relying on the logging service
> > to
> > tell you about a problem with the logging service seems - umm -
> > over-confident.
> > --
> > Jack.
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:36:28 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:36:28 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com><4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DF@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E0@GRFEXC.intern.adiscon.com>

mhh... general question: would anybody object if I would not permit spaces
inside file names? (one could introduce them by using dynafiles with a clever
template if absolutely needed...).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 5:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=170
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Friday, January 15, 2010 5:23 PM
> > To: rsyslog-users
> > Subject: [rsyslog] comments in file actions
> >
> >  -----Original Message-----
> > > If that is modified as follows:
> > >    *.*                             /var/log/syslog   # Comment goes
> > > here
> > >
> >
> > It's even worse: data is written to the
> > file "/var/log/syslog   # Comment goes here"!
> >
> > Looks like I need to find a solution at least for omfile. Thanks for
> > bringing
> > this issue up in this context ;)
> >
> > Rainer
> >
> >
> > > then (1) no message goes to stdout; (2) nothing gets logged to
> > > /var/log/syslog, because the action line specifying that action is
> > > faulty. The service starts; but given that the defective action
> line
> > is
> > > the only one in the config, it might as well have failed to start,
> > > because no log output will ever be produced. In particular,
> messages
> > > for
> > > the syslog facility will not be sent anywhere.
> > >
> > > I call that "silent"; as far as I can see, there is absolutely no
> > > message anywhere indicating that the service had any problems with
> > the
> > > config.
> > >
> > > # rsyslogd -v
> > > rsyslogd 4.5.6, compiled with:
> > >          FEATURE_REGEXP:                         Yes
> > >          FEATURE_LARGEFILE:                      Yes
> > >          FEATURE_NETZIP (message compression):   Yes
> > >          GSSAPI Kerberos 5 support:              No
> > >          FEATURE_DEBUG (debug build, slow code): No
> > >          Atomic operations supported:            Yes
> > >          Runtime Instrumentation (slow code):    No
> > >
> > > I think config parsing problems should be output unconditionally to
> > > stdout; but what do I know :-) Anyway, relying on the logging
> service
> > > to
> > > tell you about a problem with the logging service seems - umm -
> > > over-confident.
> > > --
> > > Jack.
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:54:50 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:54:50 +0000
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog
 5.3.6	(v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>
Message-ID: <4B509DDA.9070207@jackpot.uk.net>

Rainer Gerhards wrote:
>> 
>> I think config parsing problems should be output unconditionally to
>>  stdout; but what do I know :-) Anyway, relying on the logging
>> service to tell you about a problem with the logging service seems
>> - umm - over-confident.
> 
> Well, that's the meat of it. So what shall I do? I am asking this
> question for roughly 20 months now, and so far obviously did not get
> a good answer, nor do I have one. As I wrote, we can already output
> error messages to stderr. Would it really help to add another option
> to send them to stdout as well?

It outputs to stderr? I don't seem to be able to make it do that (as far
as I know, both stderr and stdout should be going to the console).

With the config file containing the invalid action line, I tried this:

# rsyslogd -c4 -N1
rsyslogd: version 4.5.6, config validation run (level 1), master config
/etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

Shouldn't it at least THEN have told me there was something wrong with
the conf? I would expect anything that tells me it's a "config
validation run" to output any errors in the config to the same channel
that message gets printed on.

> 
> All suggestions on how to handle error notifications are *very*
> welcome.
> 

Output to stderr would be fine with me; but I'm not convinced it does that.


-- 
Jack.


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:58:40 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:58:40 +0000
Subject: [rsyslog] comments in file actions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
Message-ID: <4B509EC0.3050504@jackpot.uk.net>

Rainer Gerhards wrote:
> -----Original Message-----
>> If that is modified as follows: *.*
>> /var/log/syslog   # Comment goes here
>> 
> 
> It's even worse: data is written to the file "/var/log/syslog   #
> Comment goes here"!

Aaaaahhh - that finally explains some odd files lurking in /var/log!

That also explains why there's no error message on stdout/stdderr -
there's no error, as far as rsyslog is concerned. [/me feels stupid]

I would really like to be able to put comments on the ends of config
lines, as I can with many other packages. But now I know what's going on
here. Thanks!

-- 
Jack.


From mbiebl at gmail.com  Fri Jan 15 23:57:08 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 23:57:08 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> Fix now in git, links at the bug tracker:
>
> http://bugzilla.adiscon.com/show_bug.cgi?id=169
>
> Please let me know if it works for you (the patch is a bit trickier than it
> looks, so confirmations would be good).

I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
But now I'm getting a crash when rsyslog encounters the xconsole pipe
config.

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From sanelson at gmail.com  Sun Jan 17 00:46:59 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sat, 16 Jan 2010 23:46:59 +0000
Subject: [rsyslog] Difference between versions
Message-ID: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>

Hi there,

I've spent a long time picking through changelogs, but I'm afraid I
don't have a clear understanding of what to choose between versions of
rsyslog.

My platform is RHEL 5 - the distro ships with 2.0 which seems to be
both ancient and deprecated.

If I have to build a new package, it would be good to understand which
version I should choose.  For example. Rawhide has a 4.x package,
which would be the obvious starting point.

My overall objective is to be able to aggregate syslog and also Drupal
watchdog logs in a central location, and index them with Solr, to
produce a data mart/wharehouse.

I look forward to enlightenment!

TIA,

S,

-- 
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com


From david at lang.hm  Sun Jan 17 01:19:01 2010
From: david at lang.hm (david at lang.hm)
Date: Sat, 16 Jan 2010 16:19:01 -0800 (PST)
Subject: [rsyslog] Difference between versions
In-Reply-To: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>

On Sat, 16 Jan 2010, Stephen Nelson-Smith wrote:

> Hi there,
>
> I've spent a long time picking through changelogs, but I'm afraid I
> don't have a clear understanding of what to choose between versions of
> rsyslog.

2 is ancient, it's only in RHEL because that is the version that was out 
when RHEL5 was released and they never upgrade software (by policy)

3 was the stable about a year ago. This is in Debian 5

4 is after a bunch of rapid developement, it's starting to appear in some 
distros

5 is the current version, it is _much_ faster than previous version.

unfortunantly the current 5.2 'stable' release is known to be very buggy. 
5.3.6 was released a week ago, and it is believed to be the best version. 
several of us are testing it (I put in it production on a couple dozen 
machines in friday, so I should find anything that affects my environment 
by monday). The expectation is that this will replace the broken 5.2 very 
shortly.

so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you 
find anything that doesn't work, post here and you will probably get a fix 
quickly (note that the main developer is in germany, so you do have the 
time zone lag to deal with)

David Lang

> My platform is RHEL 5 - the distro ships with 2.0 which seems to be
> both ancient and deprecated.
>
> If I have to build a new package, it would be good to understand which
> version I should choose.  For example. Rawhide has a 4.x package,
> which would be the obvious starting point.
>
> My overall objective is to be able to aggregate syslog and also Drupal
> watchdog logs in a central location, and index them with Solr, to
> produce a data mart/wharehouse.
>
> I look forward to enlightenment!
>
> TIA,
>
> S,
>
>


From sanelson at gmail.com  Sun Jan 17 09:56:51 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sun, 17 Jan 2010 08:56:51 +0000
Subject: [rsyslog] Difference between versions
In-Reply-To: <alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
Message-ID: <b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>

Hi there,

> 4 is after a bunch of rapid developement, it's starting to appear in some
> distros

So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?

> 5 is the current version, it is _much_ faster than previous version.

Fast is good!

> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you
> find anything that doesn't work, post here and you will probably get a fix
> quickly (note that the main developer is in germany, so you do have the
> time zone lag to deal with)

Right - I'll get to work on a spec file.  Anyone got any gotchas to share?

S.


From david at lang.hm  Sun Jan 17 10:07:37 2010
From: david at lang.hm (david at lang.hm)
Date: Sun, 17 Jan 2010 01:07:37 -0800 (PST)
Subject: [rsyslog] Difference between versions
In-Reply-To: <b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
	<b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>

On Sun, 17 Jan 2010, Stephen Nelson-Smith wrote:

> Hi there,
>
>> 4 is after a bunch of rapid developement, it's starting to appear in some
>> distros
>
> So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?

4 will go into stable mode like 3 is now. 4 and 5 were under development 
at the same time, but the changes for 5 were so drastic that Rainer didn't 
feel comfortable doing them in the normal development version. 4 settled 
down a few months ago, it looks like 5 is settling down now. I think we 
have already hit one bug that may not end up getting fixed in 4 as the fix 
would be too invasive (when Rainer declares a version stable he is _very_ 
careful about changes to it, even if that means leaving something broken 
to avoid a substantial risk of breaking other things)

people are finding more bugs in rsyslog in recent months, most of the bugs 
that they have been finding are not new bugs, but are instead the result 
of more people useing rsyslog in more different ways (the fact that most 
distros have switched to rsyslog for their next release, if not their last 
release, has drasticly increased it's use)

>> 5 is the current version, it is _much_ faster than previous version.
>
> Fast is good!
>
>> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you
>> find anything that doesn't work, post here and you will probably get a fix
>> quickly (note that the main developer is in germany, so you do have the
>> time zone lag to deal with)
>
> Right - I'll get to work on a spec file.  Anyone got any gotchas to share?

the big thing is that it is sensitive to config file errors, make sure 
that your startup script doesn't hide such errors from the user.

David Lang


From rgerhards at hq.adiscon.com  Sun Jan 17 11:43:18 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:43:18 +0100
Subject: [rsyslog] FW:  RHEL5 rsyslog 4 rpms
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E2@GRFEXC.intern.adiscon.com>

Hi Stephen,

this message (below) may be useful for you. Maybe you can join forces...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Daniel Anson
> Sent: Thursday, January 07, 2010 7:44 PM
> To: rsyslog-users
> Subject: [rsyslog] RHEL5 rsyslog 4 rpms
> 
> If anyone is interested, an RPM engineer I know has packaged RHEL5
> rsyslog4 rpms.  These are available for public download and testing @
> http://dl.iuscommunity.org/pub/ius  Any comments can be emailed
> directly to him at ius-coredev at lists.launchpad.net
> 
> rpms are regularly packaged by him so let him know what you think.  I
> believe you just have to add the yum repo.
> 
> --Daniel M. Anson
> --Linux Systems Engineer
> 
> 
> 
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use
> of the
> individual or entity to which this message is addressed, and unless
> otherwise
> expressly indicated, is confidential and privileged information of
> Rackspace.
> Any dissemination, distribution or copying of the enclosed material is
> prohibited.
> If you receive this transmission in error, please notify us immediately
> by e-mail
> at abuse at rackspace.com, and delete the original message.
> Your cooperation is appreciated.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Sun Jan 17 11:43:36 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:43:36 +0100
Subject: [rsyslog] Difference between versions
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E3@GRFEXC.intern.adiscon.com>

David,

thanks, that's an excellent summary :)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Sunday, January 17, 2010 1:19 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Difference between versions
> 
> On Sat, 16 Jan 2010, Stephen Nelson-Smith wrote:
> 
> > Hi there,
> >
> > I've spent a long time picking through changelogs, but I'm afraid I
> > don't have a clear understanding of what to choose between versions
> of
> > rsyslog.
> 
> 2 is ancient, it's only in RHEL because that is the version that was
> out
> when RHEL5 was released and they never upgrade software (by policy)
> 
> 3 was the stable about a year ago. This is in Debian 5
> 
> 4 is after a bunch of rapid developement, it's starting to appear in
> some
> distros
> 
> 5 is the current version, it is _much_ faster than previous version.
> 
> unfortunantly the current 5.2 'stable' release is known to be very
> buggy.
> 5.3.6 was released a week ago, and it is believed to be the best
> version.
> several of us are testing it (I put in it production on a couple dozen
> machines in friday, so I should find anything that affects my
> environment
> by monday). The expectation is that this will replace the broken 5.2
> very
> shortly.
> 
> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if
> you
> find anything that doesn't work, post here and you will probably get a
> fix
> quickly (note that the main developer is in germany, so you do have the
> time zone lag to deal with)
> 
> David Lang
> 
> > My platform is RHEL 5 - the distro ships with 2.0 which seems to be
> > both ancient and deprecated.
> >
> > If I have to build a new package, it would be good to understand
> which
> > version I should choose.  For example. Rawhide has a 4.x package,
> > which would be the obvious starting point.
> >
> > My overall objective is to be able to aggregate syslog and also
> Drupal
> > watchdog logs in a central location, and index them with Solr, to
> > produce a data mart/wharehouse.
> >
> > I look forward to enlightenment!
> >
> > TIA,
> >
> > S,
> >
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Sun Jan 17 11:47:25 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:47:25 +0100
Subject: [rsyslog] Difference between versions
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com><alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm><b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
	<alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Sunday, January 17, 2010 10:08 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Difference between versions
> 
> On Sun, 17 Jan 2010, Stephen Nelson-Smith wrote:
> 
> > Hi there,
> >
> >> 4 is after a bunch of rapid developement, it's starting to appear in
> some
> >> distros
> >
> > So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?
> 
> 4 will go into stable mode like 3 is now. 4 and 5 were under
> development
> at the same time, but the changes for 5 were so drastic that Rainer
> didn't
> feel comfortable doing them in the normal development version. 4
> settled
> down a few months ago, it looks like 5 is settling down now. I think we
> have already hit one bug that may not end up getting fixed in 4 as the
> fix
> would be too invasive (when Rainer declares a version stable he is
> _very_
> careful about changes to it, even if that means leaving something
> broken
> to avoid a substantial risk of breaking other things)

Let me elaborate a bit on the v4 bug. There are some situations in the v4
queue engine, that will lead to an unclean shutdown, maybe even a hang
condition (based on the configuration). To fix this, I would need to rewrite
the v4 queue engine very much in the same way as the v5 engine is (minus some
things, but it is a *very* substantial change). Rather than spending time on
that, I accept this issue as it is, and recommend to move to v5 for those few
that are affected. Thankfully, with 5.3.6 we will have a real stable v5 soon.
Note that the v4 bug is *very unlikely* to show up - you need many queues,
various queing params (I don't know all of them out of my head) and it will
happen only very occasionally.

Rainer


From rgerhards at hq.adiscon.com  Sun Jan 17 11:51:16 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:51:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 11:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > Fix now in git, links at the bug tracker:
> >
> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >
> > Please let me know if it works for you (the patch is a bit trickier
> than it
> > looks, so confirmations would be good).
> 
> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> But now I'm getting a crash when rsyslog encounters the xconsole pipe
> config.

I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
stock Debian config?

Rainer

> 
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From sanelson at gmail.com  Sun Jan 17 12:26:42 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sun, 17 Jan 2010 11:26:42 +0000
Subject: [rsyslog] Difference between versions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
	<b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
	<alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>
Message-ID: <b6131fdc1001170326g64142ed9ta2395883695f45d4@mail.gmail.com>

Hi,
> Thankfully, with 5.3.6 we will have a real stable v5 soon.

OK - you'll have to excuse my weak git fu:

git clone git://git.adiscon.com/git/rsyslog.git
git checkout -b v5.3.6rpm v5.3.6

Is this the place to start from?  I also want to make sure I can pull
in any patches that emerge while I'm working on the package.  Do I do
that with rebase?  But will that rebase from origin/master?  Do I need
to have checked out 5.3.6-devel first?

S.


From mbiebl at gmail.com  Sun Jan 17 12:48:59 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 17 Jan 2010 12:48:59 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>

2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> Sent: Friday, January 15, 2010 11:57 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> > Michael,
>> >
>> > Fix now in git, links at the bug tracker:
>> >
>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
>> >
>> > Please let me know if it works for you (the patch is a bit trickier
>> than it
>> > looks, so confirmations would be good).
>>
>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
>> But now I'm getting a crash when rsyslog encounters the xconsole pipe
>> config.
>
> I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
> stock Debian config?

Yes. As said, I just downloaded the 5.3.6 tarball applied the
5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
got the crash. I use the default rsyslog.conf from the official debian
package.
I attached a backtrace. Hope that helps

(gdb) run -c4 -d
Starting program: /usr/sbin/rsyslogd -c4 -d
[Thread debugging using libthread_db enabled]
[New Thread 0xb7df2b70 (LWP 5162)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7df2b70 (LWP 5162)]
qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
pUsr=0x80b72c0) at queue.c:2256
2256            if(pThis->qType != QUEUETYPE_DIRECT) {
(gdb) bt full
#0  qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
pUsr=0x80b72c0) at queue.c:2256
        iRet = <value optimized out>
        iCancelStateSave = <value optimized out>
#1  0x0807cd5b in actionWriteToAction (pAction=0x80ac8d8) at
../action.c:1169
        pMsgSave = 0x0
        iRet = <value optimized out>
#2  0x0807d4fe in doActionCallAction (pAction=0x80ac8d8,
pMsg=0x80b72c0) at ../action.c:1244
No locals.
#3  actionCallAction (pAction=0x80ac8d8, pMsg=0x80b72c0) at ../action.c:1274
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113864, 0, -1210114056, 1464119912, -422385385},
              __mask_was_saved = 0}}, __pad = {0xb7df2240, 0x0,
0xb7df20d0, 0xb7feb27f}}
        __cancel_arg = 0x80ad0a0
        not_first_call = <value optimized out>
        iRet = -1210113864
#4  0x080794d7 in processMsgDoActions (pData=0xfffff815,
pParam=0xb7df20b8) at rule.c:113
        iRet = <value optimized out>
        iRetMod = <value optimized out>
#5  0x080627ba in llExecFunc (pThis=0x80ac9a0, pFunc=0x8079480
<processMsgDoActions>, pParam=0xb7df20b8) at linkedlist.c:391
        iRet = <value optimized out>
        iRetLL = <value optimized out>
        pData = 0x80ac8d8
        llCookie = 0x80ac508
        llCookiePrev = 0x0
#6  0x08079007 in processMsg (pThis=0x80ac968, pMsg=0x80b72c0) at rule.c:299
        bProcessMsg = 1
        DoActData = {bPrevWasSuspended = 0, pMsg = 0x80b72c0}
        iRet = RS_RET_OK
#7  0x080781ba in processMsgDoRules (pData=0x80ac968,
pParam=0x80b72c0) at ruleset.c:145
        iRet = <value optimized out>
#8  0x080627ba in llExecFunc (pThis=0x809eb68, pFunc=0x8078190
<processMsgDoRules>, pParam=0x80b72c0) at linkedlist.c:391
        iRet = <value optimized out>
        iRetLL = <value optimized out>
        pData = 0x80ac968
        llCookie = 0x80ac478
        llCookiePrev = 0x80ac4f8
#9  0x0807876d in processMsg (pMsg=0x80b72c0) at ruleset.c:164
        pThis = <value optimized out>
        iRet = <value optimized out>
#10 0x080506ab in msgConsumer (notNeeded=0x0, pBatch=0x809ecc8,
pbShutdownImmediate=0x80ad348) at syslogd.c:614
        i = 0
        pMsg = 0x80b72c0
        localRet = RS_RET_IO_ERROR
#11 0x08077ca4 in ConsumerReg (pThis=0x80ad338, pWti=0x809ecb0) at queue.c:1638
        iCancelStateSave = 1
        iRet = <value optimized out>
#12 0x08070526 in wtiWorker (pThis=0x809ecb0) at wti.c:286
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113200, 0, -1210113384, 1463800424, -411898089},
              __mask_was_saved = 0}}, __pad = {0xb7df2350, 0x0, 0x0, 0x809002c}}
        not_first_call = <value optimized out>
        pWtp = 0x809ebd8
        bInactivityTOOccured = <value optimized out>
        localRet = <value optimized out>
        terminateRet = RS_RET_OK
        iCancelStateSave = <value optimized out>
#13 0x08070074 in wtpWorker (arg=0x809ecb0) at wtp.c:356
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113152, 0, -1210113096, 1463726696, -411345641},
              __mask_was_saved = 0}}, __pad = {0xb7df2460, 0x0,
0xb7feff7b, 0xb7fe0cb0}}
        not_first_call = <value optimized out>
        pszDbgHdr = <value optimized out>
        thrdName = "rs:main Q:Reg", '\000' <repeats 18 times>
---Type <return> to continue, or q <return> to quit---
        pThis = 0x809ebd8
        sigSet = {__val = {2147483647, 4294967294, 4294967295 <repeats
30 times>}}
#14 0xb7f9c585 in start_thread () from /lib/i686/cmov/libpthread.so.0
No symbol table info available.
#15 0xb7f1026e in clone () from /lib/i686/cmov/libc.so.6
No symbol table info available.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Sun Jan 17 12:51:37 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 17 Jan 2010 12:51:37 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
Message-ID: <aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>

2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>>> Sent: Friday, January 15, 2010 11:57 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>>
>>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>> > Michael,
>>> >
>>> > Fix now in git, links at the bug tracker:
>>> >
>>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
>>> >
>>> > Please let me know if it works for you (the patch is a bit trickier
>>> than it
>>> > looks, so confirmations would be good).
>>>
>>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
>>> But now I'm getting a crash when rsyslog encounters the xconsole pipe
>>> config.
>>
>> I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
>> stock Debian config?
>
> Yes. As said, I just downloaded the 5.3.6 tarball applied the
> 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> got the crash. I use the default rsyslog.conf from the official debian
> package.

As an additonal hint: If I start xconsole (a process reading from
/dev/xconsole) before I start rsyslogd, then the crash does not occur.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From pgollucci at p6m7g8.com  Sun Jan 17 12:56:07 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Sun, 17 Jan 2010 06:56:07 -0500
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
Message-ID: <4B52FAD7.305@p6m7g8.com>

On 1/17/2010 6:51 AM, Michael Biebl wrote:
> As an additonal hint: If I start xconsole (a process reading from
> /dev/xconsole) before I start rsyslogd, then the crash does not occur.
Possibly related, on FreeBSD in a jail if rsyslog ever tries to write to
/dev/console, it loops in a extremely tightly loop consuming 100% of the
core its on.  [see -dn output, possibly with ktrace/kdump]

Eventually it will consume all the memory on the box and it will go boom.


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Sr. System Admin,                 Ridecharge Inc.
Consultant,                       P6M7G8 Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From ralph at crongeyer.com  Sun Jan 17 23:50:10 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Sun, 17 Jan 2010 17:50:10 -0500
Subject: [rsyslog] fromhost-ip
Message-ID: <4B539422.3020709@crongeyer.com>

Hello list,
I'm trying to send my IPOCop Firewall logs to my rsyslog server like this:

# Firewall logs #
$template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
*.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall

But I just getting this error in /var/log/syslog:

Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
Jan 17 16:49:47 log rsyslogd: the last error occured in 
/etc/rsyslog.d/remote-logs.conf, line 10
Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
will be discarded
Jan 17 16:49:47 log rsyslogd: the last error occured in 
/etc/rsyslog.conf, line 48
Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

I'm trying to log all logs from my IPCop host to 
"/var/log/server-logs/firewall/%HOSTNAME%.log" .

Can someone help me out with this?

Thanks,
Ralph

-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From pgollucci at p6m7g8.com  Mon Jan 18 00:09:22 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Sun, 17 Jan 2010 18:09:22 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B539422.3020709@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>
Message-ID: <4B5398A2.4020604@p6m7g8.com>

On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
> # Firewall logs #
> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
> 
> But I just getting this error in /var/log/syslog:
> 
> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
> swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> /etc/rsyslog.d/remote-logs.conf, line 10
> Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
> will be discarded
> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> /etc/rsyslog.conf, line 48
> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
> master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
> 
> I'm trying to log all logs from my IPCop host to 
> "/var/log/server-logs/firewall/%HOSTNAME%.log" .

I tried for 1.5 days to figure this out cutting and pasting examples
left and right.  Finally I came up with the following with works well
for me, you should be able to tweak it slightly for yourself.


$template by_prog,"/var/log/rws/%programname%.log"

:programname, regex, "^pxy.*rc\."  ?by_prog
& :omrelp:cl.dca1.rws:2514
& ~

Just sub out %programname% for %HOSTNAME%



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Sr. System Admin,                 Ridecharge Inc.
Consultant,                       P6M7G8 Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From ralph at crongeyer.com  Mon Jan 18 16:37:22 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 10:37:22 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B5398A2.4020604@p6m7g8.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
Message-ID: <4B548032.60807@crongeyer.com>

Hi Phillip,
Thanks for the response.
The %HOSTNAME% part works fine here if I do this:
$template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
*.*    -?DynFwall

However if I try to filter by IP using the "fromhost-ip" like this:
*.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

It fails to capture logs in the DynFwall template file.

I've tried to do this with the "fromhost" and the "fromhost-ip" and 
neither seem to work?

I want to have it so that a specific host IP uses a specific template.

It looks like the fromhost and the fromhost-ip arn't working at all? Or 
my config is wrong.

Dose anyone on the list have "fromhost-ip" working?

Thanks,
Ralph

Philip M. Gollucci wrote:
> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>   
>> # Firewall logs #
>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>
>> But I just getting this error in /var/log/syslog:
>>
>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
>> swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>> /etc/rsyslog.d/remote-logs.conf, line 10
>> Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
>> will be discarded
>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>> /etc/rsyslog.conf, line 48
>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
>> master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>
>> I'm trying to log all logs from my IPCop host to 
>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>     
>
> I tried for 1.5 days to figure this out cutting and pasting examples
> left and right.  Finally I came up with the following with works well
> for me, you should be able to tweak it slightly for yourself.
>
>
> $template by_prog,"/var/log/rws/%programname%.log"
>
> :programname, regex, "^pxy.*rc\."  ?by_prog
> & :omrelp:cl.dca1.rws:2514
> & ~
>
> Just sub out %programname% for %HOSTNAME%
>
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Mon Jan 18 17:24:20 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 17:24:20 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph 
> Crongeyer
> Sent: Monday, January 18, 2010 4:37 PM
> To: Philip M. Gollucci
> Cc: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> Hi Phillip,
> Thanks for the response.
> The %HOSTNAME% part works fine here if I do this:
> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> *.*    -?DynFwall

Phillip suggested the rigth thing. 
> 
> However if I try to filter by IP using the "fromhost-ip" like this:
> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

The issue is that the config is wrong. "*.*" and ":fromhost..." are both
filters. There can only be one filter in front of an action. As *.* maeans
all messages, I assume ou actually wanted to do this:

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

Which filters alls messages based on fromhost-ip.

The config format is clumpsy. I am currently talking with some folks at
Adiscon, and we will probably create a cookbook-type doc that provides
samples for some common scenarios. I guess that would be useful. Any feedback
on that effort would be welcome.

Rainer

> 
> It fails to capture logs in the DynFwall template file.
> 
> I've tried to do this with the "fromhost" and the "fromhost-ip" and 
> neither seem to work?
> 
> I want to have it so that a specific host IP uses a specific template.
> 
> It looks like the fromhost and the fromhost-ip arn't working 
> at all? Or 
> my config is wrong.
> 
> Dose anyone on the list have "fromhost-ip" working?
> 
> Thanks,
> Ralph
> 
> Philip M. Gollucci wrote:
> > On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
> >   
> >> # Firewall logs #
> >> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> >> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
> >>
> >> But I just getting this error in /var/log/syslog:
> >>
> >> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
> >> swVersion="4.4.2" x-pid="12540" 
> x-info="http://www.rsyslog.com"] (re)start
> >> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> >> /etc/rsyslog.d/remote-logs.conf, line 10
> >> Jan 17 16:49:47 log rsyslogd: warning: selector line 
> without actions 
> >> will be discarded
> >> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> >> /etc/rsyslog.conf, line 48
> >> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not 
> interpret 
> >> master config file '/etc/rsyslog.conf'. [try 
> http://www.rsyslog.com/e/2124 ]
> >>
> >> I'm trying to log all logs from my IPCop host to 
> >> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
> >>     
> >
> > I tried for 1.5 days to figure this out cutting and pasting examples
> > left and right.  Finally I came up with the following with 
> works well
> > for me, you should be able to tweak it slightly for yourself.
> >
> >
> > $template by_prog,"/var/log/rws/%programname%.log"
> >
> > :programname, regex, "^pxy.*rc\."  ?by_prog
> > & :omrelp:cl.dca1.rws:2514
> > & ~
> >
> > Just sub out %programname% for %HOSTNAME%
> >
> >
> >
> >   
> 
> 
> -- 
> Reminds me of my expedition into the wilds of Afghanistan. We 
> lost our 
> corkscrew and were compelled to live on food and water for 
> several days. - 
> WC Fields
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ralph at crongeyer.com  Mon Jan 18 18:18:18 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 12:18:18 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
Message-ID: <4B5497DA.9070405@crongeyer.com>

Hi Rainer,
Thanks for the explanation, that helps me understand how it's working.

That works, the logs are going to the correct file, however they are 
also being sent to /var/log/syslog? How can I make all the logs from my 
host "192.168.1.1" go only to the "-?DynFwall" template file?

I would like to give feedback on the cookbook let me know how I can help.

Thanks all, for your help with this.
Ralph

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com 
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph 
>> Crongeyer
>> Sent: Monday, January 18, 2010 4:37 PM
>> To: Philip M. Gollucci
>> Cc: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> Hi Phillip,
>> Thanks for the response.
>> The %HOSTNAME% part works fine here if I do this:
>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>> *.*    -?DynFwall
>>     
>
> Phillip suggested the rigth thing. 
>   
>> However if I try to filter by IP using the "fromhost-ip" like this:
>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>     
>
> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
> filters. There can only be one filter in front of an action. As *.* maeans
> all messages, I assume ou actually wanted to do this:
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>
> Which filters alls messages based on fromhost-ip.
>
> The config format is clumpsy. I am currently talking with some folks at
> Adiscon, and we will probably create a cookbook-type doc that provides
> samples for some common scenarios. I guess that would be useful. Any feedback
> on that effort would be welcome.
>
> Rainer
>
>   
>> It fails to capture logs in the DynFwall template file.
>>
>> I've tried to do this with the "fromhost" and the "fromhost-ip" and 
>> neither seem to work?
>>
>> I want to have it so that a specific host IP uses a specific template.
>>
>> It looks like the fromhost and the fromhost-ip arn't working 
>> at all? Or 
>> my config is wrong.
>>
>> Dose anyone on the list have "fromhost-ip" working?
>>
>> Thanks,
>> Ralph
>>
>> Philip M. Gollucci wrote:
>>     
>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>   
>>>       
>>>> # Firewall logs #
>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>
>>>> But I just getting this error in /var/log/syslog:
>>>>
>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
>>>> swVersion="4.4.2" x-pid="12540" 
>>>>         
>> x-info="http://www.rsyslog.com"] (re)start
>>     
>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line 
>>>>         
>> without actions 
>>     
>>>> will be discarded
>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>>>> /etc/rsyslog.conf, line 48
>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not 
>>>>         
>> interpret 
>>     
>>>> master config file '/etc/rsyslog.conf'. [try 
>>>>         
>> http://www.rsyslog.com/e/2124 ]
>>     
>>>> I'm trying to log all logs from my IPCop host to 
>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>     
>>>>         
>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>> left and right.  Finally I came up with the following with 
>>>       
>> works well
>>     
>>> for me, you should be able to tweak it slightly for yourself.
>>>
>>>
>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>
>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>> & :omrelp:cl.dca1.rws:2514
>>> & ~
>>>
>>> Just sub out %programname% for %HOSTNAME%
>>>
>>>
>>>
>>>   
>>>       
>> -- 
>> Reminds me of my expedition into the wilds of Afghanistan. We 
>> lost our 
>> corkscrew and were compelled to live on food and water for 
>> several days. - 
>> WC Fields
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 18:29:02 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 09:29:02 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B5497DA.9070405@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Hi Rainer,
> Thanks for the explanation, that helps me understand how it's working.
>
> That works, the logs are going to the correct file, however they are
> also being sent to /var/log/syslog? How can I make all the logs from my
> host "192.168.1.1" go only to the "-?DynFwall" template file?

after you tell rsyslog to put the logs in that file, you then need to tell 
rsyslog to throw the log away.

so you would do something like

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
& ~

which is logicly the same as

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
:fromhost-ip,isequal,"192.168.1.1"    ~

David Lang


> I would like to give feedback on the cookbook let me know how I can help.
>
> Thanks all, for your help with this.
> Ralph
>
> Rainer Gerhards wrote:
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>> Crongeyer
>>> Sent: Monday, January 18, 2010 4:37 PM
>>> To: Philip M. Gollucci
>>> Cc: rsyslog-users
>>> Subject: Re: [rsyslog] fromhost-ip
>>>
>>> Hi Phillip,
>>> Thanks for the response.
>>> The %HOSTNAME% part works fine here if I do this:
>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>> *.*    -?DynFwall
>>>
>>
>> Phillip suggested the rigth thing.
>>
>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>
>>
>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>> filters. There can only be one filter in front of an action. As *.* maeans
>> all messages, I assume ou actually wanted to do this:
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>
>> Which filters alls messages based on fromhost-ip.
>>
>> The config format is clumpsy. I am currently talking with some folks at
>> Adiscon, and we will probably create a cookbook-type doc that provides
>> samples for some common scenarios. I guess that would be useful. Any feedback
>> on that effort would be welcome.
>>
>> Rainer
>>
>>
>>> It fails to capture logs in the DynFwall template file.
>>>
>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>> neither seem to work?
>>>
>>> I want to have it so that a specific host IP uses a specific template.
>>>
>>> It looks like the fromhost and the fromhost-ip arn't working
>>> at all? Or
>>> my config is wrong.
>>>
>>> Dose anyone on the list have "fromhost-ip" working?
>>>
>>> Thanks,
>>> Ralph
>>>
>>> Philip M. Gollucci wrote:
>>>
>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>
>>>>
>>>>> # Firewall logs #
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>
>>>>> But I just getting this error in /var/log/syslog:
>>>>>
>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>
>>> x-info="http://www.rsyslog.com"] (re)start
>>>
>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>
>>> without actions
>>>
>>>>> will be discarded
>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>> /etc/rsyslog.conf, line 48
>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>
>>> interpret
>>>
>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>
>>> http://www.rsyslog.com/e/2124 ]
>>>
>>>>> I'm trying to log all logs from my IPCop host to
>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>
>>>>>
>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>> left and right.  Finally I came up with the following with
>>>>
>>> works well
>>>
>>>> for me, you should be able to tweak it slightly for yourself.
>>>>
>>>>
>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>
>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>> & :omrelp:cl.dca1.rws:2514
>>>> & ~
>>>>
>>>> Just sub out %programname% for %HOSTNAME%
>>>>
>>>>
>>>>
>>>>
>>>>
>>> --
>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>> lost our
>>> corkscrew and were compelled to live on food and water for
>>> several days. -
>>> WC Fields
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 18:47:03 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 12:47:03 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
Message-ID: <4B549E97.8030108@crongeyer.com>

Oh,
I tried that but I had it on the same line. So that has to be on a 
separate line?

Thanks again for the explanation that really helps me understand how 
it's working.

Thanks again for all your help with this.

Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Hi Rainer,
>> Thanks for the explanation, that helps me understand how it's working.
>>
>> That works, the logs are going to the correct file, however they are
>> also being sent to /var/log/syslog? How can I make all the logs from my
>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>     
>
> after you tell rsyslog to put the logs in that file, you then need to tell 
> rsyslog to throw the log away.
>
> so you would do something like
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
> & ~
>
> which is logicly the same as
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
> :fromhost-ip,isequal,"192.168.1.1"    ~
>
> David Lang
>
>
>   
>> I would like to give feedback on the cookbook let me know how I can help.
>>
>> Thanks all, for your help with this.
>> Ralph
>>
>> Rainer Gerhards wrote:
>>     
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>> Crongeyer
>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>> To: Philip M. Gollucci
>>>> Cc: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> Hi Phillip,
>>>> Thanks for the response.
>>>> The %HOSTNAME% part works fine here if I do this:
>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>> *.*    -?DynFwall
>>>>
>>>>         
>>> Phillip suggested the rigth thing.
>>>
>>>       
>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>>         
>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>> filters. There can only be one filter in front of an action. As *.* maeans
>>> all messages, I assume ou actually wanted to do this:
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>
>>> Which filters alls messages based on fromhost-ip.
>>>
>>> The config format is clumpsy. I am currently talking with some folks at
>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>> on that effort would be welcome.
>>>
>>> Rainer
>>>
>>>
>>>       
>>>> It fails to capture logs in the DynFwall template file.
>>>>
>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>> neither seem to work?
>>>>
>>>> I want to have it so that a specific host IP uses a specific template.
>>>>
>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>> at all? Or
>>>> my config is wrong.
>>>>
>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>
>>>> Thanks,
>>>> Ralph
>>>>
>>>> Philip M. Gollucci wrote:
>>>>
>>>>         
>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> # Firewall logs #
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>
>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>
>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>
>>>>>>             
>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>
>>>>         
>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>
>>>>>>             
>>>> without actions
>>>>
>>>>         
>>>>>> will be discarded
>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>> /etc/rsyslog.conf, line 48
>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>
>>>>>>             
>>>> interpret
>>>>
>>>>         
>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>
>>>>>>             
>>>> http://www.rsyslog.com/e/2124 ]
>>>>
>>>>         
>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>
>>>>>>
>>>>>>             
>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>> left and right.  Finally I came up with the following with
>>>>>
>>>>>           
>>>> works well
>>>>
>>>>         
>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>
>>>>>
>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>
>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>> & :omrelp:cl.dca1.rws:2514
>>>>> & ~
>>>>>
>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> --
>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>> lost our
>>>> corkscrew and were compelled to live on food and water for
>>>> several days. -
>>>> WC Fields
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 19:15:49 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 13:15:49 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B549E97.8030108@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com>
Message-ID: <4B54A555.9010007@crongeyer.com>

Ok one more question.
I have:
$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
mail.*  -?DynMail

Which logs all mail to the %HOSTNAME%.mail.log.

My guess would be:
$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail

But as Rainer explained these are both filters which won't work.

So how do I use "fromhost-ip" to send only "mail.*" logs from a 
specified host IP to the "DynMail" template?

Thanks,
Ralph

Ralph Crongeyer wrote:
> Oh,
> I tried that but I had it on the same line. So that has to be on a 
> separate line?
>
> Thanks again for the explanation that really helps me understand how 
> it's working.
>
> Thanks again for all your help with this.
>
> Ralph
>
> david at lang.hm wrote:
>   
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>   
>>     
>>> Hi Rainer,
>>> Thanks for the explanation, that helps me understand how it's working.
>>>
>>> That works, the logs are going to the correct file, however they are
>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>     
>>>       
>> after you tell rsyslog to put the logs in that file, you then need to tell 
>> rsyslog to throw the log away.
>>
>> so you would do something like
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> & ~
>>
>> which is logicly the same as
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>
>> David Lang
>>
>>
>>   
>>     
>>> I would like to give feedback on the cookbook let me know how I can help.
>>>
>>> Thanks all, for your help with this.
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>     
>>>       
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>> Crongeyer
>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>> To: Philip M. Gollucci
>>>>> Cc: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> Hi Phillip,
>>>>> Thanks for the response.
>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.*    -?DynFwall
>>>>>
>>>>>         
>>>>>           
>>>> Phillip suggested the rigth thing.
>>>>
>>>>       
>>>>         
>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>>         
>>>>>           
>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>> all messages, I assume ou actually wanted to do this:
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>> Which filters alls messages based on fromhost-ip.
>>>>
>>>> The config format is clumpsy. I am currently talking with some folks at
>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>> on that effort would be welcome.
>>>>
>>>> Rainer
>>>>
>>>>
>>>>       
>>>>         
>>>>> It fails to capture logs in the DynFwall template file.
>>>>>
>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>> neither seem to work?
>>>>>
>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>
>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>> at all? Or
>>>>> my config is wrong.
>>>>>
>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>
>>>>> Thanks,
>>>>> Ralph
>>>>>
>>>>> Philip M. Gollucci wrote:
>>>>>
>>>>>         
>>>>>           
>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>           
>>>>>>             
>>>>>>> # Firewall logs #
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>
>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>
>>>>>         
>>>>>           
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> without actions
>>>>>
>>>>>         
>>>>>           
>>>>>>> will be discarded
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> interpret
>>>>>
>>>>>         
>>>>>           
>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>
>>>>>         
>>>>>           
>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>> left and right.  Finally I came up with the following with
>>>>>>
>>>>>>           
>>>>>>             
>>>>> works well
>>>>>
>>>>>         
>>>>>           
>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>
>>>>>>
>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>
>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>> & ~
>>>>>>
>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>           
>>>>>>             
>>>>> --
>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>> lost our
>>>>> corkscrew and were compelled to live on food and water for
>>>>> several days. -
>>>>> WC Fields
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>       
>>>>         
>>>     
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 20:14:36 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 14:14:36 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54A555.9010007@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>
Message-ID: <4B54B31C.80109@crongeyer.com>

Is it possible to use "fromhost-ip" to send only "mail.*" logs from a 
specified host IP to the "DynMail" template?

I tried this:
:fromhost-ip,isequal,"192.168.1.1" & mail.* -?DynMail

But that didn't work.

How can I accomplish this?

Thanks,
Ralph
 
Ralph Crongeyer wrote:
> Ok one more question.
> I have:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  -?DynMail
>
> Which logs all mail to the %HOSTNAME%.mail.log.
>
> My guess would be:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>
> But as Rainer explained these are both filters which won't work.
>
> So how do I use "fromhost-ip" to send only "mail.*" logs from a 
> specified host IP to the "DynMail" template?
>
> Thanks,
> Ralph
>
> Ralph Crongeyer wrote:
>   
>> Oh,
>> I tried that but I had it on the same line. So that has to be on a 
>> separate line?
>>
>> Thanks again for the explanation that really helps me understand how 
>> it's working.
>>
>> Thanks again for all your help with this.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>   
>>     
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>   
>>>     
>>>       
>>>> Hi Rainer,
>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>
>>>> That works, the logs are going to the correct file, however they are
>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>     
>>>>       
>>>>         
>>> after you tell rsyslog to put the logs in that file, you then need to tell 
>>> rsyslog to throw the log away.
>>>
>>> so you would do something like
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> & ~
>>>
>>> which is logicly the same as
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>
>>> David Lang
>>>
>>>
>>>   
>>>     
>>>       
>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>
>>>> Thanks all, for your help with this.
>>>> Ralph
>>>>
>>>> Rainer Gerhards wrote:
>>>>     
>>>>       
>>>>         
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>> Crongeyer
>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>> To: Philip M. Gollucci
>>>>>> Cc: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> Hi Phillip,
>>>>>> Thanks for the response.
>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.*    -?DynFwall
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> Phillip suggested the rigth thing.
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>> all messages, I assume ou actually wanted to do this:
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>> Which filters alls messages based on fromhost-ip.
>>>>>
>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>> on that effort would be welcome.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>
>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>> neither seem to work?
>>>>>>
>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>
>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>> at all? Or
>>>>>> my config is wrong.
>>>>>>
>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>
>>>>>> Thanks,
>>>>>> Ralph
>>>>>>
>>>>>> Philip M. Gollucci wrote:
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>>>> # Firewall logs #
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>
>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> without actions
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> will be discarded
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> interpret
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> works well
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>
>>>>>>>
>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>
>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>> & ~
>>>>>>>
>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> --
>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>> lost our
>>>>>> corkscrew and were compelled to live on food and water for
>>>>>> several days. -
>>>>>> WC Fields
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>     
>>>>       
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>   
>>>     
>>>       
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 20:49:39 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 11:49:39 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B549E97.8030108@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181149010.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Oh,
> I tried that but I had it on the same line. So that has to be on a
> separate line?

yes, one line is a filter plus an action

haveing two filters on a line (like you initially tried) doesn't work, 
neither does having two actions on a line.

David Lang

> Thanks again for the explanation that really helps me understand how
> it's working.
>
> Thanks again for all your help with this.
>
> Ralph
>
> david at lang.hm wrote:
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>
>>> Hi Rainer,
>>> Thanks for the explanation, that helps me understand how it's working.
>>>
>>> That works, the logs are going to the correct file, however they are
>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>
>>
>> after you tell rsyslog to put the logs in that file, you then need to tell
>> rsyslog to throw the log away.
>>
>> so you would do something like
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> & ~
>>
>> which is logicly the same as
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>
>> David Lang
>>
>>
>>
>>> I would like to give feedback on the cookbook let me know how I can help.
>>>
>>> Thanks all, for your help with this.
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>> Crongeyer
>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>> To: Philip M. Gollucci
>>>>> Cc: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> Hi Phillip,
>>>>> Thanks for the response.
>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.*    -?DynFwall
>>>>>
>>>>>
>>>> Phillip suggested the rigth thing.
>>>>
>>>>
>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>>
>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>> all messages, I assume ou actually wanted to do this:
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>> Which filters alls messages based on fromhost-ip.
>>>>
>>>> The config format is clumpsy. I am currently talking with some folks at
>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>> on that effort would be welcome.
>>>>
>>>> Rainer
>>>>
>>>>
>>>>
>>>>> It fails to capture logs in the DynFwall template file.
>>>>>
>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>> neither seem to work?
>>>>>
>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>
>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>> at all? Or
>>>>> my config is wrong.
>>>>>
>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>
>>>>> Thanks,
>>>>> Ralph
>>>>>
>>>>> Philip M. Gollucci wrote:
>>>>>
>>>>>
>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> # Firewall logs #
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>
>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>
>>>>>>>
>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>
>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>
>>>>>>>
>>>>> without actions
>>>>>
>>>>>
>>>>>>> will be discarded
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>
>>>>>>>
>>>>> interpret
>>>>>
>>>>>
>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>
>>>>>>>
>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>
>>>>>
>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>> left and right.  Finally I came up with the following with
>>>>>>
>>>>>>
>>>>> works well
>>>>>
>>>>>
>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>
>>>>>>
>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>
>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>> & ~
>>>>>>
>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>> lost our
>>>>> corkscrew and were compelled to live on food and water for
>>>>> several days. -
>>>>> WC Fields
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From david at lang.hm  Mon Jan 18 20:57:41 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 11:57:41 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54A555.9010007@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Ok one more question.
> I have:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  -?DynMail
>
> Which logs all mail to the %HOSTNAME%.mail.log.
>
> My guess would be:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>
> But as Rainer explained these are both filters which won't work.
>
> So how do I use "fromhost-ip" to send only "mail.*" logs from a
> specified host IP to the "DynMail" template?

you need to use the more powerful/complex

if ((condition) and (condition)) action

line format

David Lang

> Thanks,
> Ralph
>
> Ralph Crongeyer wrote:
>> Oh,
>> I tried that but I had it on the same line. So that has to be on a
>> separate line?
>>
>> Thanks again for the explanation that really helps me understand how
>> it's working.
>>
>> Thanks again for all your help with this.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>
>>>
>>>> Hi Rainer,
>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>
>>>> That works, the logs are going to the correct file, however they are
>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>
>>>>
>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>> rsyslog to throw the log away.
>>>
>>> so you would do something like
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> & ~
>>>
>>> which is logicly the same as
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>
>>> David Lang
>>>
>>>
>>>
>>>
>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>
>>>> Thanks all, for your help with this.
>>>> Ralph
>>>>
>>>> Rainer Gerhards wrote:
>>>>
>>>>
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>> Crongeyer
>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>> To: Philip M. Gollucci
>>>>>> Cc: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> Hi Phillip,
>>>>>> Thanks for the response.
>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.*    -?DynFwall
>>>>>>
>>>>>>
>>>>>>
>>>>> Phillip suggested the rigth thing.
>>>>>
>>>>>
>>>>>
>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>>
>>>>>>
>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>> all messages, I assume ou actually wanted to do this:
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>> Which filters alls messages based on fromhost-ip.
>>>>>
>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>> on that effort would be welcome.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>
>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>> neither seem to work?
>>>>>>
>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>
>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>> at all? Or
>>>>>> my config is wrong.
>>>>>>
>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>
>>>>>> Thanks,
>>>>>> Ralph
>>>>>>
>>>>>> Philip M. Gollucci wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> # Firewall logs #
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>
>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> without actions
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> will be discarded
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> interpret
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> works well
>>>>>>
>>>>>>
>>>>>>
>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>
>>>>>>>
>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>
>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>> & ~
>>>>>>>
>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>> lost our
>>>>>> corkscrew and were compelled to live on food and water for
>>>>>> several days. -
>>>>>> WC Fields
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 21:37:03 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 15:37:03 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
Message-ID: <4B54C66F.80506@crongeyer.com>

Thanks David,
Ok so now I'm trying this:

$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then 
?DynMail

After a restart of rsyslog there are no errors in /var/log/syslog 
however no logs are being collected?

Thanks for your help with this David.

Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Ok one more question.
>> I have:
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> mail.*  -?DynMail
>>
>> Which logs all mail to the %HOSTNAME%.mail.log.
>>
>> My guess would be:
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>
>> But as Rainer explained these are both filters which won't work.
>>
>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>> specified host IP to the "DynMail" template?
>>     
>
> you need to use the more powerful/complex
>
> if ((condition) and (condition)) action
>
> line format
>
> David Lang
>
>   
>> Thanks,
>> Ralph
>>
>> Ralph Crongeyer wrote:
>>     
>>> Oh,
>>> I tried that but I had it on the same line. So that has to be on a
>>> separate line?
>>>
>>> Thanks again for the explanation that really helps me understand how
>>> it's working.
>>>
>>> Thanks again for all your help with this.
>>>
>>> Ralph
>>>
>>> david at lang.hm wrote:
>>>
>>>       
>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Hi Rainer,
>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>
>>>>> That works, the logs are going to the correct file, however they are
>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>
>>>>>
>>>>>           
>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>> rsyslog to throw the log away.
>>>>
>>>> so you would do something like
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>> & ~
>>>>
>>>> which is logicly the same as
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>>
>>>>         
>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>
>>>>> Thanks all, for your help with this.
>>>>> Ralph
>>>>>
>>>>> Rainer Gerhards wrote:
>>>>>
>>>>>
>>>>>           
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>> Crongeyer
>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>> To: Philip M. Gollucci
>>>>>>> Cc: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>
>>>>>>> Hi Phillip,
>>>>>>> Thanks for the response.
>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.*    -?DynFwall
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Phillip suggested the rigth thing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>
>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>> on that effort would be welcome.
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>
>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>> neither seem to work?
>>>>>>>
>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>
>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>> at all? Or
>>>>>>> my config is wrong.
>>>>>>>
>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ralph
>>>>>>>
>>>>>>> Philip M. Gollucci wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> # Firewall logs #
>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>
>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> without actions
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> will be discarded
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> interpret
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> works well
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>
>>>>>>>>
>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>
>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>> & ~
>>>>>>>>
>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> --
>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>> lost our
>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>> several days. -
>>>>>>> WC Fields
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 21:41:26 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 12:41:26 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54C66F.80506@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
	<4B54C66F.80506@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Thanks David,
> Ok so now I'm trying this:
>
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then
> ?DynMail

you can't use single quotes, you must use double quotes (apparently the 
config language uses single quotes for something else, I don't know what)

I've tripped over this several times now.

David Lang

> After a restart of rsyslog there are no errors in /var/log/syslog
> however no logs are being collected?
>
> Thanks for your help with this David.
>
> Ralph
>
> david at lang.hm wrote:
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>
>>> Ok one more question.
>>> I have:
>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>> mail.*  -?DynMail
>>>
>>> Which logs all mail to the %HOSTNAME%.mail.log.
>>>
>>> My guess would be:
>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>>
>>> But as Rainer explained these are both filters which won't work.
>>>
>>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>>> specified host IP to the "DynMail" template?
>>>
>>
>> you need to use the more powerful/complex
>>
>> if ((condition) and (condition)) action
>>
>> line format
>>
>> David Lang
>>
>>
>>> Thanks,
>>> Ralph
>>>
>>> Ralph Crongeyer wrote:
>>>
>>>> Oh,
>>>> I tried that but I had it on the same line. So that has to be on a
>>>> separate line?
>>>>
>>>> Thanks again for the explanation that really helps me understand how
>>>> it's working.
>>>>
>>>> Thanks again for all your help with this.
>>>>
>>>> Ralph
>>>>
>>>> david at lang.hm wrote:
>>>>
>>>>
>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi Rainer,
>>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>>
>>>>>> That works, the logs are going to the correct file, however they are
>>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>>
>>>>>>
>>>>>>
>>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>>> rsyslog to throw the log away.
>>>>>
>>>>> so you would do something like
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>> & ~
>>>>>
>>>>> which is logicly the same as
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>>
>>>>>> Thanks all, for your help with this.
>>>>>> Ralph
>>>>>>
>>>>>> Rainer Gerhards wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>>> Crongeyer
>>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>>> To: Philip M. Gollucci
>>>>>>>> Cc: rsyslog-users
>>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>>
>>>>>>>> Hi Phillip,
>>>>>>>> Thanks for the response.
>>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.*    -?DynFwall
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Phillip suggested the rigth thing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>>
>>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>
>>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>>
>>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>>> on that effort would be welcome.
>>>>>>>
>>>>>>> Rainer
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>>
>>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>>> neither seem to work?
>>>>>>>>
>>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>>
>>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>>> at all? Or
>>>>>>>> my config is wrong.
>>>>>>>>
>>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ralph
>>>>>>>>
>>>>>>>> Philip M. Gollucci wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> # Firewall logs #
>>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>>
>>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>>
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> without actions
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> will be discarded
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> interpret
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> works well
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>>
>>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>>> & ~
>>>>>>>>>
>>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>>> lost our
>>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>>> several days. -
>>>>>>>> WC Fields
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 21:52:32 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 15:52:32 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>	<4B54C66F.80506@crongeyer.com>
	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
Message-ID: <4B54CA10.2060103@crongeyer.com>

When I switched to double quotes I get the error in /var/log/syslog and 
no logs are collected?
I switched back to single quots and restart and no error but still no logs?

What else may I be doing wrong?

Thanks,
Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Thanks David,
>> Ok so now I'm trying this:
>>
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then
>> ?DynMail
>>     
>
> you can't use single quotes, you must use double quotes (apparently the 
> config language uses single quotes for something else, I don't know what)
>
> I've tripped over this several times now.
>
> David Lang
>
>   
>> After a restart of rsyslog there are no errors in /var/log/syslog
>> however no logs are being collected?
>>
>> Thanks for your help with this David.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>     
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>
>>>       
>>>> Ok one more question.
>>>> I have:
>>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>>> mail.*  -?DynMail
>>>>
>>>> Which logs all mail to the %HOSTNAME%.mail.log.
>>>>
>>>> My guess would be:
>>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>>>
>>>> But as Rainer explained these are both filters which won't work.
>>>>
>>>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>>>> specified host IP to the "DynMail" template?
>>>>
>>>>         
>>> you need to use the more powerful/complex
>>>
>>> if ((condition) and (condition)) action
>>>
>>> line format
>>>
>>> David Lang
>>>
>>>
>>>       
>>>> Thanks,
>>>> Ralph
>>>>
>>>> Ralph Crongeyer wrote:
>>>>
>>>>         
>>>>> Oh,
>>>>> I tried that but I had it on the same line. So that has to be on a
>>>>> separate line?
>>>>>
>>>>> Thanks again for the explanation that really helps me understand how
>>>>> it's working.
>>>>>
>>>>> Thanks again for all your help with this.
>>>>>
>>>>> Ralph
>>>>>
>>>>> david at lang.hm wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Hi Rainer,
>>>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>>>
>>>>>>> That works, the logs are going to the correct file, however they are
>>>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>>>> rsyslog to throw the log away.
>>>>>>
>>>>>> so you would do something like
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>> & ~
>>>>>>
>>>>>> which is logicly the same as
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>>>
>>>>>>> Thanks all, for your help with this.
>>>>>>> Ralph
>>>>>>>
>>>>>>> Rainer Gerhards wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>>>> Crongeyer
>>>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>>>> To: Philip M. Gollucci
>>>>>>>>> Cc: rsyslog-users
>>>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>>>
>>>>>>>>> Hi Phillip,
>>>>>>>>> Thanks for the response.
>>>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>> *.*    -?DynFwall
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Phillip suggested the rigth thing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>>>
>>>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>
>>>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>>>
>>>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>>>> on that effort would be welcome.
>>>>>>>>
>>>>>>>> Rainer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>>>
>>>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>>>> neither seem to work?
>>>>>>>>>
>>>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>>>
>>>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>>>> at all? Or
>>>>>>>>> my config is wrong.
>>>>>>>>>
>>>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ralph
>>>>>>>>>
>>>>>>>>> Philip M. Gollucci wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> # Firewall logs #
>>>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>>>
>>>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>>>
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> without actions
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> will be discarded
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> interpret
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> works well
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>>>
>>>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>>>> & ~
>>>>>>>>>>
>>>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> --
>>>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>>>> lost our
>>>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>>>> several days. -
>>>>>>>>> WC Fields
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rsyslog mailing list
>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>>> http://www.rsyslog.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>           
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 21:56:30 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 12:56:30 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54CA10.2060103@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
	<4B54C66F.80506@crongeyer.com>
	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
	<4B54CA10.2060103@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> When I switched to double quotes I get the error in /var/log/syslog and
> no logs are collected?

what was the error you got this time?

David Lang



From rgerhards at hq.adiscon.com  Mon Jan 18 21:59:54 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 21:59:54 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>

David,

Single quotes are right in the scripting engine (double quotes are reserved
for future use - they shall provide the capability to extend macros, e.g.
$A="BC" => '$A' is the string "$A", while "$A" is supposed to be the string
"BC").

I don't have an idea what may be wrong, but running rsyslog in debug mode
will most probably pinpoint it.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 18, 2010 9:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> 
> > When I switched to double quotes I get the error in 
> /var/log/syslog and
> > no logs are collected?
> 
> what was the error you got this time?
> 
> David Lang
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Mon Jan 18 22:02:04 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 13:02:04 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Rainer Gerhards wrote:

> David,
>
> Single quotes are right in the scripting engine (double quotes are reserved
> for future use - they shall provide the capability to extend macros, e.g.
> $A="BC" => '$A' is the string "$A", while "$A" is supposed to be the string
> "BC").

that is the normal behavior of single vs double quotes, but in such 
situations it's normal for 'ABC' and "ABC" to be equivalent, it's only 
when you have variables involved that there would be a difference.

David Lang

> I don't have an idea what may be wrong, but running rsyslog in debug mode
> will most probably pinpoint it.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 18, 2010 9:57 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>> When I switched to double quotes I get the error in
>> /var/log/syslog and
>>> no logs are collected?
>>
>> what was the error you got this time?
>>
>> David Lang
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From ralph at crongeyer.com  Mon Jan 18 22:02:27 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:02:27 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>	<4B54C66F.80506@crongeyer.com>	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>	<4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
Message-ID: <4B54CC63.3010103@crongeyer.com>

With double quots I get this in /var/log/syslog:

Jan 18 16:00:22 log rsyslogd: [origin software="rsyslogd" 
swVersion="4.4.2" x-pid="15703" x-info="http://www.rsyslog.com"] (re)start
Jan 18 16:00:22 log rsyslogd: the last error occured in 
/etc/rsyslog.d/remote-logs.conf, line 6
Jan 18 16:00:22 log rsyslogd: warning: selector line without actions 
will be discarded
Jan 18 16:00:22 log rsyslogd: the last error occured in 
/etc/rsyslog.conf, line 48
Jan 18 16:00:22 log rsyslogd-2124: CONFIG ERROR: could not interpret 
master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> When I switched to double quotes I get the error in /var/log/syslog and
>> no logs are collected?
>>     
>
> what was the error you got this time?
>
> David Lang
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Mon Jan 18 22:03:50 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 22:03:50 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 18, 2010 10:02 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> 
> > David,
> >
> > Single quotes are right in the scripting engine (double 
> quotes are reserved
> > for future use - they shall provide the capability to 
> extend macros, e.g.
> > $A="BC" => '$A' is the string "$A", while "$A" is supposed 
> to be the string
> > "BC").
> 
> that is the normal behavior of single vs double quotes, but in such 
> situations it's normal for 'ABC' and "ABC" to be equivalent, 
> it's only 
> when you have variables involved that there would be a difference.

Jup, that's right - but double quotes are not yet implemented ;)

Rainer
> 
> David Lang
> 
> > I don't have an idea what may be wrong, but running rsyslog 
> in debug mode
> > will most probably pinpoint it.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com
> >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
> david at lang.hm
> >> Sent: Monday, January 18, 2010 9:57 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] fromhost-ip
> >>
> >> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >>
> >>> When I switched to double quotes I get the error in
> >> /var/log/syslog and
> >>> no logs are collected?
> >>
> >> what was the error you got this time?
> >>
> >> David Lang
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ralph at crongeyer.com  Mon Jan 18 22:27:49 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:27:49 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
Message-ID: <4B54D255.30505@crongeyer.com>

Here's the debug output when configured with single quotes.
I'm sending this off the list to Rainer.
David, let me know if you want this also.

Thanks guys,
Ralph

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com 
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 18, 2010 10:02 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>
>>     
>>> David,
>>>
>>> Single quotes are right in the scripting engine (double 
>>>       
>> quotes are reserved
>>     
>>> for future use - they shall provide the capability to 
>>>       
>> extend macros, e.g.
>>     
>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed 
>>>       
>> to be the string
>>     
>>> "BC").
>>>       
>> that is the normal behavior of single vs double quotes, but in such 
>> situations it's normal for 'ABC' and "ABC" to be equivalent, 
>> it's only 
>> when you have variables involved that there would be a difference.
>>     
>
> Jup, that's right - but double quotes are not yet implemented ;)
>
> Rainer
>   
>> David Lang
>>
>>     
>>> I don't have an idea what may be wrong, but running rsyslog 
>>>       
>> in debug mode
>>     
>>> will most probably pinpoint it.
>>>
>>> Rainer
>>>
>>>       
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
>>>>         
>> david at lang.hm
>>     
>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>
>>>>         
>>>>> When I switched to double quotes I get the error in
>>>>>           
>>>> /var/log/syslog and
>>>>         
>>>>> no logs are collected?
>>>>>           
>>>> what was the error you got this time?
>>>>
>>>> David Lang
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 22:47:53 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:47:53 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54D255.30505@crongeyer.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
	<4B54D255.30505@crongeyer.com>
Message-ID: <4B54D709.4050408@crongeyer.com>

This ma be of help:

0928.085091536:imrelp.c: Message has legacy syslog format.
0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
0928.085443830:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
0928.085812887:imrelp.c: tcpSend returns 17
0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
0928.086029125:imrelp.c: relp engine is dispatching frame with command 
'syslog'
0928.086053430:imrelp.c: in 'syslog' command handler
0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg 
2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost 
connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
0928.086124392:imrelp.c: Message has legacy syslog format.
0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
0928.086514402:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
0928.087044659:imrelp.c: tcpSend returns 17
0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
0928.087110313:imrelp.c: relp engine is dispatching frame with command 
'syslog'
0928.087131545:imrelp.c: in 'syslog' command handler
0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg 
2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect 
from 81-64-60-151.rev.numericable.fr[81.64.60.151]
0928.087200552:imrelp.c: Message has legacy syslog format.
0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
0928.087609280:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
0928.088020802:imrelp.c: tcpSend returns 17
0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
0928.088099586:imrelp.c: ***<librelp> calling select, active file 
descriptors (max 23): 6 7 23
0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout, 
worker terminating...
0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08, 
terminated, num workers now 0
0988.088339377:main queue:Reg/w0: destructor for debug call stack 
0x9bd1260 called


Ralph Crongeyer wrote:
> Here's the debug output when configured with single quotes.
> I'm sending this off the list to Rainer.
> David, let me know if you want this also.
>
> Thanks guys,
> Ralph
>
> Rainer Gerhards wrote:
>   
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com 
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Monday, January 18, 2010 10:02 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] fromhost-ip
>>>
>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>
>>>     
>>>       
>>>> David,
>>>>
>>>> Single quotes are right in the scripting engine (double 
>>>>       
>>>>         
>>> quotes are reserved
>>>     
>>>       
>>>> for future use - they shall provide the capability to 
>>>>       
>>>>         
>>> extend macros, e.g.
>>>     
>>>       
>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed 
>>>>       
>>>>         
>>> to be the string
>>>     
>>>       
>>>> "BC").
>>>>       
>>>>         
>>> that is the normal behavior of single vs double quotes, but in such 
>>> situations it's normal for 'ABC' and "ABC" to be equivalent, 
>>> it's only 
>>> when you have variables involved that there would be a difference.
>>>     
>>>       
>> Jup, that's right - but double quotes are not yet implemented ;)
>>
>> Rainer
>>   
>>     
>>> David Lang
>>>
>>>     
>>>       
>>>> I don't have an idea what may be wrong, but running rsyslog 
>>>>       
>>>>         
>>> in debug mode
>>>     
>>>       
>>>> will most probably pinpoint it.
>>>>
>>>> Rainer
>>>>
>>>>       
>>>>         
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
>>>>>         
>>>>>           
>>> david at lang.hm
>>>     
>>>       
>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>
>>>>>         
>>>>>           
>>>>>> When I switched to double quotes I get the error in
>>>>>>           
>>>>>>             
>>>>> /var/log/syslog and
>>>>>         
>>>>>           
>>>>>> no logs are collected?
>>>>>>           
>>>>>>             
>>>>> what was the error you got this time?
>>>>>
>>>>> David Lang
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>       
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>     
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 22:52:32 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 13:52:32 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54D709.4050408@crongeyer.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
	<4B54D255.30505@crongeyer.com> <4B54D709.4050408@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>

Ok, this says that fromhost-ip is not being set in your case.

I think I ran into a similar problem before, are you starting with -x to 
disable name lookups?

try changing from fromhost-ip to fromhost

David Lang

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> This ma be of help:
>
> 0928.085091536:imrelp.c: Message has legacy syslog format.
> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
> 0928.085443830:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.085812887:imrelp.c: tcpSend returns 17
> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> 0928.086029125:imrelp.c: relp engine is dispatching frame with command
> 'syslog'
> 0928.086053430:imrelp.c: in 'syslog' command handler
> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
> connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> 0928.086124392:imrelp.c: Message has legacy syslog format.
> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
> 0928.086514402:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.087044659:imrelp.c: tcpSend returns 17
> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> 0928.087110313:imrelp.c: relp engine is dispatching frame with command
> 'syslog'
> 0928.087131545:imrelp.c: in 'syslog' command handler
> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect
> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> 0928.087200552:imrelp.c: Message has legacy syslog format.
> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
> 0928.087609280:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.088020802:imrelp.c: tcpSend returns 17
> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
> descriptors (max 23): 6 7 23
> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout,
> worker terminating...
> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08,
> terminated, num workers now 0
> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> 0x9bd1260 called
>
>
> Ralph Crongeyer wrote:
>> Here's the debug output when configured with single quotes.
>> I'm sending this off the list to Rainer.
>> David, let me know if you want this also.
>>
>> Thanks guys,
>> Ralph
>>
>> Rainer Gerhards wrote:
>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>> Sent: Monday, January 18, 2010 10:02 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>>
>>>>
>>>>
>>>>> David,
>>>>>
>>>>> Single quotes are right in the scripting engine (double
>>>>>
>>>>>
>>>> quotes are reserved
>>>>
>>>>
>>>>> for future use - they shall provide the capability to
>>>>>
>>>>>
>>>> extend macros, e.g.
>>>>
>>>>
>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>>>>>
>>>>>
>>>> to be the string
>>>>
>>>>
>>>>> "BC").
>>>>>
>>>>>
>>>> that is the normal behavior of single vs double quotes, but in such
>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>>>> it's only
>>>> when you have variables involved that there would be a difference.
>>>>
>>>>
>>> Jup, that's right - but double quotes are not yet implemented ;)
>>>
>>> Rainer
>>>
>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>>> I don't have an idea what may be wrong, but running rsyslog
>>>>>
>>>>>
>>>> in debug mode
>>>>
>>>>
>>>>> will most probably pinpoint it.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>>>>>>
>>>>>>
>>>> david at lang.hm
>>>>
>>>>
>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> When I switched to double quotes I get the error in
>>>>>>>
>>>>>>>
>>>>>> /var/log/syslog and
>>>>>>
>>>>>>
>>>>>>> no logs are collected?
>>>>>>>
>>>>>>>
>>>>>> what was the error you got this time?
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 23:12:24 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 17:12:24 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>	<4B54D255.30505@crongeyer.com>
	<4B54D709.4050408@crongeyer.com>
	<alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
Message-ID: <4B54DCC8.3020504@crongeyer.com>

No, I'm starting with -c4.

I'll give it a try but ultimately  I need to filter in IP.

I'll try it when I get back from dinner......

Thanks again for your help with this guys.

david at lang.hm wrote:
> Ok, this says that fromhost-ip is not being set in your case.
>
> I think I ran into a similar problem before, are you starting with -x to 
> disable name lookups?
>
> try changing from fromhost-ip to fromhost
>
> David Lang
>
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> This ma be of help:
>>
>> 0928.085091536:imrelp.c: Message has legacy syslog format.
>> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.085443830:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.085812887:imrelp.c: tcpSend returns 17
>> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
>> 0928.086029125:imrelp.c: relp engine is dispatching frame with command
>> 'syslog'
>> 0928.086053430:imrelp.c: in 'syslog' command handler
>> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
>> connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> 0928.086124392:imrelp.c: Message has legacy syslog format.
>> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.086514402:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.087044659:imrelp.c: tcpSend returns 17
>> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
>> 0928.087110313:imrelp.c: relp engine is dispatching frame with command
>> 'syslog'
>> 0928.087131545:imrelp.c: in 'syslog' command handler
>> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect
>> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> 0928.087200552:imrelp.c: Message has legacy syslog format.
>> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.087609280:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.088020802:imrelp.c: tcpSend returns 17
>> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
>> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
>> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
>> descriptors (max 23): 6 7 23
>> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout,
>> worker terminating...
>> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
>> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
>> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08,
>> terminated, num workers now 0
>> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
>> 0x9bd1260 called
>>
>>
>> Ralph Crongeyer wrote:
>>     
>>> Here's the debug output when configured with single quotes.
>>> I'm sending this off the list to Rainer.
>>> David, let me know if you want this also.
>>>
>>> Thanks guys,
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>
>>>       
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>>> Sent: Monday, January 18, 2010 10:02 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> David,
>>>>>>
>>>>>> Single quotes are right in the scripting engine (double
>>>>>>
>>>>>>
>>>>>>             
>>>>> quotes are reserved
>>>>>
>>>>>
>>>>>           
>>>>>> for future use - they shall provide the capability to
>>>>>>
>>>>>>
>>>>>>             
>>>>> extend macros, e.g.
>>>>>
>>>>>
>>>>>           
>>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>>>>>>
>>>>>>
>>>>>>             
>>>>> to be the string
>>>>>
>>>>>
>>>>>           
>>>>>> "BC").
>>>>>>
>>>>>>
>>>>>>             
>>>>> that is the normal behavior of single vs double quotes, but in such
>>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>>>>> it's only
>>>>> when you have variables involved that there would be a difference.
>>>>>
>>>>>
>>>>>           
>>>> Jup, that's right - but double quotes are not yet implemented ;)
>>>>
>>>> Rainer
>>>>
>>>>
>>>>         
>>>>> David Lang
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I don't have an idea what may be wrong, but running rsyslog
>>>>>>
>>>>>>
>>>>>>             
>>>>> in debug mode
>>>>>
>>>>>
>>>>>           
>>>>>> will most probably pinpoint it.
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>> david at lang.hm
>>>>>
>>>>>
>>>>>           
>>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>>>> To: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>
>>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> When I switched to double quotes I get the error in
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> /var/log/syslog and
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> no logs are collected?
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> what was the error you got this time?
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Tue Jan 19 10:44:04 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 10:44:04 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>	<4B54D255.30505@crongeyer.com><4B54D709.4050408@crongeyer.com><alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
	<4B54DCC8.3020504@crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F9@GRFEXC.intern.adiscon.com>

RELP did not provide fromhost-ip until recently. You need to use the most
recent development version of the git master branch (to be released soon)
TOGETHER with the most recent version of librelp to get that information.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> Sent: Monday, January 18, 2010 11:12 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> No, I'm starting with -c4.
> 
> I'll give it a try but ultimately  I need to filter in IP.
> 
> I'll try it when I get back from dinner......
> 
> Thanks again for your help with this guys.
> 
> david at lang.hm wrote:
> > Ok, this says that fromhost-ip is not being set in your case.
> >
> > I think I ran into a similar problem before, are you starting with -x
> to
> > disable name lookups?
> >
> > try changing from fromhost-ip to fromhost
> >
> > David Lang
> >
> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >
> >
> >> This ma be of help:
> >>
> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.085812887:imrelp.c: tcpSend returns 17
> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
> command
> >> 'syslog'
> >> 0928.086053430:imrelp.c: in 'syslog' command handler
> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
> >> connection after RCPT from 81-64-60-
> 151.rev.numericable.fr[81.64.60.151]
> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.087044659:imrelp.c: tcpSend returns 17
> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
> command
> >> 'syslog'
> >> 0928.087131545:imrelp.c: in 'syslog' command handler
> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
> disconnect
> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.088020802:imrelp.c: tcpSend returns 17
> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> >> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
> >> descriptors (max 23): 6 7 23
> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
> timeout,
> >> worker terminating...
> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
> command 1
> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
> terminating
> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
> 9bb5a08,
> >> terminated, num workers now 0
> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> >> 0x9bd1260 called
> >>
> >>
> >> Ralph Crongeyer wrote:
> >>
> >>> Here's the debug output when configured with single quotes.
> >>> I'm sending this off the list to Rainer.
> >>> David, let me know if you want this also.
> >>>
> >>> Thanks guys,
> >>> Ralph
> >>>
> >>> Rainer Gerhards wrote:
> >>>
> >>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces at lists.adiscon.com
> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> david at lang.hm
> >>>>> Sent: Monday, January 18, 2010 10:02 PM
> >>>>> To: rsyslog-users
> >>>>> Subject: Re: [rsyslog] fromhost-ip
> >>>>>
> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> David,
> >>>>>>
> >>>>>> Single quotes are right in the scripting engine (double
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> quotes are reserved
> >>>>>
> >>>>>
> >>>>>
> >>>>>> for future use - they shall provide the capability to
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> extend macros, e.g.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> to be the string
> >>>>>
> >>>>>
> >>>>>
> >>>>>> "BC").
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> that is the normal behavior of single vs double quotes, but in
> such
> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
> >>>>> it's only
> >>>>> when you have variables involved that there would be a
> difference.
> >>>>>
> >>>>>
> >>>>>
> >>>> Jup, that's right - but double quotes are not yet implemented ;)
> >>>>
> >>>> Rainer
> >>>>
> >>>>
> >>>>
> >>>>> David Lang
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> I don't have an idea what may be wrong, but running rsyslog
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> in debug mode
> >>>>>
> >>>>>
> >>>>>
> >>>>>> will most probably pinpoint it.
> >>>>>>
> >>>>>> Rainer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>> david at lang.hm
> >>>>>
> >>>>>
> >>>>>
> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
> >>>>>>> To: rsyslog-users
> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
> >>>>>>>
> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> When I switched to double quotes I get the error in
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> /var/log/syslog and
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> no logs are collected?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> what was the error you got this time?
> >>>>>>>
> >>>>>>> David Lang
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> rsyslog mailing list
> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>>> http://www.rsyslog.com
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> rsyslog mailing list
> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>> http://www.rsyslog.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> 
> --
> Reminds me of my expedition into the wilds of Afghanistan. We lost our
> corkscrew and were compelled to live on food and water for several
> days. -
> WC Fields
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 14:53:44 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 14:53:44 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>

Michael,

I tried to reproduce, but I can not get to this error. Could you provide me a
debug log of the failed startup?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, January 17, 2010 12:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> Sent: Friday, January 15, 2010 11:57 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> > Michael,
> >> >
> >> > Fix now in git, links at the bug tracker:
> >> >
> >> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >> >
> >> > Please let me know if it works for you (the patch is a bit
> trickier
> >> than it
> >> > looks, so confirmations would be good).
> >>
> >> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> >> But now I'm getting a crash when rsyslog encounters the xconsole
> pipe
> >> config.
> >
> > I am a bit puzzled, but will try to reproduce that on my Debian box.
> I assume
> > stock Debian config?
> 
> Yes. As said, I just downloaded the 5.3.6 tarball applied the
> 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> got the crash. I use the default rsyslog.conf from the official debian
> package.
> I attached a backtrace. Hope that helps
> 
> (gdb) run -c4 -d
> Starting program: /usr/sbin/rsyslogd -c4 -d
> [Thread debugging using libthread_db enabled]
> [New Thread 0xb7df2b70 (LWP 5162)]
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xb7df2b70 (LWP 5162)]
> qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
> pUsr=0x80b72c0) at queue.c:2256
> 2256            if(pThis->qType != QUEUETYPE_DIRECT) {
> (gdb) bt full
> #0  qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
> pUsr=0x80b72c0) at queue.c:2256
>         iRet = <value optimized out>
>         iCancelStateSave = <value optimized out>
> #1  0x0807cd5b in actionWriteToAction (pAction=0x80ac8d8) at
> ../action.c:1169
>         pMsgSave = 0x0
>         iRet = <value optimized out>
> #2  0x0807d4fe in doActionCallAction (pAction=0x80ac8d8,
> pMsg=0x80b72c0) at ../action.c:1244
> No locals.
> #3  actionCallAction (pAction=0x80ac8d8, pMsg=0x80b72c0) at
> ../action.c:1274
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113864, 0, -1210114056, 1464119912, -422385385},
>               __mask_was_saved = 0}}, __pad = {0xb7df2240, 0x0,
> 0xb7df20d0, 0xb7feb27f}}
>         __cancel_arg = 0x80ad0a0
>         not_first_call = <value optimized out>
>         iRet = -1210113864
> #4  0x080794d7 in processMsgDoActions (pData=0xfffff815,
> pParam=0xb7df20b8) at rule.c:113
>         iRet = <value optimized out>
>         iRetMod = <value optimized out>
> #5  0x080627ba in llExecFunc (pThis=0x80ac9a0, pFunc=0x8079480
> <processMsgDoActions>, pParam=0xb7df20b8) at linkedlist.c:391
>         iRet = <value optimized out>
>         iRetLL = <value optimized out>
>         pData = 0x80ac8d8
>         llCookie = 0x80ac508
>         llCookiePrev = 0x0
> #6  0x08079007 in processMsg (pThis=0x80ac968, pMsg=0x80b72c0) at
> rule.c:299
>         bProcessMsg = 1
>         DoActData = {bPrevWasSuspended = 0, pMsg = 0x80b72c0}
>         iRet = RS_RET_OK
> #7  0x080781ba in processMsgDoRules (pData=0x80ac968,
> pParam=0x80b72c0) at ruleset.c:145
>         iRet = <value optimized out>
> #8  0x080627ba in llExecFunc (pThis=0x809eb68, pFunc=0x8078190
> <processMsgDoRules>, pParam=0x80b72c0) at linkedlist.c:391
>         iRet = <value optimized out>
>         iRetLL = <value optimized out>
>         pData = 0x80ac968
>         llCookie = 0x80ac478
>         llCookiePrev = 0x80ac4f8
> #9  0x0807876d in processMsg (pMsg=0x80b72c0) at ruleset.c:164
>         pThis = <value optimized out>
>         iRet = <value optimized out>
> #10 0x080506ab in msgConsumer (notNeeded=0x0, pBatch=0x809ecc8,
> pbShutdownImmediate=0x80ad348) at syslogd.c:614
>         i = 0
>         pMsg = 0x80b72c0
>         localRet = RS_RET_IO_ERROR
> #11 0x08077ca4 in ConsumerReg (pThis=0x80ad338, pWti=0x809ecb0) at
> queue.c:1638
>         iCancelStateSave = 1
>         iRet = <value optimized out>
> #12 0x08070526 in wtiWorker (pThis=0x809ecb0) at wti.c:286
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113200, 0, -1210113384, 1463800424, -411898089},
>               __mask_was_saved = 0}}, __pad = {0xb7df2350, 0x0, 0x0,
> 0x809002c}}
>         not_first_call = <value optimized out>
>         pWtp = 0x809ebd8
>         bInactivityTOOccured = <value optimized out>
>         localRet = <value optimized out>
>         terminateRet = RS_RET_OK
>         iCancelStateSave = <value optimized out>
> #13 0x08070074 in wtpWorker (arg=0x809ecb0) at wtp.c:356
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113152, 0, -1210113096, 1463726696, -411345641},
>               __mask_was_saved = 0}}, __pad = {0xb7df2460, 0x0,
> 0xb7feff7b, 0xb7fe0cb0}}
>         not_first_call = <value optimized out>
>         pszDbgHdr = <value optimized out>
>         thrdName = "rs:main Q:Reg", '\000' <repeats 18 times>
> ---Type <return> to continue, or q <return> to quit---
>         pThis = 0x809ebd8
>         sigSet = {__val = {2147483647, 4294967294, 4294967295 <repeats
> 30 times>}}
> #14 0xb7f9c585 in start_thread () from /lib/i686/cmov/libpthread.so.0
> No symbol table info available.
> #15 0xb7f1026e in clone () from /lib/i686/cmov/libc.so.6
> No symbol table info available.
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 14:55:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 14:55:59 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<4B52FAD7.305@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FD@GRFEXC.intern.adiscon.com>

Philip,

I wil try to set up this as well. In the mean time, could you tell me if it
happens with the plain 5.3.6 or with the newer git tree (with the patch).
Without the patch, I can already see why it can happen, with it, I do not yet
have a clear understanding of the issue.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Sunday, January 17, 2010 12:56 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On 1/17/2010 6:51 AM, Michael Biebl wrote:
> > As an additonal hint: If I start xconsole (a process reading from
> > /dev/xconsole) before I start rsyslogd, then the crash does not
> occur.
> Possibly related, on FreeBSD in a jail if rsyslog ever tries to write
> to
> /dev/console, it loops in a extremely tightly loop consuming 100% of
> the
> core its on.  [see -dn output, possibly with ktrace/kdump]
> 
> Eventually it will consume all the memory on the box and it will go
> boom.
> 
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Sr. System Admin,                 Ridecharge Inc.
> Consultant,                       P6M7G8 Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Tue Jan 19 14:58:50 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 19 Jan 2010 14:58:50 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001190558k138e1868y2ec8763e8a574a00@mail.gmail.com>

2010/1/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> I tried to reproduce, but I can not get to this error. Could you provide me a
> debug log of the failed startup?

There is no debug output of rsyslog before it crashes. All I can get
is the gdb output I already attached

The missing debug output when using -d is another bug I already
mentioned in this thread.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Tue Jan 19 15:20:03 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:20:03 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
	<4B509EC0.3050504@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>

Jack,

I have written this small patch for v4:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c89ae6b
41d1c5838b9a42b0

I assume it will apply without problems in v5 as well, but I have not yet
tried as I am doing some more work on v4 first (hoping to be able to save a
merge or two, which clutter up the git history...).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 5:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> Rainer Gerhards wrote:
> > -----Original Message-----
> >> If that is modified as follows: *.*
> >> /var/log/syslog   # Comment goes here
> >>
> >
> > It's even worse: data is written to the file "/var/log/syslog   #
> > Comment goes here"!
> 
> Aaaaahhh - that finally explains some odd files lurking in /var/log!
> 
> That also explains why there's no error message on stdout/stdderr -
> there's no error, as far as rsyslog is concerned. [/me feels stupid]
> 
> I would really like to be able to put comments on the ends of config
> lines, as I can with many other packages. But now I know what's going
> on
> here. Thanks!
> 
> --
> Jack.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 15:50:42 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:50:42 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
	<aa76cdac1001190558k138e1868y2ec8763e8a574a00@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FF@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Tuesday, January 19, 2010 2:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > I tried to reproduce, but I can not get to this error. Could you
> provide me a
> > debug log of the failed startup?
> 
> There is no debug output of rsyslog before it crashes. All I can get
> is the gdb output I already attached
> 
> The missing debug output when using -d is another bug I already
> mentioned in this thread.

slipped my mind, I should have opened a bug tracker. As I thought, it was a
regression from "debug on demand" mode. Patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bd03b86c6322c82fc9f66712
2f4365e339f28ccc

Rainer

> 
> Cheers,
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mrdemeanour at jackpot.uk.net  Tue Jan 19 15:50:17 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Tue, 19 Jan 2010 14:50:17 +0000
Subject: [rsyslog] comments in file actions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>	<4B509EC0.3050504@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
Message-ID: <4B55C6A9.5010008@jackpot.uk.net>

Rainer Gerhards wrote:
> Jack,
> 
> I have written this small patch for v4:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c89ae6b
>  41d1c5838b9a42b0
> 
> I assume it will apply without problems in v5 as well, but I have not
> yet tried as I am doing some more work on v4 first (hoping to be able
> to save a merge or two, which clutter up the git history...).

OK - have to go out now, but I will try this tomorrow and report back.

-- 
Jack.



From rgerhards at hq.adiscon.com  Tue Jan 19 15:51:33 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:51:33 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>	<4B509EC0.3050504@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
	<4B55C6A9.5010008@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103700@GRFEXC.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Tuesday, January 19, 2010 3:50 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> Rainer Gerhards wrote:
> > Jack,
> >
> > I have written this small patch for v4:
> >
> >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c
> 89ae6b
> >  41d1c5838b9a42b0
> >
> > I assume it will apply without problems in v5 as well, but I have not
> > yet tried as I am doing some more work on v4 first (hoping to be able
> > to save a merge or two, which clutter up the git history...).
> 
> OK - have to go out now, but I will try this tomorrow and report back.

excellent - thanks!

Rainer

> 
> --
> Jack.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ralph at crongeyer.com  Tue Jan 19 16:22:14 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Tue, 19 Jan 2010 10:22:14 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F9@GRFEXC.intern.adiscon.com>
Message-ID: <5fa6e0144d8003c7c72edff17f9f1675@webmail.crongeyer.com>

 Ok.
I'll try it with TCP (@@).
This weekend I'll build a deb of the latest rsyslog and relp and check it
out.

Would I ned the latest on both the rsyslog server and the client or just the
server?

Thanks,
Ralph
 
----------------original message-----------------
From: "Rainer Gerhards" rgerhards at hq.adiscon.com
To: "rsyslog-users" rsyslog at lists.adiscon.com
Date: Tue, 19 Jan 2010 10:44:04 +0100
-------------------------------------------------
 
 
> RELP did not provide fromhost-ip until recently. You need to use the most
> recent development version of the git master branch (to be released soon)
> TOGETHER with the most recent version of librelp to get that information.
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
>> Sent: Monday, January 18, 2010 11:12 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>> 
>> No, I'm starting with -c4.
>> 
>> I'll give it a try but ultimately I need to filter in IP.
>> 
>> I'll try it when I get back from dinner......
>> 
>> Thanks again for your help with this guys.
>> 
>> david at lang.hm wrote:
>> > Ok, this says that fromhost-ip is not being set in your case.
>> >
>> > I think I ran into a similar problem before, are you starting with -x
>> to
>> > disable name lookups?
>> >
>> > try changing from fromhost-ip to fromhost
>> >
>> > David Lang
>> >
>> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>> >
>> >
>> >> This ma be of help:
>> >>
>> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
>> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.085812887:imrelp.c: tcpSend returns 17
>> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
>> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
>> command
>> >> 'syslog'
>> >> 0928.086053430:imrelp.c: in 'syslog' command handler
>> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: 
>> lost
>> >> connection after RCPT from 81-64-60-
>> 151.rev.numericable.fr[81.64.60.151]
>> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
>> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.087044659:imrelp.c: tcpSend returns 17
>> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
>> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
>> command
>> >> 'syslog'
>> >> 0928.087131545:imrelp.c: in 'syslog' command handler
>> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
>> disconnect
>> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
>> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.088020802:imrelp.c: tcpSend returns 17
>> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
>> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
>> >> 0928.088099586:imrelp.c: ***
 calling select, active file
>> >> descriptors (max 23): 6 7 23
>> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
>> timeout,
>> >> worker terminating...
>> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
>> command 1
>> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
>> terminating
>> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
>> 9bb5a08,
>> >> terminated, num workers now 0
>> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
>> >> 0x9bd1260 called
>> >>
>> >>
>> >> Ralph Crongeyer wrote:
>> >>
>> >>> Here's the debug output when configured with single quotes.
>> >>> I'm sending this off the list to Rainer.
>> >>> David, let me know if you want this also.
>> >>>
>> >>> Thanks guys,
>> >>> Ralph
>> >>>
>> >>> Rainer Gerhards wrote:
>> >>>
>> >>>
>> >>>>> -----Original Message-----
>> >>>>> From: rsyslog-bounces at lists.adiscon.com
>> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>> david at lang.hm
>> >>>>> Sent: Monday, January 18, 2010 10:02 PM
>> >>>>> To: rsyslog-users
>> >>>>> Subject: Re: [rsyslog] fromhost-ip
>> >>>>>
>> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> David,
>> >>>>>>
>> >>>>>> Single quotes are right in the scripting engine (double
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> quotes are reserved
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> for future use - they shall provide the capability to
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> extend macros, e.g.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> to be the string
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> "BC").
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> that is the normal behavior of single vs double quotes, but in
>> such
>> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>> >>>>> it's only
>> >>>>> when you have variables involved that there would be a
>> difference.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>> Jup, that's right - but double quotes are not yet implemented ;)
>> >>>>
>> >>>> Rainer
>> >>>>
>> >>>>
>> >>>>
>> >>>>> David Lang
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> I don't have an idea what may be wrong, but running rsyslog
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> in debug mode
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> will most probably pinpoint it.
>> >>>>>>
>> >>>>>> Rainer
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> -----Original Message-----
>> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
>> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>> david at lang.hm
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>> >>>>>>> To: rsyslog-users
>> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
>> >>>>>>>
>> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> When I switched to double quotes I get the error in
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>> /var/log/syslog and
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> no logs are collected?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>> what was the error you got this time?
>> >>>>>>>
>> >>>>>>> David Lang
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> rsyslog mailing list
>> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>>>> http://www.rsyslog.com
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> rsyslog mailing list
>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>>> http://www.rsyslog.com
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> _______________________________________________
>> >>>>> rsyslog mailing list
>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>> http://www.rsyslog.com
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>> _______________________________________________
>> >>>> rsyslog mailing list
>> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>> http://www.rsyslog.com
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >>
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> 
>> 
>> --
>> Reminds me of my expedition into the wilds of Afghanistan. We lost our
>> corkscrew and were compelled to live on food and water for several
>> days. -
>> WC Fields
>> 
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 




From rgerhards at hq.adiscon.com  Tue Jan 19 16:28:01 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 16:28:01 +0100
Subject: [rsyslog] fromhost-ip
References: <5fa6e0144d8003c7c72edff17f9f1675@webmail.crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103702@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> Sent: Tuesday, January 19, 2010 4:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
>  Ok.
> I'll try it with TCP (@@).
> This weekend I'll build a deb of the latest rsyslog and relp and check
> it
> out.
> 
> Would I ned the latest on both the rsyslog server and the client or
> just the
> server?

The server should be sufficient. The issue is that librelp < 1.0.0 has the
information, but does not pass it down to the call (imrelp in rsyslog case).
So imrelp decided to use "[unset]" instead of anything else (librelp actually
passes down the hostname twice). In librelp >= 1.0.0 this is corrected, it
now provides the ip address. However, you also need the new imrelp, as it now
needs to use that property. All of this, however, is done on the server, so
no dependency on the client should exist.

I have done these changes in early december 2009 as a side-activity for
something else relp related. My memory has a bit vanished since them, but I
think I conveyed the right information (but you now know I may be wrong in
case something works other than expected - in that case, ask here first
before getting nuts ;)).

Rainer

> 
> Thanks,
> Ralph
> 
> ----------------original message-----------------
> From: "Rainer Gerhards" rgerhards at hq.adiscon.com
> To: "rsyslog-users" rsyslog at lists.adiscon.com
> Date: Tue, 19 Jan 2010 10:44:04 +0100
> -------------------------------------------------
> 
> 
> > RELP did not provide fromhost-ip until recently. You need to use the
> most
> > recent development version of the git master branch (to be released
> soon)
> > TOGETHER with the most recent version of librelp to get that
> information.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> >> Sent: Monday, January 18, 2010 11:12 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] fromhost-ip
> >>
> >> No, I'm starting with -c4.
> >>
> >> I'll give it a try but ultimately I need to filter in IP.
> >>
> >> I'll try it when I get back from dinner......
> >>
> >> Thanks again for your help with this guys.
> >>
> >> david at lang.hm wrote:
> >> > Ok, this says that fromhost-ip is not being set in your case.
> >> >
> >> > I think I ran into a similar problem before, are you starting with
> -x
> >> to
> >> > disable name lookups?
> >> >
> >> > try changing from fromhost-ip to fromhost
> >> >
> >> > David Lang
> >> >
> >> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >> >
> >> >
> >> >> This ma be of help:
> >> >>
> >> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
> >> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.085416731:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.085812887:imrelp.c: tcpSend returns 17
> >> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> >> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
> >> command
> >> >> 'syslog'
> >> >> 0928.086053430:imrelp.c: in 'syslog' command handler
> >> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5',
> msg
> >> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]:
> >> lost
> >> >> connection after RCPT from 81-64-60-
> >> 151.rev.numericable.fr[81.64.60.151]
> >> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
> >> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.086486185:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.087044659:imrelp.c: tcpSend returns 17
> >> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> >> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
> >> command
> >> >> 'syslog'
> >> >> 0928.087131545:imrelp.c: in 'syslog' command handler
> >> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5',
> msg
> >> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
> >> disconnect
> >> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> >> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
> >> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.087581622:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.088020802:imrelp.c: tcpSend returns 17
> >> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> >> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> >> >> 0928.088099586:imrelp.c: ***
>  calling select, active file
> >> >> descriptors (max 23): 6 7 23
> >> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
> >> timeout,
> >> >> worker terminating...
> >> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
> >> command 1
> >> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
> >> terminating
> >> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
> >> 9bb5a08,
> >> >> terminated, num workers now 0
> >> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> >> >> 0x9bd1260 called
> >> >>
> >> >>
> >> >> Ralph Crongeyer wrote:
> >> >>
> >> >>> Here's the debug output when configured with single quotes.
> >> >>> I'm sending this off the list to Rainer.
> >> >>> David, let me know if you want this also.
> >> >>>
> >> >>> Thanks guys,
> >> >>> Ralph
> >> >>>
> >> >>> Rainer Gerhards wrote:
> >> >>>
> >> >>>
> >> >>>>> -----Original Message-----
> >> >>>>> From: rsyslog-bounces at lists.adiscon.com
> >> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >> david at lang.hm
> >> >>>>> Sent: Monday, January 18, 2010 10:02 PM
> >> >>>>> To: rsyslog-users
> >> >>>>> Subject: Re: [rsyslog] fromhost-ip
> >> >>>>>
> >> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> David,
> >> >>>>>>
> >> >>>>>> Single quotes are right in the scripting engine (double
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> quotes are reserved
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> for future use - they shall provide the capability to
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> extend macros, e.g.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> to be the string
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> "BC").
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> that is the normal behavior of single vs double quotes, but in
> >> such
> >> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
> >> >>>>> it's only
> >> >>>>> when you have variables involved that there would be a
> >> difference.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>> Jup, that's right - but double quotes are not yet implemented
> ;)
> >> >>>>
> >> >>>> Rainer
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>> David Lang
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> I don't have an idea what may be wrong, but running rsyslog
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> in debug mode
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> will most probably pinpoint it.
> >> >>>>>>
> >> >>>>>> Rainer
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> -----Original Message-----
> >> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
> >> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>> david at lang.hm
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
> >> >>>>>>> To: rsyslog-users
> >> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
> >> >>>>>>>
> >> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> When I switched to double quotes I get the error in
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>> /var/log/syslog and
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> no logs are collected?
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>> what was the error you got this time?
> >> >>>>>>>
> >> >>>>>>> David Lang
> >> >>>>>>>
> >> >>>>>>> _______________________________________________
> >> >>>>>>> rsyslog mailing list
> >> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>>>> http://www.rsyslog.com
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>> _______________________________________________
> >> >>>>>> rsyslog mailing list
> >> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>>> http://www.rsyslog.com
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> _______________________________________________
> >> >>>>> rsyslog mailing list
> >> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>> http://www.rsyslog.com
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>> _______________________________________________
> >> >>>> rsyslog mailing list
> >> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>> http://www.rsyslog.com
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> >
> >>
> >>
> >> --
> >> Reminds me of my expedition into the wilds of Afghanistan. We lost
> our
> >> corkscrew and were compelled to live on food and water for several
> >> days. -
> >> WC Fields
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From epiphani at gmail.com  Tue Jan 19 19:55:55 2010
From: epiphani at gmail.com (Aaron Wiebe)
Date: Tue, 19 Jan 2010 13:55:55 -0500
Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
Message-ID: <e7ca40f71001191055k308bfa4dw2682e4fc702fefa8@mail.gmail.com>

Greetings,

I'm trying to sort out applying rulesets to IMUDP, and there is not
module-specific documentation for imudp as there is with imtcp.

What is the equivilent for udp input of:

$InputTCPServerInputName
$InputTCPServerBindRuleSet

?

I want to be able to apply rules to specific ports in the same way I
can with tcp...  Changing TCP to UDP doesn't seem to work.

-Aaron


From david at lang.hm  Wed Jan 20 00:26:15 2010
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jan 2010 15:26:15 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
	<aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
	<alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
Message-ID: <alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>

I've now had v5.3.6 running on my production environemnets since friday 
with no problems

one side effect of the cleanups is that previously when I had multiple 
filters write to one file I was getting lots of corrupt lines, but the 
change to have omfile write each transaction rather than just as the 
buffer filled up seems to have eliminated this (it went from 10's of 
thousands of corrupted lines/day to none over the weekend and monday, 
tonight's report will be the acid test to see if it's fully cleaned up)

I realize there is still a window for corruption (if two output threads 
running at the same time both decide they need to write at the same time), 
but it seems that in practice it's effectively gone.

David Lang

On Fri, 15 Jan 2010, david at lang.hm wrote:

> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> BTW, I'm actually surprised that you don't encounter those problems 
>> yourself.
>
> I'm running 5.3.5 still, I haven't had time to build a new version (hopefully 
> tomorrow)
>
> David Lang
>


From rgerhards at hq.adiscon.com  Wed Jan 20 12:09:18 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 12:09:18 +0100
Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
References: <e7ca40f71001191055k308bfa4dw2682e4fc702fefa8@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103710@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Aaron Wiebe
> Sent: Tuesday, January 19, 2010 7:56 PM
> To: rsyslog-users
> Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
> 
> Greetings,
> 
> I'm trying to sort out applying rulesets to IMUDP, and there is not
> module-specific documentation for imudp as there is with imtcp.
> 
> What is the equivilent for udp input of:
> 
> $InputTCPServerInputName
> $InputTCPServerBindRuleSet
> 
> ?
> 
> I want to be able to apply rules to specific ports in the same way I
> can with tcp...  Changing TCP to UDP doesn't seem to work.

In v4, imudp has considerable less functionality than imtcp has. I think I
changed that only in v5. For example, you can NOT bind a ruleset to a
listener in imudp.

Rainer


From rgerhards at hq.adiscon.com  Wed Jan 20 16:19:46 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 16:19:46 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>

David,

thanks for the feedback - and a quick note.

With the new engine, you can do ruleset inclusion (via omruleset[1]). That is
probably *the* method to handle files that are written to by multiple
actions. Of course, no need to change if all works in default config. But you
can gain some extra performance by using buffered mode (for busy files) and
to use that, you need to have only one action write to each file. This is
where ruleset inclusion enters the game.

Rainer

[1] http://www.rsyslog.com/doc-omruleset.html

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 12:26 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> I've now had v5.3.6 running on my production environemnets since friday
> with no problems
> 
> one side effect of the cleanups is that previously when I had multiple
> filters write to one file I was getting lots of corrupt lines, but the
> change to have omfile write each transaction rather than just as the
> buffer filled up seems to have eliminated this (it went from 10's of
> thousands of corrupted lines/day to none over the weekend and monday,
> tonight's report will be the acid test to see if it's fully cleaned up)
> 
> I realize there is still a window for corruption (if two output threads
> running at the same time both decide they need to write at the same
> time),
> but it seems that in practice it's effectively gone.
> 
> David Lang
> 
> On Fri, 15 Jan 2010, david at lang.hm wrote:
> 
> > On Fri, 15 Jan 2010, Michael Biebl wrote:
> >
> >> BTW, I'm actually surprised that you don't encounter those problems
> >> yourself.
> >
> > I'm running 5.3.5 still, I haven't had time to build a new version
> (hopefully
> > tomorrow)
> >
> > David Lang
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 20 16:22:12 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 16:22:12 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>

I forgot to mention:

> I realize there is still a window for corruption (if two output threads
> running at the same time both decide they need to write at the same
> time),
> but it seems that in practice it's effectively gone.

The current code writes a single line with a single API call. I guess that
call is rather atomic from an OS point of view, so the window of corruption
probably don't even exists with current rsyslog and linux code.

Rainer


From david at lang.hm  Wed Jan 20 17:57:02 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 08:57:02 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001200851350.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Rainer Gerhards wrote:

> David,
>
> thanks for the feedback - and a quick note.
>
> With the new engine, you can do ruleset inclusion (via omruleset[1]). That is
> probably *the* method to handle files that are written to by multiple
> actions. Of course, no need to change if all works in default config. But you
> can gain some extra performance by using buffered mode (for busy files) and
> to use that, you need to have only one action write to each file. This is
> where ruleset inclusion enters the game.

thanks for this, I was thinking about how this could be improved, but this 
looks like it deals with the issue.

on my central box I currently have all the logs written to one file, roll 
that every 5 min, and then at night split this into 45 different files 
based on 100 simplified program names (where I strip out versions so that 
blah-2.3[123] and blah-2.4[123] end up in the same file). I was thinking 
of experimenting to see what happened if I did this in rsyslog instead. 
This is a very good pointer to what I would need to do.

David Lang

> Rainer
>
> [1] http://www.rsyslog.com/doc-omruleset.html
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Wednesday, January 20, 2010 12:26 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> I've now had v5.3.6 running on my production environemnets since friday
>> with no problems
>>
>> one side effect of the cleanups is that previously when I had multiple
>> filters write to one file I was getting lots of corrupt lines, but the
>> change to have omfile write each transaction rather than just as the
>> buffer filled up seems to have eliminated this (it went from 10's of
>> thousands of corrupted lines/day to none over the weekend and monday,
>> tonight's report will be the acid test to see if it's fully cleaned up)
>>
>> I realize there is still a window for corruption (if two output threads
>> running at the same time both decide they need to write at the same
>> time),
>> but it seems that in practice it's effectively gone.
>>
>> David Lang
>>
>> On Fri, 15 Jan 2010, david at lang.hm wrote:
>>
>>> On Fri, 15 Jan 2010, Michael Biebl wrote:
>>>
>>>> BTW, I'm actually surprised that you don't encounter those problems
>>>> yourself.
>>>
>>> I'm running 5.3.5 still, I haven't had time to build a new version
>> (hopefully
>>> tomorrow)
>>>
>>> David Lang
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Wed Jan 20 18:00:51 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 18:00:51 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm><alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001200851350.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371D@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 5:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On Wed, 20 Jan 2010, Rainer Gerhards wrote:
> 
> > David,
> >
> > thanks for the feedback - and a quick note.
> >
> > With the new engine, you can do ruleset inclusion (via omruleset[1]).
> That is
> > probably *the* method to handle files that are written to by multiple
> > actions. Of course, no need to change if all works in default config.
> But you
> > can gain some extra performance by using buffered mode (for busy
> files) and
> > to use that, you need to have only one action write to each file.
> This is
> > where ruleset inclusion enters the game.
> 
> thanks for this, I was thinking about how this could be improved, but
> this
> looks like it deals with the issue.
> 
> on my central box I currently have all the logs written to one file,
> roll
> that every 5 min, and then at night split this into 45 different files
> based on 100 simplified program names (where I strip out versions so
> that
> blah-2.3[123] and blah-2.4[123] end up in the same file). I was
> thinking
> of experimenting to see what happened if I did this in rsyslog instead.
> This is a very good pointer to what I would need to do.

I would be quite interested in feedback on omruleset. I doubt anyone has put
it into production yet, at least in a demanding environment (aka "bugs to be
expected" ;)). Note that this functionality is very hard to configure with
the current config language... (it was omruleset that made me believe that
finally something must be done to improve that part of the system).

Rainer

> 
> David Lang
> 
> > Rainer
> >
> > [1] http://www.rsyslog.com/doc-omruleset.html
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >> Sent: Wednesday, January 20, 2010 12:26 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> I've now had v5.3.6 running on my production environemnets since
> friday
> >> with no problems
> >>
> >> one side effect of the cleanups is that previously when I had
> multiple
> >> filters write to one file I was getting lots of corrupt lines, but
> the
> >> change to have omfile write each transaction rather than just as the
> >> buffer filled up seems to have eliminated this (it went from 10's of
> >> thousands of corrupted lines/day to none over the weekend and
> monday,
> >> tonight's report will be the acid test to see if it's fully cleaned
> up)
> >>
> >> I realize there is still a window for corruption (if two output
> threads
> >> running at the same time both decide they need to write at the same
> >> time),
> >> but it seems that in practice it's effectively gone.
> >>
> >> David Lang
> >>
> >> On Fri, 15 Jan 2010, david at lang.hm wrote:
> >>
> >>> On Fri, 15 Jan 2010, Michael Biebl wrote:
> >>>
> >>>> BTW, I'm actually surprised that you don't encounter those
> problems
> >>>> yourself.
> >>>
> >>> I'm running 5.3.5 still, I haven't had time to build a new version
> >> (hopefully
> >>> tomorrow)
> >>>
> >>> David Lang
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Wed Jan 20 18:00:56 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 09:00:56 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001200857260.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Rainer Gerhards wrote:

> I forgot to mention:
>
>> I realize there is still a window for corruption (if two output threads
>> running at the same time both decide they need to write at the same
>> time),
>> but it seems that in practice it's effectively gone.
>
> The current code writes a single line with a single API call. I guess that
> call is rather atomic from an OS point of view, so the window of corruption
> probably don't even exists with current rsyslog and linux code.

even when things are batched? with 5.3.5 I was very definantly 
experianceing problems with lines getting combined in the writes when I 
had multiple outputs to the same file (using different formats to fix up 
bad input)

David Lang


From rgerhards at hq.adiscon.com  Wed Jan 20 18:02:22 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 18:02:22 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm><alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001200857260.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371E@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 6:01 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On Wed, 20 Jan 2010, Rainer Gerhards wrote:
> 
> > I forgot to mention:
> >
> >> I realize there is still a window for corruption (if two output
> threads
> >> running at the same time both decide they need to write at the same
> >> time),
> >> but it seems that in practice it's effectively gone.
> >
> > The current code writes a single line with a single API call. I guess
> that
> > call is rather atomic from an OS point of view, so the window of
> corruption
> > probably don't even exists with current rsyslog and linux code.
> 
> even when things are batched? with 5.3.5 I was very definantly
> experianceing problems with lines getting combined in the writes when I
> had multiple outputs to the same file (using different formats to fix
> up
> bad input)

good point. No, you are right. With batches, buffered mode is used by
default, with a flush at the end of batch.

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From sur5r at sur5r.net  Wed Jan 20 19:20:31 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Wed, 20 Jan 2010 19:20:31 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
Message-ID: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 6 Jan 2010 16:14:59 +0100
Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:

> which encoding should be chosen for the database when using postgres?

As far as I understand the syslog protocol (at least the legacy one), it has
no concept of character encodings at all.  So if you simply want to make sure
that everything ends up in the database "as is", then choose SQL_ASCII.

> My rsyslog version is 4.4.3.
> 
> Which client_encoding does rsyslog use in ompgsql?

Right now, it does net set an encoding by itself, so the database default
applies. If I'm not mistaken, you can even set that per user from inside of
postgres. So I would rather vote against another configuration parameter here.

> I currently have set UTF-8 on the database. It worked for a while until
> some special message arrived at the server where postgres denies the INSERT:
> 
> 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for 
> encoding "UTF8": 0xd220
> 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
> the byte sequence does not match the encoding expected by the server, which
> is controlled by "client_encoding".

Were you able to isolate the message? Or find out which program was sending
it?

> Now rsyslog is not able to log anything... it is currently spooling to disk 
> because it "hangs" at this message not being accepted by postgres.

This is bad, because if the machine is an open syslog server that simply
collects everything it gets, we have a potential DoS vector here.

I can think of three options:

* Drop the message and report that we did so. That would be rather easy,
  but might not be what people want.

* Re-insert the message after converting it from ASCII to UTF-8 or whatever
  the DB encoding is. But this might/will produce garbage if the input is not
  ASCII. It also creates more load on the system if these messages are
  frequent. Guessing the input encoding is hard or even impossible, depending
  on the set you guess from.

* Make the database SQL_ASCII. This will silently accept anything but will
  create nonsense from UTF/UCS encoded messages. Also might create trouble
  for programs like phplogcon that analyze the logs.

For me, this sums up to one question:

Can we make ompgsql UTF/UCS-clean and at the same time not choke on non-UTF8
strings? Everyone is trying to be UTF-8 clean these days, so it would be bad
if ompgsql could not keep up.

Comments please.

Regards,
Jakab Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktXSW8ACgkQ1YAhDic+adbqXACeIJcx6GW6PhSXFO1YF72PafJG
7t8AoLNwnJYMZ4bssqMZt/nkTIPWs0LI
=vuWN
-----END PGP SIGNATURE-----

From david at lang.hm  Wed Jan 20 19:44:42 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 10:44:42 -0800 (PST)
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Jakob Haufe wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> This is bad, because if the machine is an open syslog server that simply
> collects everything it gets, we have a potential DoS vector here.
>
> I can think of three options:
>
> * Drop the message and report that we did so. That would be rather easy,
>  but might not be what people want.
>
> * Re-insert the message after converting it from ASCII to UTF-8 or whatever
>  the DB encoding is. But this might/will produce garbage if the input is not
>  ASCII. It also creates more load on the system if these messages are
>  frequent. Guessing the input encoding is hard or even impossible, depending
>  on the set you guess from.
>
> * Make the database SQL_ASCII. This will silently accept anything but will
>  create nonsense from UTF/UCS encoded messages. Also might create trouble
>  for programs like phplogcon that analyze the logs.
>
> For me, this sums up to one question:
>
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on non-UTF8
> strings? Everyone is trying to be UTF-8 clean these days, so it would be bad
> if ompgsql could not keep up.

my thought is that just like we have a filter to change control characters 
to escape sequences, it would be good to have a filter to escape non-ascii 
characters. this will mangle other character sets, but they are unlikly to 
go through cleanly anyway.

David Lang


From marc.schiffbauer at mightycare.de  Thu Jan 21 01:49:47 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Thu, 21 Jan 2010 01:49:47 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <201001210149.48041.marc.schiffbauer@mightycare.de>

Am Mittwoch, 20. Januar 2010 19:20:31 schrieb Jakob Haufe:
> On Wed, 6 Jan 2010 16:14:59 +0100
> 
> Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:
> > which encoding should be chosen for the database when using postgres?
> 
> As far as I understand the syslog protocol (at least the legacy one), it
> has no concept of character encodings at all.  So if you simply want to
> make sure that everything ends up in the database "as is", then choose
> SQL_ASCII.

This is what I did in the end. And it works good now.

> 
> > My rsyslog version is 4.4.3.
> > 
> > Which client_encoding does rsyslog use in ompgsql?
> 
> Right now, it does net set an encoding by itself, so the database default
> applies. If I'm not mistaken, you can even set that per user from inside of
> postgres. So I would rather vote against another configuration parameter
> here.

ACK

> 
> > I currently have set UTF-8 on the database. It worked for a while until
> > some special message arrived at the server where postgres denies the
> > INSERT:
> > 
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen
> > if the byte sequence does not match the encoding expected by the server,
> > which is controlled by "client_encoding".
> 
> Were you able to isolate the message? Or find out which program was sending
> it?

I was able to identify it: Some servers sent data about strings found in 
system BIOS (read by dmidecode so something like that)
It was just some strange charcters in a model or device name string set by a 
hardware vendor (compaq IIRC)


> 
> > Now rsyslog is not able to log anything... it is currently spooling to
> > disk because it "hangs" at this message not being accepted by postgres.
> 
> This is bad, because if the machine is an open syslog server that simply
> collects everything it gets, we have a potential DoS vector here.
> 

True.


> I can think of three options:
> 
> * Drop the message and report that we did so. That would be rather easy,
>   but might not be what people want.
> 

But this might be the best option I guess. Maybe the original message could 
then be written to a special logfile on disk.

> * Re-insert the message after converting it from ASCII to UTF-8 or whatever
>   the DB encoding is. But this might/will produce garbage if the input is
> not ASCII. It also creates more load on the system if these messages are
> frequent. Guessing the input encoding is hard or even impossible,
> depending on the set you guess from.

Yes but this would be an option. I would vote for creating a warning message 
in these cases as well.

> 
> * Make the database SQL_ASCII. This will silently accept anything but will
>   create nonsense from UTF/UCS encoded messages. Also might create trouble
>   for programs like phplogcon that analyze the logs.
> 

This is what I did. And phplogcon had no problems at all displaying everything 
as expected. Even those strange messages that were not accepted by postgres 
look as in the original message that came via syslog.

This might only work if apache and the browser all "speak" UTF-8.



> For me, this sums up to one question:
> 
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> non-UTF8 strings? Everyone is trying to be UTF-8 clean these days, so it
> would be bad if ompgsql could not keep up.

I think this is a special case because rsyslog is not the originator of those 
messages. It "just" transports them. And because the syslog-Protocol does not 
define something like encoding in any way the best thing to do is just leave 
those strings "as-is" and make the database behind it do so as well with 
SQL_ASCII.

I thing everythign else will be error prone in some way. The Documentation of 
rsyslog should bring a big fat NOTE that the database must be SQL_ASCII as 
other wise thesrings might not be accepted.


-Marc

> 
> Comments please.
> 
> Regards,
> Jakab Haufe (sur5r)


From xkubina at fi.muni.cz  Thu Jan 21 11:21:27 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Thu, 21 Jan 2010 11:21:27 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>	<4B4DCF31.6090105@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
Message-ID: <4B582AA7.3040906@fi.muni.cz>

Rainer Gerhards wrote:
> Thanks for the code. Unfortunately, adding the config switch to it is not
> quite easy in that case (good I asked for the actual code). I'd say that you
> best do it similar to the other config directives, like the authentication
> mode. They actual directives are in the upper level code (imtcp/omfwd).
> There, they are shuffled over to the instance data, which goes along with
> each of the configured listeners/sender. Then, when a new network stream is
> created, the params are passed down to the generic stream interface and there
> passed down to the selected stream driver, which finally stores and acts on
> them. It's clumpsy and quite some work, but that is what is needed for the
> old config system. You probably need to add around 50 to 100 lines of code
> altogether to the various files. It's not complex, but easy to forget
> something. Best start by a directive (like $..AuthMode), see how it is
> handled (and passed down) in imtcp and work your way down the stack ;)
>
> Rainer 
>
>   
Hi Rainer,

I have added some code that I have thought was necessary, but
I am stuck now. In nsd_gtls.c is added function:

static rsRetVal
SetAddClientCN(nsd_t *pNsd, int mode)
{
    DEFiRet;
    nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;

    ISOBJ_TYPE_assert((pThis), nsd_gtls);
    if(mode != 0 && mode != 1) {
        errmsg.LogError(0, RS_RET_INVALID_DRVR_MODE, "error: driver mode 
%d not supported by "
                "gtls netstream driver", mode);
        ABORT_FINALIZE(RS_RET_INVALID_DRVR_MODE);
    }
       
    pThis->iAddClientCN = mode;
        dbgprintf("GTLS:%d\n", pThis->iAddClientCN);
finalize_it:
    RETiRet;
}

The "dbgprintf" shows correct value in pThis, but if I check 
pThis->iAddClientCN
later in function:

static rsRetVal
Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
{
    DEFiRet;
    ssize_t iBytesCopy; /* how many bytes are to be copied to the client 
buffer? */
    nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
    ISOBJ_TYPE_assert(pThis, nsd_gtls);

        cstr_t *pstrCN = NULL;
        const gnutls_datum *cert_list;
    unsigned int cert_list_size = 0;
        gnutls_x509_crt cert;       
        int len = 0;       
        char *buf_temp;

    if(pThis->bAbortConn)
        ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);

    if(pThis->iMode == 0) {
        CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
        FINALIZE;
    }

    /* --- in TLS mode now --- */

    /* Buffer logic applies only if we are in TLS mode. Here we
     * assume that we will switch from plain to TLS, but never back. This
     * assumption may be unsafe, but it is the model for the time being 
and I
     * do not see any valid reason why we should switch back to plain 
TCP after
     * we were in TLS mode. However, in that case we may lose something that
     * is already in the receive buffer ... risk accepted. -- rgerhards, 
2008-06-23
     */

    if(pThis->pszRcvBuf == NULL) {
        /* we have no buffer, so we need to malloc one */
        CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
        pThis->lenRcvBuf = -1;
    }

    /* now check if we have something in our buffer. If so, we satisfy
     * the request from buffer contents.
     */
    if(pThis->lenRcvBuf == -1) { /* no data present, must read */
        CHKiRet(gtlsRecordRecv(pThis));
    }

    if(pThis->lenRcvBuf == 0) { /* EOS */
        *pLenBuf = 0;
        /* in this case, we also need to free the receive buffer, if we
         * allocated one. -- rgerhards, 2008-12-03
         */
        if(pThis->pszRcvBuf != NULL) {
            free(pThis->pszRcvBuf);
            pThis->pszRcvBuf = NULL;
        }
        ABORT_FINALIZE(RS_RET_CLOSED);
    }

    /* if we reach this point, data is present in the buffer and must be 
copied */
    iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
    if(iBytesCopy > *pLenBuf) {
        iBytesCopy = *pLenBuf;
    } else {
        pThis->lenRcvBuf = -1; /* buffer will be emptied below */
    }

        dbgprintf("!!!!!!!!!!!%d!!!!!!!!!!!!!!\n\n", pThis->iAddClientCN);
        if (pThis->iAddClientCN)
        {
          if (pThis->clientCNValid != 1)
          {
            cert_list = gnutls_certificate_get_peers(pThis->sess, 
&cert_list_size);

        if(cert_list_size > 0)
            {
          // we only print information about the first certificate
          gnutls_x509_crt_init(&cert);
          gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER);

              CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));

              len = snprintf(NULL, 0, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
              if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char))) )
                return -1;

              snprintf(pThis->clientCN, len + 1, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
              pThis->clientCN[len] = '\0';               
              pThis->clientCNLen = len + 1;

              pThis->clientCNValid = 1;
            }
          }
       
          iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ? 
iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;

          buf_temp = (char*)malloc(iBytesCopy);
       
          if (buf_temp)
          {
            memset(buf_temp, 0, iBytesCopy);
            strncpy(buf_temp, pThis->clientCN, iBytesCopy);
            buf_temp[strlen(buf_temp)] ='\0';
            strncat(buf_temp, pThis->pszRcvBuf, iBytesCopy - 
strlen(buf_temp));
            buf_temp[strlen(buf_temp)] ='\0';
          }        
       
          memset(pBuf, 0, *pLenBuf);
      memcpy(pBuf, buf_temp + pThis->ptrRcvBuf, iBytesCopy);

          if (buf_temp)
            free(buf_temp);
        }
        else
        {
          memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, iBytesCopy);
    }

        pThis->ptrRcvBuf += iBytesCopy;
    *pLenBuf = iBytesCopy;

finalize_it:
    dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf 
%d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
    RETiRet;
}

The value is zero. Can you help me what I have to check in the sources code?

Thanks.

Regards,

Tomas


From sur5r at sur5r.net  Thu Jan 21 21:33:00 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Thu, 21 Jan 2010 21:33:00 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
Message-ID: <20100121213300.2abb07bf@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 20 Jan 2010 10:44:42 -0800 (PST)
david at lang.hm wrote:

> my thought is that just like we have a filter to change control characters 
> to escape sequences, it would be good to have a filter to escape non-ascii 
> characters. this will mangle other character sets, but they are unlikly to 
> go through cleanly anyway.

This is not an escaping issue, but an issue of byte sequences that are not
valid UTF8. That's why PostgreSQL rejects them. So we either need to make
ompgsql set SQL_ASCII as a client encoding (which will result in extended
characters being transcoded to UTF-8, which results in garbage) or make the
database SQL_ASCII.

Regards,
Jakob Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktYuf0ACgkQ1YAhDic+adY60QCbBqyEzDJtaEiWmg1cqKlMEJ2N
PnwAn2wAfPIpGlCOx2LdPJivrElU83Bu
=eTVw
-----END PGP SIGNATURE-----

From sur5r at sur5r.net  Thu Jan 21 22:26:26 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Thu, 21 Jan 2010 22:26:26 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<201001210149.48041.marc.schiffbauer@mightycare.de>
Message-ID: <20100121222626.083c7a49@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 21 Jan 2010 01:49:47 +0100
Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:

> Am Mittwoch, 20. Januar 2010 19:20:31 schrieb Jakob Haufe:
> > * Drop the message and report that we did so. That would be rather easy,
> >   but might not be what people want.
> > 
> 
> But this might be the best option I guess. Maybe the original message could 
> then be written to a special logfile on disk.

And then you have to check every now and then whether something ended up
there? That's not nice, and rather complex to implement as well (file name
should be configurable, maybe size limited, rotated, whatever)

> > For me, this sums up to one question:
> > 
> > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days, so it
> > would be bad if ompgsql could not keep up.
> 
> I think this is a special case because rsyslog is not the originator of
> those messages. It "just" transports them. And because the syslog-Protocol
> does not define something like encoding in any way the best thing to do is
> just leave those strings "as-is" and make the database behind it do so as
> well with SQL_ASCII.

I like the idea of seeing rsyslog as some kind of transport only. This is the
best argument for switching to SQL_ASCII altogether so far.

Rainer, do you have any thoughts on this?

> I thing everythign else will be error prone in some way. The Documentation
> of rsyslog should bring a big fat NOTE that the database must be SQL_ASCII
> as other wise thesrings might not be accepted.

Yes, and the createDB.sql for ompgsql should be changed as well.

Regards,
Jakob Haufe (sur5r)


- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
=1POP
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Fri Jan 22 10:51:41 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 10:51:41 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>

V5 has the capability to discard messages that cause an action failure.
However, this is mostly untested yet, AND the action must support it by
providing proper status information - it must differentiate between
system-induced errors (which can be retried) and message-induced errors
(which need the discard). ompgsql currently does not provide that status
information.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Wednesday, January 20, 2010 7:21 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Wed, 6 Jan 2010 16:14:59 +0100
> Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:
> 
> > which encoding should be chosen for the database when using postgres?
> 
> As far as I understand the syslog protocol (at least the legacy one),
> it has
> no concept of character encodings at all.  So if you simply want to
> make sure
> that everything ends up in the database "as is", then choose SQL_ASCII.
> 
> > My rsyslog version is 4.4.3.
> >
> > Which client_encoding does rsyslog use in ompgsql?
> 
> Right now, it does net set an encoding by itself, so the database
> default
> applies. If I'm not mistaken, you can even set that per user from
> inside of
> postgres. So I would rather vote against another configuration
> parameter here.
> 
> > I currently have set UTF-8 on the database. It worked for a while
> until
> > some special message arrived at the server where postgres denies the
> INSERT:
> >
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence
> for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also
> happen if
> > the byte sequence does not match the encoding expected by the server,
> which
> > is controlled by "client_encoding".
> 
> Were you able to isolate the message? Or find out which program was
> sending
> it?
> 
> > Now rsyslog is not able to log anything... it is currently spooling
> to disk
> > because it "hangs" at this message not being accepted by postgres.
> 
> This is bad, because if the machine is an open syslog server that
> simply
> collects everything it gets, we have a potential DoS vector here.
> 
> I can think of three options:
> 
> * Drop the message and report that we did so. That would be rather
> easy,
>   but might not be what people want.
> 
> * Re-insert the message after converting it from ASCII to UTF-8 or
> whatever
>   the DB encoding is. But this might/will produce garbage if the input
> is not
>   ASCII. It also creates more load on the system if these messages are
>   frequent. Guessing the input encoding is hard or even impossible,
> depending
>   on the set you guess from.
> 
> * Make the database SQL_ASCII. This will silently accept anything but
> will
>   create nonsense from UTF/UCS encoded messages. Also might create
> trouble
>   for programs like phplogcon that analyze the logs.
> 
> For me, this sums up to one question:
> 
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> non-UTF8
> strings? Everyone is trying to be UTF-8 clean these days, so it would
> be bad
> if ompgsql could not keep up.
> 
> Comments please.
> 
> Regards,
> Jakab Haufe (sur5r)
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktXSW8ACgkQ1YAhDic+adbqXACeIJcx6GW6PhSXFO1YF72PafJG
> 7t8AoLNwnJYMZ4bssqMZt/nkTIPWs0LI
> =vuWN
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 10:54:04 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 10:54:04 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>


> my thought is that just like we have a filter to change control
> characters
> to escape sequences, it would be good to have a filter to escape non-
> ascii
> characters. this will mangle other character sets, but they are unlikly
> to
> go through cleanly anyway.

Just to be on the right path, you suggest escaping charactes with hex values
> 7f?

Rainer


From david at lang.hm  Fri Jan 22 11:07:02 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 22 Jan 2010 02:07:02 -0800 (PST)
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>

On Fri, 22 Jan 2010, Rainer Gerhards wrote:

>> my thought is that just like we have a filter to change control
>> characters
>> to escape sequences, it would be good to have a filter to escape non-
>> ascii
>> characters. this will mangle other character sets, but they are unlikly
>> to
>> go through cleanly anyway.
>
> Just to be on the right path, you suggest escaping charactes with hex values
>> 7f?

correct. they can cause as much grief (or more) than control characters.

since control characters get escaped by default, rsyslog will already 
mangle UTF8 text sent to it if the final byte is in that range.

David Lang


From rgerhards at hq.adiscon.com  Fri Jan 22 11:09:52 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:09:52 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710372A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 11:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> >> my thought is that just like we have a filter to change control
> >> characters
> >> to escape sequences, it would be good to have a filter to escape
> non-
> >> ascii
> >> characters. this will mangle other character sets, but they are
> unlikly
> >> to
> >> go through cleanly anyway.
> >
> > Just to be on the right path, you suggest escaping charactes with hex
> values
> >> 7f?
> 
> correct. they can cause as much grief (or more) than control
> characters.
> 
> since control characters get escaped by default, rsyslog will already
> mangle UTF8 text sent to it if the final byte is in that range.

jup, just wanted to be sure. that can probably be best implemented as a
property replacer option (or at the parser level, but then it applies to
everything). Note that many European languages use these characters (and
without grief), much as Asian languages use sequences which would be
destroyed by the current escaping (which thus can be turned off). But I
definitely see the value. Given it looks easy to implement, I'll see if I can
integrate an option.

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 11:15:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:15:59 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems with
	character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>

Been on the road and will be over the weekend and part of next week (thus
sluggish responses ;)).

> > > For me, this sums up to one question:
> > >
> > > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days,
> so it
> > > would be bad if ompgsql could not keep up.
> >
> > I think this is a special case because rsyslog is not the originator
> of
> > those messages. It "just" transports them. And because the syslog-
> Protocol
> > does not define something like encoding in any way the best thing to
> do is
> > just leave those strings "as-is" and make the database behind it do
> so as
> > well with SQL_ASCII.
> 
> I like the idea of seeing rsyslog as some kind of transport only. This
> is the
> best argument for switching to SQL_ASCII altogether so far.
> 
> Rainer, do you have any thoughts on this?

Let me elaborte a bit: the new IETF syslog standards *do* specify character
encoding and strongly recommend Unicode (UTF-8) to be used. Of course, this
does not solve the issue with original senders that use another, unspecified,
coding. But it helps.

Unfortunately, rsyslog's "old" code is far from being Unicode-aware. As a
side-activity, I am upgrading "old" code to "new" code, which then uses
rsyslog's string classes. While they do not yet support Unicode, it is much
easier to make them support it once all string handling is done consistently.

However, even then I need to have a build time switch to turn this on/off,
because rsyslog in Unicode mode will take not only considerably more space
(especially with larger in-memory queues), it will also considerably affect
its performance (in terms of bytes, the memory transfer rate is effectively
cut in half, as most data in syslog is character-based - also think about the
effects on cache performance).

So moving the whole system to Unicode, while desirable, is far from being a
trivial task. Having seen extremely low demand for that, I have so far opted
to do this at a very low priorty (even though that means I violate RFC5424).
	
> 
> > I thing everythign else will be error prone in some way. The
> Documentation
> > of rsyslog should bring a big fat NOTE that the database must be
> SQL_ASCII
> > as other wise thesrings might not be accepted.
> 
> Yes, and the createDB.sql for ompgsql should be changed as well.
> 

The doc needs to be written so that I can add this warning ;) Is someone with
actual Postgres knowledge up for this task. Plain text is OK, I can then
copy&paste that into a module doc template.

As for createDB.sql: let me know what I need to change, and I'll apply that
change.

Rainer

> Regards,
> Jakob Haufe (sur5r)
> 
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
> m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
> =1POP
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 11:50:20 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:50:20 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>	<4B4DCF31.6090105@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
	<4B582AA7.3040906@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103735@GRFEXC.intern.adiscon.com>

mhhh... doesn't look too bad. Maybe it's a problem with the calling sequence.
When do you call your new function? If should be called after the
nsdConstruct but before the nsdConsructFinalize (actual function names may be
slightly different).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Thursday, January 21, 2010 11:21 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to add new configuration option
> 
> Rainer Gerhards wrote:
> > Thanks for the code. Unfortunately, adding the config switch to it is
> not
> > quite easy in that case (good I asked for the actual code). I'd say
> that you
> > best do it similar to the other config directives, like the
> authentication
> > mode. They actual directives are in the upper level code
> (imtcp/omfwd).
> > There, they are shuffled over to the instance data, which goes along
> with
> > each of the configured listeners/sender. Then, when a new network
> stream is
> > created, the params are passed down to the generic stream interface
> and there
> > passed down to the selected stream driver, which finally stores and
> acts on
> > them. It's clumpsy and quite some work, but that is what is needed
> for the
> > old config system. You probably need to add around 50 to 100 lines of
> code
> > altogether to the various files. It's not complex, but easy to forget
> > something. Best start by a directive (like $..AuthMode), see how it
> is
> > handled (and passed down) in imtcp and work your way down the stack
> ;)
> >
> > Rainer
> >
> >
> Hi Rainer,
> 
> I have added some code that I have thought was necessary, but
> I am stuck now. In nsd_gtls.c is added function:
> 
> static rsRetVal
> SetAddClientCN(nsd_t *pNsd, int mode)
> {
>     DEFiRet;
>     nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 
>     ISOBJ_TYPE_assert((pThis), nsd_gtls);
>     if(mode != 0 && mode != 1) {
>         errmsg.LogError(0, RS_RET_INVALID_DRVR_MODE, "error: driver
> mode
> %d not supported by "
>                 "gtls netstream driver", mode);
>         ABORT_FINALIZE(RS_RET_INVALID_DRVR_MODE);
>     }
> 
>     pThis->iAddClientCN = mode;
>         dbgprintf("GTLS:%d\n", pThis->iAddClientCN);
> finalize_it:
>     RETiRet;
> }
> 
> The "dbgprintf" shows correct value in pThis, but if I check
> pThis->iAddClientCN
> later in function:
> 
> static rsRetVal
> Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
> {
>     DEFiRet;
>     ssize_t iBytesCopy; /* how many bytes are to be copied to the
> client
> buffer? */
>     nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
>     ISOBJ_TYPE_assert(pThis, nsd_gtls);
> 
>         cstr_t *pstrCN = NULL;
>         const gnutls_datum *cert_list;
>     unsigned int cert_list_size = 0;
>         gnutls_x509_crt cert;
>         int len = 0;
>         char *buf_temp;
> 
>     if(pThis->bAbortConn)
>         ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);
> 
>     if(pThis->iMode == 0) {
>         CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
>         FINALIZE;
>     }
> 
>     /* --- in TLS mode now --- */
> 
>     /* Buffer logic applies only if we are in TLS mode. Here we
>      * assume that we will switch from plain to TLS, but never back.
> This
>      * assumption may be unsafe, but it is the model for the time being
> and I
>      * do not see any valid reason why we should switch back to plain
> TCP after
>      * we were in TLS mode. However, in that case we may lose something
> that
>      * is already in the receive buffer ... risk accepted. --
> rgerhards,
> 2008-06-23
>      */
> 
>     if(pThis->pszRcvBuf == NULL) {
>         /* we have no buffer, so we need to malloc one */
>         CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
>         pThis->lenRcvBuf = -1;
>     }
> 
>     /* now check if we have something in our buffer. If so, we satisfy
>      * the request from buffer contents.
>      */
>     if(pThis->lenRcvBuf == -1) { /* no data present, must read */
>         CHKiRet(gtlsRecordRecv(pThis));
>     }
> 
>     if(pThis->lenRcvBuf == 0) { /* EOS */
>         *pLenBuf = 0;
>         /* in this case, we also need to free the receive buffer, if we
>          * allocated one. -- rgerhards, 2008-12-03
>          */
>         if(pThis->pszRcvBuf != NULL) {
>             free(pThis->pszRcvBuf);
>             pThis->pszRcvBuf = NULL;
>         }
>         ABORT_FINALIZE(RS_RET_CLOSED);
>     }
> 
>     /* if we reach this point, data is present in the buffer and must
> be
> copied */
>     iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
>     if(iBytesCopy > *pLenBuf) {
>         iBytesCopy = *pLenBuf;
>     } else {
>         pThis->lenRcvBuf = -1; /* buffer will be emptied below */
>     }
> 
>         dbgprintf("!!!!!!!!!!!%d!!!!!!!!!!!!!!\n\n", pThis-
> >iAddClientCN);
>         if (pThis->iAddClientCN)
>         {
>           if (pThis->clientCNValid != 1)
>           {
>             cert_list = gnutls_certificate_get_peers(pThis->sess,
> &cert_list_size);
> 
>         if(cert_list_size > 0)
>             {
>           // we only print information about the first certificate
>           gnutls_x509_crt_init(&cert);
>           gnutls_x509_crt_import(cert, &cert_list[0],
> GNUTLS_X509_FMT_DER);
> 
>               CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));
> 
>               len = snprintf(NULL, 0, "CN:%s ",
> (char*)cstrGetSzStr(pstrCN));
>               if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char)))
> )
>                 return -1;
> 
>               snprintf(pThis->clientCN, len + 1, "CN:%s ",
> (char*)cstrGetSzStr(pstrCN));
>               pThis->clientCN[len] = '\0';
>               pThis->clientCNLen = len + 1;
> 
>               pThis->clientCNValid = 1;
>             }
>           }
> 
>           iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ?
> iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;
> 
>           buf_temp = (char*)malloc(iBytesCopy);
> 
>           if (buf_temp)
>           {
>             memset(buf_temp, 0, iBytesCopy);
>             strncpy(buf_temp, pThis->clientCN, iBytesCopy);
>             buf_temp[strlen(buf_temp)] ='\0';
>             strncat(buf_temp, pThis->pszRcvBuf, iBytesCopy -
> strlen(buf_temp));
>             buf_temp[strlen(buf_temp)] ='\0';
>           }
> 
>           memset(pBuf, 0, *pLenBuf);
>       memcpy(pBuf, buf_temp + pThis->ptrRcvBuf, iBytesCopy);
> 
>           if (buf_temp)
>             free(buf_temp);
>         }
>         else
>         {
>           memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf,
> iBytesCopy);
>     }
> 
>         pThis->ptrRcvBuf += iBytesCopy;
>     *pLenBuf = iBytesCopy;
> 
> finalize_it:
>     dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf
> %d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
>     RETiRet;
> }
> 
> The value is zero. Can you help me what I have to check in the sources
> code?
> 
> Thanks.
> 
> Regards,
> 
> Tomas
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From marc.schiffbauer at mightycare.de  Fri Jan 22 15:20:01 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Fri, 22 Jan 2010 15:20:01 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
	<9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
Message-ID: <201001221520.02432.marc.schiffbauer@mightycare.de>

Am Freitag, 22. Januar 2010 11:15:59 schrieb Rainer Gerhards:
> 
> The doc needs to be written so that I can add this warning ;) Is someone
> with actual Postgres knowledge up for this task. Plain text is OK, I can
> then copy&paste that into a module doc template.
> 
> As for createDB.sql: let me know what I need to change, and I'll apply that
> change.
> 

I can write it. I will send it to you/this thread next week!

Have a nice weekend
-Marc


From rgerhards at hq.adiscon.com  Fri Jan 22 17:09:21 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 17:09:21 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problemswith
	character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<201001221520.02432.marc.schiffbauer@mightycare.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710373A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer
> Sent: Friday, January 22, 2010 3:20 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problemswith character encoding
> 
> Am Freitag, 22. Januar 2010 11:15:59 schrieb Rainer Gerhards:
> >
> > The doc needs to be written so that I can add this warning ;) Is
> someone
> > with actual Postgres knowledge up for this task. Plain text is OK, I
> can
> > then copy&paste that into a module doc template.
> >
> > As for createDB.sql: let me know what I need to change, and I'll
> apply that
> > change.
> >
> 
> I can write it. I will send it to you/this thread next week!

excellent! looking forward to it :)

Rainer


From david at lang.hm  Fri Jan 22 19:19:25 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 22 Jan 2010 10:19:25 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
 with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
	<9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>

On Fri, 22 Jan 2010, Rainer Gerhards wrote:

> However, even then I need to have a build time switch to turn this on/off,
> because rsyslog in Unicode mode will take not only considerably more space
> (especially with larger in-memory queues), it will also considerably affect
> its performance (in terms of bytes, the memory transfer rate is effectively
> cut in half, as most data in syslog is character-based - also think about the
> effects on cache performance).

if the code uses UTF-8 throughout this doesn't make sense. assuming the 
input is plain ascii, UTF-8 strings and ASCII strings should be the same 
size (there is some additional cpu cycles involved to figure out the 
length in characters for any output routines that grab substrings, but 
that should be all)

the only way things would take double the space (and therefor halve the 
memory transfer rate) is if it converts everything to UTF-16 strings 
internally. This is a bad idea to start with as UTF-16 does not handle all 
characters (which is why there is UTF-32 as well), but also because UTF-16 
is significantly more expensive to store/copy/etc than UTF-8 for the 
common case where most of the characters are ASCII.

It may be that you have picked the wrong string library to use. prior to 
UTF-8 being defined 'unicode' and UTF-16 were basicly synonomous and a 
_lot_ of string libraries have been written with this assumption 
(converting everything to UTF-16 on input and to whatever on output). If 
you can find one that can handle the strings as UTF-8 internally it should 
be able to just about eliminate the overhead.

David Lang


From sur5r at sur5r.net  Sun Jan 24 20:14:41 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Sun, 24 Jan 2010 20:14:41 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>
Message-ID: <20100124201441.7990c850@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 22 Jan 2010 10:51:41 +0100
"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:

> V5 has the capability to discard messages that cause an action failure.
> However, this is mostly untested yet, AND the action must support it by
> providing proper status information - it must differentiate between
> system-induced errors (which can be retried) and message-induced errors
> (which need the discard). ompgsql currently does not provide that status
> information.

If you can point me at some example code or docs on how to do this, I would
like to try and add this functionality to ompgsql. Does ommysql already
implement that?

Regards,
Jakob Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktcnCEACgkQ1YAhDic+ada9WACeMkawcNTL/lt5E70mWeVjd38G
ARoAn1OAkEqm7NXRMwwVzUDC3B/2TeCB
=eDPw
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Mon Jan 25 08:34:53 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 08:34:53 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>
	<20100124201441.7990c850@samsa>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710373F@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Sunday, January 24, 2010 8:15 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Fri, 22 Jan 2010 10:51:41 +0100
> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:
> 
> > V5 has the capability to discard messages that cause an action
> failure.
> > However, this is mostly untested yet, AND the action must support it
> by
> > providing proper status information - it must differentiate between
> > system-induced errors (which can be retried) and message-induced
> errors
> > (which need the discard). ompgsql currently does not provide that
> status
> > information.
> 
> If you can point me at some example code or docs on how to do this, I
> would
> like to try and add this functionality to ompgsql. Does ommysql already
> implement that?


It's pretty new functionality and there is not yet a good example plugin that
uses it (it makes most sense for database plugins, where I have limited
knowledge). It would be useful to read this first (unfortunately not an easy
read:

http://download.rsyslog.com/design.pdf

Actually implementing it is rather easy. The core point is that for
system-induced errors (those that can be retried) the plugin must return
RS_RET_SUSPENDED and for message-induced errors it must return an "real"
error state (like RS_RET_ERR, but it would be better, and I'd be glad to
include, more precise error codes). The core engine than knows what to do.
Well, the core may have undiscovered bugs right now, as this functionality
was never before used in practice. It is very criticial to think about which
error class a failure belongs to. Messages with message-induced errors are
simply thrown away, so one needs to think twice before assigning this class -
but on the contrary if such a message is flagged as system-induced, it will
block the system, just as you can currently see...

HTH
Rainer

> 
> Regards,
> Jakob Haufe (sur5r)
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktcnCEACgkQ1YAhDic+ada9WACeMkawcNTL/lt5E70mWeVjd38G
> ARoAn1OAkEqm7NXRMwwVzUDC3B/2TeCB
> =eDPw
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Jan 25 09:12:08 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 09:12:08 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 7:19 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> > However, even then I need to have a build time switch to turn this
> on/off,
> > because rsyslog in Unicode mode will take not only considerably more
> space
> > (especially with larger in-memory queues), it will also considerably
> affect
> > its performance (in terms of bytes, the memory transfer rate is
> effectively
> > cut in half, as most data in syslog is character-based - also think
> about the
> > effects on cache performance).

David,

we need to make a distinction between UTF, a transformation (and transfer)
format and UCS, the actual native encoding format here. I think you mix these
two things up. Unicode has two (primary) flavors, which are usually encoded
in UCS-16 and UCS-32 (or ws it named UCS-2 and UCS-4 - guess so), being 2 and
4 bytes respectively. UCS-16 is what is implemented for example in Windows.
It covers many of this worlds scripts, but has proven to not cover all, which
caused additional code tables and UCS-32 presentation (at least as far as I
know, I am not an Unicode expert ;)).

UTF-8 is an encoding of Unicode code tables. You can think of it as
traditional multi-byte character set which means each character takes up a
varying number of bytes. Usually, UTF representations are converted into UCS
and then UCS is used to do the processing. While UCS requires more bytes, UTF
requires parsing of the message *each time* it is processed (e.g. to check
for a string match, count character sizes, obtain a substring). So using UTF
may use up fewer bytes, but can very considerably increase processing time
need and program complexity. For US-ASCII, of course, this is no problem. But
for other encodings, the performance hit can be very sever, much more than
the hit by double memory consumption (UCS-2 is still being considered as
"sufficient" for almost all cases, even in the future).

So I don't think it would serve the non-US-ASCII world well to process the
transformation formats. I guess that's a good option if you have a US-ASCII
based system that only very occasionally needs to process a foreign language
string (and even then, you need to parse the message *each* time you access
it, specifically when obtaining substrings...).

My conclusion is that rsyslog needs to do a UTF to UCS conversion on entry to
the system and then uses UCS internally (and converts back when messages are
output). Many software systems do so, and, as I said, IMHO do so for good
reasons.

Rainer

> 
> if the code uses UTF-8 throughout this doesn't make sense. assuming the
> input is plain ascii, UTF-8 strings and ASCII strings should be the
> same
> size (there is some additional cpu cycles involved to figure out the
> length in characters for any output routines that grab substrings, but
> that should be all)
> 
> the only way things would take double the space (and therefor halve the
> memory transfer rate) is if it converts everything to UTF-16 strings
> internally. This is a bad idea to start with as UTF-16 does not handle
> all
> characters (which is why there is UTF-32 as well), but also because
> UTF-16
> is significantly more expensive to store/copy/etc than UTF-8 for the
> common case where most of the characters are ASCII.
> 
> It may be that you have picked the wrong string library to use. prior
> to
> UTF-8 being defined 'unicode' and UTF-16 were basicly synonomous and a
> _lot_ of string libraries have been written with this assumption
> (converting everything to UTF-16 on input and to whatever on output).
> If
> you can find one that can handle the strings as UTF-8 internally it
> should
> be able to just about eliminate the overhead.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Mon Jan 25 09:42:32 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 25 Jan 2010 00:42:32 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
 Problems	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001250020090.3093@asgard.lang.hm>

On Mon, 25 Jan 2010, Rainer Gerhards wrote:

> David,
>
> we need to make a distinction between UTF, a transformation (and transfer)
> format and UCS, the actual native encoding format here. I think you mix these
> two things up. Unicode has two (primary) flavors, which are usually encoded
> in UCS-16 and UCS-32 (or ws it named UCS-2 and UCS-4 - guess so), being 2 and
> 4 bytes respectively. UCS-16 is what is implemented for example in Windows.
> It covers many of this worlds scripts, but has proven to not cover all, which
> caused additional code tables and UCS-32 presentation (at least as far as I
> know, I am not an Unicode expert ;)).
>
> UTF-8 is an encoding of Unicode code tables. You can think of it as
> traditional multi-byte character set which means each character takes up a
> varying number of bytes. Usually, UTF representations are converted into UCS
> and then UCS is used to do the processing. While UCS requires more bytes, UTF
> requires parsing of the message *each time* it is processed (e.g. to check
> for a string match, count character sizes, obtain a substring). So using UTF
> may use up fewer bytes, but can very considerably increase processing time
> need and program complexity. For US-ASCII, of course, this is no problem. But
> for other encodings, the performance hit can be very sever, much more than
> the hit by double memory consumption (UCS-2 is still being considered as
> "sufficient" for almost all cases, even in the future).

thanks for the clarification on terms. I had the basic understanding, but 
not the exact terminology.

> So I don't think it would serve the non-US-ASCII world well to process the
> transformation formats. I guess that's a good option if you have a US-ASCII
> based system that only very occasionally needs to process a foreign language
> string (and even then, you need to parse the message *each* time you access
> it, specifically when obtaining substrings...).
>
> My conclusion is that rsyslog needs to do a UTF to UCS conversion on entry to
> the system and then uses UCS internally (and converts back when messages are
> output). Many software systems do so, and, as I said, IMHO do so for good
> reasons.

the question is how many different places/times are we parsing the data as 
strings, vs how many places are we just moving the data around as 
essentially opaque blobs.

when we receive and parse the message we have to deal with the data as 
strings of characters, but this is generally done in one pass through the 
input data, so it would be about the same to process the data as-is as to 
convert it to UCS-2 (let alone then processing it as UCS-2). This pass can 
calculate the number of characters in the string (i.e. 'length') and store 
it

then these parsed chunks of data get copied around (in complex 
configurations with many queues, they get copied around a LOT).

At some point (or points) comparisons are made, but in most cases these 
comparisons can be done byte-by-byte, you don't actually have to parse the 
data (for regex matches you do, and for contains you would have to check 
the byte prior to the start of the match to make sure that that first 
matching byte isn't the tail end of a prior character, but I think that's 
it)

and then eventually we create the output string. At that point we are 
assembling the string from the various substrings that we have stored 
(which still can be treated as a series of bytes). It's only when the 
property replacer is invoked with either character positions or options 
that the data needs to be treated as a UTF-8 string instead of a series of 
bytes again. Yes there are a lot of things that it can do, but how much 
are they used in real life (other than setting a max length, which could 
be special cased to not be checked if the number of bytes is less than 
the length you are checking against)?

Remember that this is not general-purpose input and output that we are 
dealing with, it's logs. And like it or not, most logs really are in 
ASCII, simply because for so many years there was no option.

Also consider that the input and output stages can be split into multiple 
worker threads, while the queue manipulation (and copying) is done inside 
locks.

It may be best to leave the data as UTF-8 unless the property replacer has 
been given options, and then let the property replacer convert the data, 
work on it, and convert it back (if there is more than one option being 
invoked)

David Lang


From zhengfeng at cn.fujitsu.com  Mon Jan 25 11:36:47 2010
From: zhengfeng at cn.fujitsu.com (zhengfeng)
Date: Mon, 25 Jan 2010 18:36:47 +0800
Subject: [rsyslog] help: what induced syslogd test results are so fluctuated?
Message-ID: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>

Hi~,all

For test the performance of syslogd in RHEL5.4GA, I had written one small program by calling the interfaces syslog()...

But the results is not very steady.
Please look the results below:

The 1st time, some days ago, I test 10 times, every time sending logs to syslogd 30 secs, after then reboot.

Results:
      1,110  1,101  1,103  1,092  1,088  1,101  1,098  1,096  1,087  1,087  2.12% 



But Today , I use the same codes and method to test , the results are:

      1,295  1,292  1,297  1,291  1,288  1,287  1,284  1,279  1,275  1,270  2.13% 


(1290-1100)/1100 > 10%

Why the resluts upwards are so different? What induced? And how can i avoid that?

Thanks a lot.:-D  

From david at lang.hm  Mon Jan 25 14:10:44 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 25 Jan 2010 05:10:44 -0800 (PST)
Subject: [rsyslog] help: what induced syslogd test results are so
 fluctuated?
In-Reply-To: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>
References: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>
Message-ID: <alpine.DEB.2.00.1001250500200.3093@asgard.lang.hm>

On Mon, 25 Jan 2010, zhengfeng wrote:

> Hi~,all
>
> For test the performance of syslogd in RHEL5.4GA, I had written one small program by calling the interfaces syslog()...
>
> But the results is not very steady.
> Please look the results below:
>
> The 1st time, some days ago, I test 10 times, every time sending logs to syslogd 30 secs, after then reboot.
>
> Results:
>      1,110  1,101  1,103  1,092  1,088  1,101  1,098  1,096  1,087  1,087  2.12%
>
>
>
> But Today , I use the same codes and method to test , the results are:
>
>      1,295  1,292  1,297  1,291  1,288  1,287  1,284  1,279  1,275  1,270  2.13%
>
>
> (1290-1100)/1100 > 10%
>
> Why the resluts upwards are so different? What induced? And how can i avoid that?

there are a lot of things that could be causing this. However you didn't 
give us enough information to figure it out.

what else is running on the system?

what are the specs of the system? (is it a single core single processor, 
or do you have multiple processors)

what filesystem are you using?

when you say that you write logs for 30 seconds and reboot, are you 
allowing rsyslog to flush out pending writes, or are you loosing all logs 
that haven't been written yet (this will also involve what version of 
rsyslog are you testing and how do you have it configured)

how much cpu is rsyslog using during the time that it is running the test? 
(total time, and if you have multiple cpu cores on the system, the peak 
cpu of individual threads)

how large is the queue that rsyslog is allowed to use?

if the numbers you are reporting are the total logs written, they seem 
very low. on a current rsyslog with reasonable hardware I would expect the 
numbers to be tens of thousands of log messages per second, it may be that 
the bottleneck is the process writing to syslog() rather than rsyslog 
itself.

30 seconds is a very short time for a test, depending on your filesystem 
it may not have written anything out to the disk by the time you finish 
the test.


David Lang


From rgerhards at hq.adiscon.com  Mon Jan 25 14:38:30 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 14:38:30 +0100
Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog 5.3.6
	(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>

Hi Michael,

finally, good news: I finally managed to reproduce the problem under 32-bit
Debian sid. It looks like only v5 is affected, and not the quite similar
v4-beta. I will now try to pinpoint the problem (hoping that the repro is
stable).

Will post more news when I have it.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, January 17, 2010 12:52 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>> -----Original Message-----
> >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >>> Sent: Friday, January 15, 2010 11:57 PM
> >>> To: rsyslog-users
> >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>>
> >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>> > Michael,
> >>> >
> >>> > Fix now in git, links at the bug tracker:
> >>> >
> >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >>> >
> >>> > Please let me know if it works for you (the patch is a bit
> trickier
> >>> than it
> >>> > looks, so confirmations would be good).
> >>>
> >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> >>> But now I'm getting a crash when rsyslog encounters the xconsole
> pipe
> >>> config.
> >>
> >> I am a bit puzzled, but will try to reproduce that on my Debian box.
> I assume
> >> stock Debian config?
> >
> > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> > got the crash. I use the default rsyslog.conf from the official
> debian
> > package.
> 
> As an additonal hint: If I start xconsole (a process reading from
> /dev/xconsole) before I start rsyslogd, then the crash does not occur.
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Tue Jan 26 05:18:04 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Tue, 26 Jan 2010 04:18:04 +0000
Subject: [rsyslog] [patch]: fix from-host dns name reporting,
 add microseconds to MySQL
In-Reply-To: <mailman.112.1264479218.2883.rsyslog@lists.adiscon.com>
References: <mailman.112.1264479218.2883.rsyslog@lists.adiscon.com>
Message-ID: <4B5E6CFC.7070303@p6m7g8.com>

re-send from subscribed address

> ------------------------------------------------------------------------
> 
> Subject:
> [patch]: fix from-host dns name reporting, add microseconds to MySQL
> From:
> "Philip M. Gollucci" <pgollucci at ridecharge.com>
> Date:
> Mon, 25 Jan 2010 20:04:10 -0800
> To:
> rsyslog-users <rsyslog at lists.adiscon.com>
> 
> To:
> rsyslog-users <rsyslog at lists.adiscon.com>
> CC:
> "cristianorolim at hotmail.com" <cristianorolim at hotmail.com>
> 
> 
> Hi,
> 
> I have the following local patches running on a patched 5.3.6 on 50+ 
> FreeBSD machines at $work.
> 
> 1) I wanted the FQDN for $from-host
>     yes, I have this var set $PreserveFQDN to on
> 
> 2) I *need* microseconds int time:::*
> 3) Optionally add an #ifdef for the _PATH_MODDIR
>     to get the right default for fbsd
> 
> Maybe someone can explain to me why getting the host name is so complex, 
> it shouldn't be.
> 
> You can fetch them here --
> ASF mirror
> 1)
> http://people.freebsd.org/~pgollucci/patch-runtime__datetime.c
> http://people.freebsd.org/~pgollucci/patch-runtime__msg.c
> 2,3)
> http://people.freebsd.org/~pgollucci/patch-tools__syslogd.c
> 
> FreeBSD mirror
> 1)
> http://people.apache.org/~pgollucci/patch-runtime__datetime.c
> 2,3)
> http://people.apache.org/~pgollucci/patch-runtime__msg.c
> http://people.apache.org/~pgollucci/patch-tools__syslogd.c
> 
> 
> 
> 


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From rgerhards at hq.adiscon.com  Tue Jan 26 16:44:57 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 16:44:57 +0100
Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
	5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>

An update: this bug is *not* related to debian, but requires certain compiler
settings. I now also get it on Fedora. Also, it has nothing to do with the
named pipe. In fact, there seems to be a problem with the way direct queues
are handled. I don't have full details yet, but finally I begin to understand
the issue. It is a v5-only bug, introduced by the new queue engine. Direct
queues (at least action queues) can cause a segfault, at least if something
goes wrong in the action. Will post more details and/or a fix when I have
better info.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, January 25, 2010 2:39 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
> 5.3.6(v5-beta) released
> 
> Hi Michael,
> 
> finally, good news: I finally managed to reproduce the problem under
> 32-bit
> Debian sid. It looks like only v5 is affected, and not the quite
> similar
> v4-beta. I will now try to pinpoint the problem (hoping that the repro
> is
> stable).
> 
> Will post more news when I have it.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > Sent: Sunday, January 17, 2010 12:52 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >
> > 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > >>> -----Original Message-----
> > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > >>> Sent: Friday, January 15, 2010 11:57 PM
> > >>> To: rsyslog-users
> > >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > >>>
> > >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > >>> > Michael,
> > >>> >
> > >>> > Fix now in git, links at the bug tracker:
> > >>> >
> > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> > >>> >
> > >>> > Please let me know if it works for you (the patch is a bit
> > trickier
> > >>> than it
> > >>> > looks, so confirmations would be good).
> > >>>
> > >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of
> 5.3.6.
> > >>> But now I'm getting a crash when rsyslog encounters the xconsole
> > pipe
> > >>> config.
> > >>
> > >> I am a bit puzzled, but will try to reproduce that on my Debian
> box.
> > I assume
> > >> stock Debian config?
> > >
> > > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and
> then
> > > got the crash. I use the default rsyslog.conf from the official
> > debian
> > > package.
> >
> > As an additonal hint: If I start xconsole (a process reading from
> > /dev/xconsole) before I start rsyslogd, then the crash does not
> occur.
> >
> >
> >
> > --
> > Why is it that all of the instruments seeking intelligent life in the
> > universe are pointed away from Earth?
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 26 17:48:19 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 17:48:19 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid - was:RE:
	rsyslog5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>

OK, once the problematic spot is found, a fix is not far away...

The (very small) patch is self-explanatory, please see:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bab3ee566c883ac88df369ec
32df0c9100f97343

I've run it through a couple of tests now, and both theory and practice seem
to agree that this was the bug. 

Michael,
I'd appreciate if you could check if this solves the issue for you as well.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, January 26, 2010 4:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog v5 crash on Debian sid - was:RE:
> rsyslog5.3.6(v5-beta) released
> 
> An update: this bug is *not* related to debian, but requires certain
> compiler
> settings. I now also get it on Fedora. Also, it has nothing to do with
> the
> named pipe. In fact, there seems to be a problem with the way direct
> queues
> are handled. I don't have full details yet, but finally I begin to
> understand
> the issue. It is a v5-only bug, introduced by the new queue engine.
> Direct
> queues (at least action queues) can cause a segfault, at least if
> something
> goes wrong in the action. Will post more details and/or a fix when I
> have
> better info.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Monday, January 25, 2010 2:39 PM
> > To: rsyslog-users
> > Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
> > 5.3.6(v5-beta) released
> >
> > Hi Michael,
> >
> > finally, good news: I finally managed to reproduce the problem under
> > 32-bit
> > Debian sid. It looks like only v5 is affected, and not the quite
> > similar
> > v4-beta. I will now try to pinpoint the problem (hoping that the
> repro
> > is
> > stable).
> >
> > Will post more news when I have it.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > > Sent: Sunday, January 17, 2010 12:52 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > >
> > > 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > > > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > >>> -----Original Message-----
> > > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > > >>> Sent: Friday, January 15, 2010 11:57 PM
> > > >>> To: rsyslog-users
> > > >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > > >>>
> > > >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > >>> > Michael,
> > > >>> >
> > > >>> > Fix now in git, links at the bug tracker:
> > > >>> >
> > > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> > > >>> >
> > > >>> > Please let me know if it works for you (the patch is a bit
> > > trickier
> > > >>> than it
> > > >>> > looks, so confirmations would be good).
> > > >>>
> > > >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of
> > 5.3.6.
> > > >>> But now I'm getting a crash when rsyslog encounters the
> xconsole
> > > pipe
> > > >>> config.
> > > >>
> > > >> I am a bit puzzled, but will try to reproduce that on my Debian
> > box.
> > > I assume
> > > >> stock Debian config?
> > > >
> > > > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > > > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and
> > then
> > > > got the crash. I use the default rsyslog.conf from the official
> > > debian
> > > > package.
> > >
> > > As an additonal hint: If I start xconsole (a process reading from
> > > /dev/xconsole) before I start rsyslogd, then the crash does not
> > occur.
> > >
> > >
> > >
> > > --
> > > Why is it that all of the instruments seeking intelligent life in
> the
> > > universe are pointed away from Earth?
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Tue Jan 26 18:57:10 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 26 Jan 2010 18:57:10 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid - was:RE:
	rsyslog5.3.6(v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001260957r6d7cae7gc2ad4f21e3955fc2@mail.gmail.com>

010/1/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> OK, once the problematic spot is found, a fix is not far away...
>
> The (very small) patch is self-explanatory, please see:
>
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bab3ee566c883ac88df369ec
> 32df0c9100f97343
>
> I've run it through a couple of tests now, and both theory and practice seem
> to agree that this was the bug.
>
> Michael,
> I'd appreciate if you could check if this solves the issue for you as well.

Looks like you nailed the bug.

I can no longer reproduce the crash with the above patch applied.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Tue Jan 26 18:59:07 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 18:59:07 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid -
	was:RE:rsyslog5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
	<aa76cdac1001260957r6d7cae7gc2ad4f21e3955fc2@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103769@GRFEXC.intern.adiscon.com>

> > Michael,
> > I'd appreciate if you could check if this solves the issue for you as
> well.
> 
> Looks like you nailed the bug.
> 
> I can no longer reproduce the crash with the above patch applied.

Excellent. I am going through some minor things which may be useful to fix,
but that probably means we'll have a re-release soon :)

Rainer


From rgerhards at hq.adiscon.com  Wed Jan 27 07:28:02 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 07:28:02 +0100
Subject: [rsyslog] Tools to detect stack Adressing Problems?
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710376D@GRFEXC.intern.adiscon.com>

Hi all,

since I have begun to use the valgrind memory debugger routinely in
development (some two years ago), the quality of the source has much
increased. Unfortunately, however, valgrind is not able to detect problems
related to misaddressing variables on the stack. The 5.3.6 bug I was hunting
for almost a week is a good example of this. Valgrind also provides only
limited support for global data, as far as I know (and see from testing
results).

This becomes an even more important restriction as I moved a lot of former
heap memory use to the stack for performance reasons. I remember at least one
more major bug hunting effort that was hard to find because it affected only
stack space.

So I am currently looking for tools that could complement valgrind by
providing good stack checking capabilities. As one tool, mudflap was
suggested to me. It sounds interesting, but gives me a very hard time [very
hard to read debug output (no symbolic names for dlloade'ed modules, (false?)
reports for areas where I can not see anything wrong as well as frequent
(threading-related?) crashes when running under instrumentation). Maybe I am
just misinterpreting the output...

In short: I would highly appreciate suggestions for tools that can help with
debugging stack memory access (global data would be a plus) - and/or
instructions on how to interpret mudflap, if that is considered to be *the*
tool for that use case.

Thanks,
Rainer


From janfrode at tanso.net  Wed Jan 27 13:42:38 2010
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Wed, 27 Jan 2010 13:42:38 +0100
Subject: [rsyslog] filtering postfix/smtpd
Message-ID: <20100127124238.GA25239@janfrode.ibm.com>

I'm drowning in logs from postfix/smtpd, and need to filter these
messages out to a separate file. The maillog looks something like:

Jan 27 13:34:02 asav5.example.net postfix/lmtp[31977]::  53843908E2: to=<me at example.com>, relay=127.0.0.1[127.0.0.1]:10020, delay=0.54, delays=0.03/0.33/0.01/0.49, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 249FB906AD)
Jan 27 13:34:02 asav3.example.net postfix/smtpd[12077]::  connect from 26.81-111-54.customer.example.net[21.111.54.26]
Jan 27 13:34:02 asav5.example.net postfix/qmgr[32165]::  53843908E2: removed
Jan 27 13:34:02 asav3.mro.example.net postfix/smtpd[12077]::  disconnect from 26.81-111-54.customer.example.net[21.111.54.26]

So I want to separate out the lines from "postfix/smtpd" to
its own file, and not touch the postfix/lmtp or postfix/qmgr
or whatever-lines.

>From the documentation it seems to me that I should be able
to use:

	:programname,   isequal,        "postfix/smtpd" -?HourlyMaillogNonSplunked;MaillogTemplate
	:programname,   isequal,        "postfix/smtpd"        ~

But these doesn't match anything. If I use simply "postfix",
it matched all "postfix/*" messages:

	:programname,   isequal,        "postfix" -?HourlyMaillogNonSplunked;MaillogTemplate
	:programname,   isequal,        "postfix"        ~

So, any idea for how I can match just "postfix/smtpd" ?


  -jf



From rgerhards at hq.adiscon.com  Wed Jan 27 14:40:37 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 14:40:37 +0100
Subject: [rsyslog] filtering postfix/smtpd
References: <20100127124238.GA25239@janfrode.ibm.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103776@GRFEXC.intern.adiscon.com>

Hi,

could you run it in debug mode and post the relevant part of a log message
being processed? I guess that %programname% gets some weird value...

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Wednesday, January 27, 2010 1:43 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] filtering postfix/smtpd
> 
> I'm drowning in logs from postfix/smtpd, and need to filter these
> messages out to a separate file. The maillog looks something like:
> 
> Jan 27 13:34:02 asav5.example.net postfix/lmtp[31977]::  53843908E2:
> to=<me at example.com>, relay=127.0.0.1[127.0.0.1]:10020, delay=0.54,
> delays=0.03/0.33/0.01/0.49, dsn=2.0.0, status=sent (250 2.0.0 Ok:
> queued as 249FB906AD)
> Jan 27 13:34:02 asav3.example.net postfix/smtpd[12077]::  connect from
> 26.81-111-54.customer.example.net[21.111.54.26]
> Jan 27 13:34:02 asav5.example.net postfix/qmgr[32165]::  53843908E2:
> removed
> Jan 27 13:34:02 asav3.mro.example.net postfix/smtpd[12077]::
> disconnect from 26.81-111-54.customer.example.net[21.111.54.26]
> 
> So I want to separate out the lines from "postfix/smtpd" to
> its own file, and not touch the postfix/lmtp or postfix/qmgr
> or whatever-lines.
> 
> >From the documentation it seems to me that I should be able
> to use:
> 
> 	:programname,   isequal,        "postfix/smtpd" -
> ?HourlyMaillogNonSplunked;MaillogTemplate
> 	:programname,   isequal,        "postfix/smtpd"        ~
> 
> But these doesn't match anything. If I use simply "postfix",
> it matched all "postfix/*" messages:
> 
> 	:programname,   isequal,        "postfix" -
> ?HourlyMaillogNonSplunked;MaillogTemplate
> 	:programname,   isequal,        "postfix"        ~
> 
> So, any idea for how I can match just "postfix/smtpd" ?
> 
> 
>   -jf
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 27 15:27:51 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 15:27:51 +0100
Subject: [rsyslog] 8Bit character escaping - was: RE: PostgreSQL: Problems
	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103778@GRFEXC.intern.adiscon.com>

David,

I have now added the functionality to escape 8-bit characters. Patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=85045270f69e4dcb25c409c9
661e96e3172d7f30

I hope it is useful.

I plan to release a new v5 devel soon, probably tomorrow or friday.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 11:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> >> my thought is that just like we have a filter to change control
> >> characters
> >> to escape sequences, it would be good to have a filter to escape
> non-
> >> ascii
> >> characters. this will mangle other character sets, but they are
> unlikly
> >> to
> >> go through cleanly anyway.
> >
> > Just to be on the right path, you suggest escaping charactes with hex
> values
> >> 7f?
> 
> correct. they can cause as much grief (or more) than control
> characters.
> 
> since control characters get escaped by default, rsyslog will already
> mangle UTF8 text sent to it if the final byte is in that range.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Wed Jan 27 20:16:36 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Wed, 27 Jan 2010 19:16:36 +0000
Subject: [rsyslog] config file help
Message-ID: <4B609114.9090103@p6m7g8.com>

rsyslog.conf:

...
if $facility == '1' && $priority == '7' then ~
*.* :ommysql:localhost,logs,logs,logs;db_std ## not actual l/p
EOF

select facility,priority, count(1) as c
from syslogs
where facility = 1
and priority = 7
group by facility,priority;
+----------+----------+------+
| facility | priority | c    |
+----------+----------+------+
|        1 |        7 | 1637 |
+----------+----------+------+
1 row in set (0.00 sec)

am I missing something ? I just want to throw it away.



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From joe at joetify.com  Wed Jan 27 20:16:57 2010
From: joe at joetify.com (Joe Williams)
Date: Wed, 27 Jan 2010 11:16:57 -0800
Subject: [rsyslog] tripling of log lines
Message-ID: <4B609129.6040301@joetify.com>


I have an odd issue where with a specific config I see triple of each 
line in the log but using another config that should effectively be 
doing the same thing it does not.

Doing something like the following produces three identical lines in the 
log.

$template DbFormat,"%timegenerated% %HOSTNAME% %msg:::drop-last-lf%\n"
$template DbNotice,"<%= @log_dir 
%>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
local2.notice      -?DbNotice;DbFormat

Example:

Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0


The following produces the expected one line in the log without duplication.

$template DbFormat,"%timegenerated% %fromhost% %msg:::drop-last-lf%\n"
$template DbNotice,"<%= @log_dir 
%>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"

if \
   ( $syslogfacility-text == 'local2' ) \
     and \
   ( $syslogseverity-text == 'notice' ) \
then -?DbNotice;DbFormat


For brevity in both examples I just showed an example for one severity 
level, we have individual log templates and filters for all of them.

Any ideas what could be going on here? To me these should be equivalent.

Thanks.

-Joe

-- 
Name: Joseph A. Williams
Email: joe at joetify.com
Blog: http://www.joeandmotorboat.com/



From rgerhards at hq.adiscon.com  Wed Jan 27 21:34:48 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 21:34:48 +0100
Subject: [rsyslog] tripling of log lines
References: <4B609129.6040301@joetify.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103783@GRFEXC.intern.adiscon.com>

Maybe you don't discard the message after writing it?

Please see:

http://cookbook.rsyslog.com/node7.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Joe Williams
> Sent: Wednesday, January 27, 2010 8:17 PM
> To: rsyslog-users
> Subject: [rsyslog] tripling of log lines
> 
> 
> I have an odd issue where with a specific config I see triple of each
> line in the log but using another config that should effectively be
> doing the same thing it does not.
> 
> Doing something like the following produces three identical lines in
> the
> log.
> 
> $template DbFormat,"%timegenerated% %HOSTNAME% %msg:::drop-last-lf%\n"
> $template DbNotice,"<%= @log_dir
> %>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
> local2.notice      -?DbNotice;DbFormat
> 
> Example:
> 
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> 
> 
> The following produces the expected one line in the log without
> duplication.
> 
> $template DbFormat,"%timegenerated% %fromhost% %msg:::drop-last-lf%\n"
> $template DbNotice,"<%= @log_dir
> %>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
> 
> if \
>    ( $syslogfacility-text == 'local2' ) \
>      and \
>    ( $syslogseverity-text == 'notice' ) \
> then -?DbNotice;DbFormat
> 
> 
> For brevity in both examples I just showed an example for one severity
> level, we have individual log templates and filters for all of them.
> 
> Any ideas what could be going on here? To me these should be
> equivalent.
> 
> Thanks.
> 
> -Joe
> 
> --
> Name: Joseph A. Williams
> Email: joe at joetify.com
> Blog: http://www.joeandmotorboat.com/
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 27 22:26:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 22:26:24 +0100
Subject: [rsyslog] config file help
References: <4B609114.9090103@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Wednesday, January 27, 2010 8:17 PM
> To: rsyslog-users
> Subject: [rsyslog] config file help
> 
> rsyslog.conf:
> 
> ...
> if $facility == '1' && $priority == '7' then ~

I don't have the code at hand right now, but I guess the codes must be
numeric:

if $facility == 1 && $priority == 7 then ~

The scripting engine may not spit out a meaningful error message - it is in
its infancy with no time til today to complete it...

Rainer

> *.* :ommysql:localhost,logs,logs,logs;db_std ## not actual l/p
> EOF
> 
> select facility,priority, count(1) as c
> from syslogs
> where facility = 1
> and priority = 7
> group by facility,priority;
> +----------+----------+------+
> | facility | priority | c    |
> +----------+----------+------+
> |        1 |        7 | 1637 |
> +----------+----------+------+
> 1 row in set (0.00 sec)
> 
> am I missing something ? I just want to throw it away.
> 
> 
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Consultant,                       P6M7G8 Inc.
> Sr. System Admin,                 Ridecharge Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Wed Jan 27 22:59:00 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Wed, 27 Jan 2010 21:59:00 +0000
Subject: [rsyslog] config file help
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
Message-ID: <4B60B724.8060506@p6m7g8.com>

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
>> Sent: Wednesday, January 27, 2010 8:17 PM
>> To: rsyslog-users
>> Subject: [rsyslog] config file help
>>
>> rsyslog.conf:
>>
>> ...
>> if $facility == '1' && $priority == '7' then ~
> 
> I don't have the code at hand right now, but I guess the codes must be
> numeric:
> 
> if $facility == 1 && $priority == 7 then ~

Ha, you think I didn't try that too.  No dice either way.

Forget meaningful, it spits out nothing [with debugging and/or ktracing]
Just merely goes along and 'works' too well.


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From pgollucci at p6m7g8.com  Thu Jan 28 02:35:45 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Thu, 28 Jan 2010 01:35:45 +0000
Subject: [rsyslog] config file help
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
Message-ID: <4B60E9F1.6000800@p6m7g8.com>

Rainer Gerhards wrote:
> if $facility == 1 && $priority == 7 then ~
looking up the text values in includes/syslog.h does  work

user.debug ~

but

1.7 ~

does not.

-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From jbondc at openmv.com  Thu Jan 28 03:32:13 2010
From: jbondc at openmv.com (Jonathan Bond-Caron)
Date: Wed, 27 Jan 2010 21:32:13 -0500
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
	Problems	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
Message-ID: <000f01ca9fc2$1a574840$4f05d8c0$@com>

On Mon Jan 25 03:12 AM, Rainer Gerhards wrote:
> So I don't think it would serve the non-US-ASCII world well to process 
> the transformation formats. I guess that's a good option if you have a 
> US-ASCII based system that only very occasionally needs to process a 
> foreign language string (and even then, you need to parse the message
> *each* time you access it, specifically when obtaining substrings...).
> 
> My conclusion is that rsyslog needs to do a UTF to UCS conversion on 
> entry to the system and then uses UCS internally (and converts back 
> when messages are output). Many software systems do so, and, as I 
> said, IMHO do so for good reasons.
> 

What about adding a property option ~ 'normalize-utf8' where invalid utf8
bytes would be escaped?

$template dbFormat,"insert into text_logs (utf8_message) values
('%msg:::normalize-utf8%')",stdsql

I can probably dig through postgresql to find the code to detect invalid
utf8 bytes.

I'm not sure if I understood but are you suggesting that all input to
rsyslog is converted to UCS internally?
That seems like a huge performance penalty to pay when most people (?) log
US-ascii or UTF-8 data.




From david at lang.hm  Thu Jan 28 06:32:07 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 27 Jan 2010 21:32:07 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE:
 PostgreSQL:	Problems	with character encoding
In-Reply-To: <000f01ca9fc2$1a574840$4f05d8c0$@com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
	<000f01ca9fc2$1a574840$4f05d8c0$@com>
Message-ID: <alpine.DEB.2.00.1001272129130.28980@asgard.lang.hm>

On Wed, 27 Jan 2010, Jonathan Bond-Caron wrote:

> On Mon Jan 25 03:12 AM, Rainer Gerhards wrote:
>> So I don't think it would serve the non-US-ASCII world well to process
>> the transformation formats. I guess that's a good option if you have a
>> US-ASCII based system that only very occasionally needs to process a
>> foreign language string (and even then, you need to parse the message
>> *each* time you access it, specifically when obtaining substrings...).
>>
>> My conclusion is that rsyslog needs to do a UTF to UCS conversion on
>> entry to the system and then uses UCS internally (and converts back
>> when messages are output). Many software systems do so, and, as I
>> said, IMHO do so for good reasons.
>>
>
> What about adding a property option ~ 'normalize-utf8' where invalid utf8
> bytes would be escaped?
>
> $template dbFormat,"insert into text_logs (utf8_message) values
> ('%msg:::normalize-utf8%')",stdsql
>
> I can probably dig through postgresql to find the code to detect invalid
> utf8 bytes.

Rainer just added a property option to escape characters > 127. you could 
probably take that patch and basicly clone it to make a version that only 
escapes things if they aren't valid UTF8 instead.

> I'm not sure if I understood but are you suggesting that all input to
> rsyslog is converted to UCS internally?
> That seems like a huge performance penalty to pay when most people (?) log
> US-ascii or UTF-8 data.

right now rsyslog doesn't do any unicode stuff, it treats everything as a 
string of bytes (with some code to escape specific characters). He is 
saying that the path he has been planning to take would convert everything 
to UCS internally. you saw my argument against that.

David Lang


From rgerhards at hq.adiscon.com  Thu Jan 28 08:52:00 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 08:52:00 +0100
Subject: [rsyslog] config file help
References: <4B609114.9090103@p6m7g8.com><9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60E9F1.6000800@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103788@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Thursday, January 28, 2010 2:36 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] config file help
> 
> Rainer Gerhards wrote:
> > if $facility == 1 && $priority == 7 then ~
> looking up the text values in includes/syslog.h does  work
> 
> user.debug ~
> 
> but
> 
> 1.7 ~

These kind of filters are a different beast (and the traditional ones).
Rsyslog has three types of filters:

- the traditional ones
- property based
- script bases

Functionality increases on the way down, but also performance decreases.
Filters evolved, so each class has the syntax that best fits it. Note that
the if statement above and the traditional filter user.debug is *very*
different when looking from the executed code. User.debug is *much* faster
than starting up the script logic for the same thing. Should have mentioned
that yesterday...

Rainer

> 
> does not.
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Consultant,                       P6M7G8 Inc.
> Sr. System Admin,                 Ridecharge Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Jan 28 09:04:40 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 09:04:40 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE:
	PostgreSQL:	Problems	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com><000f01ca9fc2$1a574840$4f05d8c0$@com>
	<alpine.DEB.2.00.1001272129130.28980@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710378A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 28, 2010 6:32 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problems with character encoding
> 
> On Wed, 27 Jan 2010, Jonathan Bond-Caron wrote:
> > I'm not sure if I understood but are you suggesting that all input to
> > rsyslog is converted to UCS internally?
> > That seems like a huge performance penalty to pay when most people
> (?) log
> > US-ascii or UTF-8 data.
> 
> right now rsyslog doesn't do any unicode stuff, it treats everything as
> a
> string of bytes (with some code to escape specific characters). He is
> saying that the path he has been planning to take would convert
> everything
> to UCS internally. you saw my argument against that.

I didn't yet respond to the original message because David's argument is a
good one and I did not yet have time to think it over. Please note that there
are many subtle issues, especially when combining it with the demands of the
relevant RFCs (and if I implement it, I will definitely take a path that is
standards-compliant). David's argument and proposed solutions sounds good to
me, though I have some long-term concerns (eg. Can we really expect that
Japanese/Chinese systems always use US-ASCII for the core logging information
- I do not truly believe in that...).

However, I simply have no time to implement Unicode right now, so what I most
probably will do is copy over this valuable discussion and arguments into the
design doc, so that I have them ready at hand when I can turn into that
direction.

But in general, I now tend to agree to David's argument and think that it can
probably even speed up the process of a full Unicode implementation.

Rainer


From tbergfeld at hq.adiscon.com  Thu Jan 28 09:11:08 2010
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Thu, 28 Jan 2010 09:11:08 +0100
Subject: [rsyslog] rsyslog 5.3.7 (v5-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710378B@GRFEXC.intern.adiscon.com>

Hi all,

We have released a new v5-beta, version 5.3.7. Most importantly, it contains
the fixes for the problem with named pipes that Michael Biebl discovered.
There are also some other fixes (see changelog for detail). No new
functionality is included.

Once again, this is scheduled to become the new v5-stable, if no further
issues exist. As such, we would appreciate if you could try out the version
and report back your experience (even if everything works).
See Changelog for more details.

ChangeLog:

http://www.rsyslog.com/Article437.phtml

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-192.phtml


As always, feedback is appreciated.

Best regards,
Tom Bergfeld
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html .
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Jan 28 18:38:55 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 18:38:55 +0100
Subject: [rsyslog] config file help
In-Reply-To: <4B60B724.8060506@p6m7g8.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60B724.8060506@p6m7g8.com>
Message-ID: <1264700335.11821.2.camel@localhost>

On Wed, 2010-01-27 at 22:59 +0100, Philip M. Gollucci wrote:
> Rainer Gerhards wrote:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> >> Sent: Wednesday, January 27, 2010 8:17 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] config file help
> >>
> >> rsyslog.conf:
> >>
> >> ...
> >> if $facility == '1' && $priority == '7' then ~
> >
> > I don't have the code at hand right now, but I guess the codes must
> be
> > numeric:
> >
> > if $facility == 1 && $priority == 7 then ~
> 
> Ha, you think I didn't try that too.  No dice either way.
> 
> Forget meaningful, it spits out nothing [with debugging and/or
> ktracing]
> Just merely goes along and 'works' too well.

I just tried it out. I got the following error message:

===
2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
expression [try http://www.rsyslog.com/e/2051 ]
2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
$priority == 7 then ~"
2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
without actions will be discarded
2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
could not interpret master config file
'/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]
===

Not too precise (as expected), but far from not existent ;)

The syntax error is &&, you need to use "and". Also, the property names
were incorrect. So the correct line would have been:

if $syslogfacility == 1 and $syslogseverity == 7 then ~

While I have verified that this line works, you are far better of
(performance-wise) with the traditional priority filter that you now
use.

Rainer



From pgollucci at p6m7g8.com  Thu Jan 28 18:41:41 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Thu, 28 Jan 2010 17:41:41 +0000
Subject: [rsyslog] config file help
In-Reply-To: <1264700335.11821.2.camel@localhost>
References: <4B609114.9090103@p6m7g8.com>	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>	<4B60B724.8060506@p6m7g8.com>
	<1264700335.11821.2.camel@localhost>
Message-ID: <4B61CC55.1030106@p6m7g8.com>

> I just tried it out. I got the following error message:
> 
> ===
> 2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
> expression [try http://www.rsyslog.com/e/2051 ]
> 2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
> in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
> $priority == 7 then ~"
> 2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
> without actions will be discarded
> 2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
> could not interpret master config file
> '/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]

well thats useful at least.  I wonder why I don't see it.

> The syntax error is &&, you need to use "and". Also, the property names
> were incorrect. So the correct line would have been:
> 
> if $syslogfacility == 1 and $syslogseverity == 7 then ~
d'oh

> 
> While I have verified that this line works, you are far better of
> (performance-wise) with the traditional priority filter that you now
> use.
Yes!

Thx!


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From rgerhards at hq.adiscon.com  Thu Jan 28 18:44:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 18:44:24 +0100
Subject: [rsyslog] config file help
In-Reply-To: <4B61CC55.1030106@p6m7g8.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60B724.8060506@p6m7g8.com> <1264700335.11821.2.camel@localhost>
	<4B61CC55.1030106@p6m7g8.com>
Message-ID: <1264700664.11821.4.camel@localhost>

On Thu, 2010-01-28 at 17:41 +0000, Philip M. Gollucci wrote:
> > I just tried it out. I got the following error message:
> > 
> > ===
> > 2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
> > expression [try http://www.rsyslog.com/e/2051 ]
> > 2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
> > in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
> > $priority == 7 then ~"
> > 2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
> > without actions will be discarded
> > 2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
> > could not interpret master config file
> > '/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]
> 
> well thats useful at least.  I wonder why I don't see it.
> 

well, I guess that's the ole question on alternatives to using the
logging system itself to log error messages... The mailing list has a
couple of posts on this, one I thread I think in December or early this
month. I guess you did not capture syslog messages themselves.

Rainer

> > The syntax error is &&, you need to use "and". Also, the property names
> > were incorrect. So the correct line would have been:
> > 
> > if $syslogfacility == 1 and $syslogseverity == 7 then ~
> d'oh
> 
> > 
> > While I have verified that this line works, you are far better of
> > (performance-wise) with the traditional priority filter that you now
> > use.
> Yes!
> 
> Thx!
> 
> 




From david at lang.hm  Fri Jan 29 04:21:00 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 28 Jan 2010 19:21:00 -0800 (PST)
Subject: [rsyslog] no v5.3.7 announcement?
Message-ID: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>

I see it in git, I even see an announcement on freshmeat, but I didn't see 
an announcement that it was released here ;-)

for those who have missed it, 5.3.7 includes a couple fixes that were 
discussed here over the last couple of weeks.

David Lang


From rgerhards at hq.adiscon.com  Fri Jan 29 14:56:21 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jan 2010 14:56:21 +0100
Subject: [rsyslog] no v5.3.7 announcement?
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>

Mhhh... Tom sent it out yesterday, and I also see it in the archive:

http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html

Maybe we have some mail delivery problems...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 29, 2010 4:21 AM
> To: rsyslog-users
> Subject: [rsyslog] no v5.3.7 announcement?
> 
> I see it in git, I even see an announcement on freshmeat, but I didn't
> see
> an announcement that it was released here ;-)
> 
> for those who have missed it, 5.3.7 includes a couple fixes that were
> discussed here over the last couple of weeks.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ktm at rice.edu  Fri Jan 29 14:59:17 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Fri, 29 Jan 2010 07:59:17 -0600
Subject: [rsyslog] no v5.3.7 announcement?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
Message-ID: <20100129135917.GT1221@it.is.rice.edu>

I saw it here.

Ken
On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
> 
> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
> 
> Maybe we have some mail delivery problems...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Friday, January 29, 2010 4:21 AM
> > To: rsyslog-users
> > Subject: [rsyslog] no v5.3.7 announcement?
> > 
> > I see it in git, I even see an announcement on freshmeat, but I didn't
> > see
> > an announcement that it was released here ;-)
> > 
> > for those who have missed it, 5.3.7 includes a couple fixes that were
> > discussed here over the last couple of weeks.
> > 
> > David Lang
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Fri Jan 29 16:15:53 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 29 Jan 2010 07:15:53 -0800 (PST)
Subject: [rsyslog] no v5.3.7 announcement?
In-Reply-To: <20100129135917.GT1221@it.is.rice.edu>
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
	<20100129135917.GT1221@it.is.rice.edu>
Message-ID: <alpine.DEB.2.00.1001290715390.28980@asgard.lang.hm>

in that case, sorry for the noise.

David Lang

On Fri, 29 Jan 2010, Kenneth Marshall wrote:

> I saw it here.
>
> Ken
> On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
>> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
>>
>> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
>>
>> Maybe we have some mail delivery problems...
>>
>> Rainer
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Friday, January 29, 2010 4:21 AM
>>> To: rsyslog-users
>>> Subject: [rsyslog] no v5.3.7 announcement?
>>>
>>> I see it in git, I even see an announcement on freshmeat, but I didn't
>>> see
>>> an announcement that it was released here ;-)
>>>
>>> for those who have missed it, 5.3.7 includes a couple fixes that were
>>> discussed here over the last couple of weeks.
>>>
>>> David Lang
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Fri Jan 29 16:50:38 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jan 2010 16:50:38 +0100
Subject: [rsyslog] no v5.3.7 announcement?
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com><20100129135917.GT1221@it.is.rice.edu>
	<alpine.DEB.2.00.1001290715390.28980@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71037A3@GRFEXC.intern.adiscon.com>

no probalem at all - better twice than never. We had some problems with mail
delivery in december, and so I am always alerted if something in that
direction comes up...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 29, 2010 4:16 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] no v5.3.7 announcement?
> 
> in that case, sorry for the noise.
> 
> David Lang
> 
> On Fri, 29 Jan 2010, Kenneth Marshall wrote:
> 
> > I saw it here.
> >
> > Ken
> > On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
> >> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
> >>
> >> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
> >>
> >> Maybe we have some mail delivery problems...
> >>
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >>> Sent: Friday, January 29, 2010 4:21 AM
> >>> To: rsyslog-users
> >>> Subject: [rsyslog] no v5.3.7 announcement?
> >>>
> >>> I see it in git, I even see an announcement on freshmeat, but I
> didn't
> >>> see
> >>> an announcement that it was released here ;-)
> >>>
> >>> for those who have missed it, 5.3.7 includes a couple fixes that
> were
> >>> discussed here over the last couple of weeks.
> >>>
> >>> David Lang
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From gmanea-ma at lists.mszet.de  Sat Jan  2 11:43:51 2010
From: gmanea-ma at lists.mszet.de (Michael =?UTF-8?B?U3RyYXXDnw==?=)
Date: Sat, 2 Jan 2010 11:43:51 +0100
Subject: [rsyslog] rsyslog config-test croaks
Message-ID: <20100102114351.00f8f080@merkur.home.mszet.de>

Hello

I'm Michael Strau? and i am tying to use rsyslog.

I'm using debian Lenny width rsyslog 4.4.2 from backports.org.
I changed the debian standard configuration and adapted it for my needs.

It works in my opinion fully correctly. So I am impossible to
comprehend this error:

# rsyslogd -c4 -N1
rsyslogd: version 4.4.2, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: the last error occured in /etc/rsyslog.conf, line 25
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

These are the criticized Lines:

24: $ModLoad imrelp
25: $InputRELPServerRun 2514
26:

I can't detect a failure.

Best regards
Michael

-- 




From mbiebl at gmail.com  Sat Jan  2 15:41:05 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sat, 2 Jan 2010 15:41:05 +0100
Subject: [rsyslog] rsyslog config-test croaks
In-Reply-To: <20100102114351.00f8f080@merkur.home.mszet.de>
References: <20100102114351.00f8f080@merkur.home.mszet.de>
Message-ID: <aa76cdac1001020641r2be6200cnbfa29607a100e963@mail.gmail.com>

2010/1/2 Michael Strau? <gmanea-ma at lists.mszet.de>:
> Hello
>
> I'm Michael Strau? and i am tying to use rsyslog.
>
> I'm using debian Lenny width rsyslog 4.4.2 from backports.org.
> I changed the debian standard configuration and adapted it for my needs.
>
> It works in my opinion fully correctly. So I am impossible to
> comprehend this error:
>
> # rsyslogd -c4 -N1
> rsyslogd: version 4.4.2, config validation run (level 1), master config /etc/rsyslog.conf
> rsyslogd: the last error occured in /etc/rsyslog.conf, line 25
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> These are the criticized Lines:
>
> 24: $ModLoad imrelp
> 25: $InputRELPServerRun 2514
> 26:
>
> I can't detect a failure.

Have you installed the rsyslog-relp package which contains the imrelp module?

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From gmanea-ma at lists.mszet.de  Sat Jan  2 21:20:26 2010
From: gmanea-ma at lists.mszet.de (Michael =?UTF-8?B?U3RyYXXDnw==?=)
Date: Sat, 2 Jan 2010 21:20:26 +0100
Subject: [rsyslog] rsyslog config-test croaks
References: <20100102114351.00f8f080@merkur.home.mszet.de>
	<aa76cdac1001020641r2be6200cnbfa29607a100e963@mail.gmail.com>
Message-ID: <20100102212026.65b0fd56@merkur.home.mszet.de>

On Sat, 2 Jan 2010 15:41:05 +0100, Michael Biebl <mbiebl at gmail.com> wrote:

> 
> Have you installed the rsyslog-relp package which contains the imrelp module?
> 

Yes, and also it receives the messages from the client correctly.
This is merely a cosmetic problem.

Regards,
Michael

-- 




From ktm at rice.edu  Tue Jan  5 20:53:49 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Tue, 5 Jan 2010 13:53:49 -0600
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
Message-ID: <20100105195349.GO18110@it.is.rice.edu>

I am running rsyslog version 4.2.0 on a Redhat 5 machine
and noticed slow logins to the box. The strace on the login
sshd shows the following:

9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
9937       0.000019 connect(4, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be restarted)
5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 9844
5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [], SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted system call)
5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished ...>
9937       8.001608 <... sendto resumed> ) = 90
9937       0.000028 close(4)            = 0
9937       0.000039 read(6, "\0\0\5\36", 4) = 4
9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
9937       0.000104 close(6)            = 0
9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9938
9937       0.000032 alarm(0)            = 102
9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
...

The problem seems to be caused by writing to /dev/log which should
be being managed by the rsyslog program. I see a similar problem
reported earlier on the forum:

rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)

This was for version 3.18.4 but the symptom sounded very similar.
I restarted the rsyslog process and the login times returned to normal.
Let me know if there is something further I can do to help you debug
this matter.

Regards,
Ken


From david at lang.hm  Tue Jan  5 22:12:43 2010
From: david at lang.hm (david at lang.hm)
Date: Tue, 5 Jan 2010 13:12:43 -0800 (PST)
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
In-Reply-To: <20100105195349.GO18110@it.is.rice.edu>
References: <20100105195349.GO18110@it.is.rice.edu>
Message-ID: <alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>

this sounds like rsyslog is failing to send the logs out to the RELP 
server, and so is building up a large queue. restarting rsyslog would 
clear the queued up log messages and make it fast again.

David Lang


  On Tue, 5 Jan 2010, Kenneth Marshall wrote:

> Date: Tue, 5 Jan 2010 13:53:49 -0600
> From: Kenneth Marshall <ktm at rice.edu>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Cc: sandmant at rice.edu
> Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
> 
> I am running rsyslog version 4.2.0 on a Redhat 5 machine
> and noticed slow logins to the box. The strace on the login
> sshd shows the following:
>
> 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> 9937       0.000019 connect(4, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be restarted)
> 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 9844
> 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [], SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted system call)
> 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished ...>
> 9937       8.001608 <... sendto resumed> ) = 90
> 9937       0.000028 close(4)            = 0
> 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> 9937       0.000104 close(6)            = 0
> 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9938
> 9937       0.000032 alarm(0)            = 102
> 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> ...
>
> The problem seems to be caused by writing to /dev/log which should
> be being managed by the rsyslog program. I see a similar problem
> reported earlier on the forum:
>
> rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
>
> This was for version 3.18.4 but the symptom sounded very similar.
> I restarted the rsyslog process and the login times returned to normal.
> Let me know if there is something further I can do to help you debug
> this matter.
>
> Regards,
> Ken
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From kenneho.ndu at gmail.com  Wed Jan  6 15:57:25 2010
From: kenneho.ndu at gmail.com (Kenneth Holter)
Date: Wed, 6 Jan 2010 15:57:25 +0100
Subject: [rsyslog] rsyslog+stunnel works only when running "rsyslogd"
	fromthe shell
In-Reply-To: <3A240503F9F2194780469F072D9A70541162FF2B@m342.silverspringnet.com>
References: <c25f25140912230012s5570db5r9c1959ab0f2d5b5e@mail.gmail.com>
	<3A240503F9F2194780469F072D9A70541162FF2B@m342.silverspringnet.com>
Message-ID: <c25f25141001060657p475097b8o44be587c2a1a1fb3@mail.gmail.com>

Yeah, it's really old, but for now I'll have to stick with it. :(


I added the "-d" option to rsyslog daemon, and came across this:

1098717504: Called fprintlog, logging to builtin-fwd 127.0.0.1:61514/tcp
1098717504: create tcp connection failed, reason Permission denied
1098717504: no working socket could be obtained
1098717504: error forwarding via tcp, suspending

Seems like the reason why it doesn't work is that it fails to create the TCP
session from itself (i.e. rsyslog) to the stunnel port. I've sent this
information to Red Hat support, but if anyone here have an ideas as to
what's causing this please do let me know.


- Kenneth

On Wed, Dec 23, 2009 at 9:59 PM, Siddhartha Jain
<sjain at silverspringnet.com>wrote:

> Kenneth,
>
> Not sure why RedHat/CentOS continue to bundle rsyslog 2.0.6. This
> version is ancient. Since 2.x, rsyslog has gone through 2.x, 4.x and now
> the current, 5.x.
>
> I would highly recommend rolling your own RPM from recent 5.x or 4.x
> code.
>
> - Siddhartha
>
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Kenneth Holter
> > Sent: Wednesday, December 23, 2009 12:13 AM
> > To: rsyslog at lists.adiscon.com
>  > Subject: [rsyslog] rsyslog+stunnel works only when running "rsyslogd"
> > fromthe shell
> >
> > Hi.
> >
> >
> > I'm running rsyslog v2.0.6 provided with my RHEL 5 installation. For
> > some
> > time now I've had rsyslog issues with some of my RHEL 5 servers, and
> > I've
> > not been able to figure out the problems, and would like to hear from
> > others
> > that may have experienced the same problem. I've been in contact with
> > Red
> > Hat support, but they've not been able to reproduce this problem, so
> > we'be
> > not succeeded in resolving the issue.
> >
> > First, let me describe my setup: My RHEL 5 servers have set up a TLS
> > tunnel
> > (using stunnel) between themselves and the log host. This works
> > perfectly.
> > I've configured rsyslog to forward messages to this tunnel by adding a
> > "
> > *.*  @@127.0.0.1:61514 " line to the bottom of /etc/rsyslog.conf file.
> > The
> > stunnel is listening on port 61514.
> >
> > On almost all my servers, this works as planned. But for some reason,
> a
> > few
> > servers are having problems forwarding messages to their stunnel
> > connection.
> > By running "tcpdump -i lo" I can see that these servers are not
> > transmitting
> > anything on the loopback interface, and are thus not forwarding
> > anything to
> > the stunnel port. One of my theories was that the line above simply
> > wasn't
> > picked up by rsyslog daemon. So I stopped the daemon, ran "rsyslogd
> -d"
> > to
> > view the debug output, and everthing works fine.
> >
> > For some reason, when I run rsyslog like this (i.e by issuing
> > "rsyslogd" in
> > the command prompt) instead of issuing "/etc/init.d/rsyslog start",
> > everything work fine. I'm really puzzled as to why this is so. Does
> > anyone
> > know why this is so? I have the exact same setup one all my servers,
> > but one
> > a small number of them have this problem.
> >
> >
> > Best regards,
> > Kenneth Holter
>  > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From marc.schiffbauer at mightycare.de  Wed Jan  6 16:14:59 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 16:14:59 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
Message-ID: <201001061615.00121.marc.schiffbauer@mightycare.de>

Hi all,

which encoding should be chosen for the database when using postgres?

My rsyslog version is 4.4.3.

Which client_encoding does rsyslog use in ompgsql?


I currently have set UTF-8 on the database. It worked for a while until some 
special message arrived at the server where postgres denies the INSERT:

2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for 
encoding "UTF8": 0xd220
2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if the 
byte sequence does not match the encoding expected by the server, which is 
controlled by "client_encoding".

Now rsyslog is not able to log anything... it is currently spooling to disk 
because it "hangs" at this message not being accepted by postgres.

Any hints?
TIA
-Marc



From marc.schiffbauer at mightycare.de  Wed Jan  6 16:48:02 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 16:48:02 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061615.00121.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
Message-ID: <201001061648.02984.marc.schiffbauer@mightycare.de>

Hi all again,

replying to myself because I think I found the solution:

With an db encoding of SQL_ASCII the postgres server will not do any character 
conversion which seems to be the right thing for syslog messages where the 
encoding cannot be determined reliably.

Maybe this is an important piece for the rsyslog documentation as well.

Now everthing is working again.

To convert my existing database I switch to user postgres and used "pg_dump -C 
syslog > syslog.sql" to dump the database. Then added a "DROP DATABASE syslog" 
before the "CREATE DATABASE", changed any encodings from "UTF-8" to 
"SQL_ASCII" (client_encoding and in the CREATE DATABASE statement) and then 
loaded the data again with "psql < syslog.sql".

-Marc



Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> Hi all,
> 
> which encoding should be chosen for the database when using postgres?
> 
> My rsyslog version is 4.4.3.
> 
> Which client_encoding does rsyslog use in ompgsql?
> 
> 
> I currently have set UTF-8 on the database. It worked for a while until
>  some special message arrived at the server where postgres denies the
>  INSERT:
> 
> 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> encoding "UTF8": 0xd220
> 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
>  the byte sequence does not match the encoding expected by the server,
>  which is controlled by "client_encoding".
> 
> Now rsyslog is not able to log anything... it is currently spooling to disk
> because it "hangs" at this message not being accepted by postgres.
> 
> Any hints?
> TIA
> -Marc
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Wed Jan  6 16:53:52 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Wed, 6 Jan 2010 09:53:52 -0600
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061648.02984.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
Message-ID: <20100106155352.GU18110@it.is.rice.edu>

Would it be possible to send the poorly behaving loggers to
a different port to allow it to be cleaned up properly? Using
SQL_ASCII does allow truly anything into the database, which
means that all the output pieces need to process it appropriately
too.

Regards,
Ken

On Wed, Jan 06, 2010 at 04:48:02PM +0100, Marc Schiffbauer wrote:
> Hi all again,
> 
> replying to myself because I think I found the solution:
> 
> With an db encoding of SQL_ASCII the postgres server will not do any character 
> conversion which seems to be the right thing for syslog messages where the 
> encoding cannot be determined reliably.
> 
> Maybe this is an important piece for the rsyslog documentation as well.
> 
> Now everthing is working again.
> 
> To convert my existing database I switch to user postgres and used "pg_dump -C 
> syslog > syslog.sql" to dump the database. Then added a "DROP DATABASE syslog" 
> before the "CREATE DATABASE", changed any encodings from "UTF-8" to 
> "SQL_ASCII" (client_encoding and in the CREATE DATABASE statement) and then 
> loaded the data again with "psql < syslog.sql".
> 
> -Marc
> 
> 
> 
> Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> > Hi all,
> > 
> > which encoding should be chosen for the database when using postgres?
> > 
> > My rsyslog version is 4.4.3.
> > 
> > Which client_encoding does rsyslog use in ompgsql?
> > 
> > 
> > I currently have set UTF-8 on the database. It worked for a while until
> >  some special message arrived at the server where postgres denies the
> >  INSERT:
> > 
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
> >  the byte sequence does not match the encoding expected by the server,
> >  which is controlled by "client_encoding".
> > 
> > Now rsyslog is not able to log anything... it is currently spooling to disk
> > because it "hangs" at this message not being accepted by postgres.
> > 
> > Any hints?
> > TIA
> > -Marc
> > 
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From marc.schiffbauer at mightycare.de  Wed Jan  6 17:32:43 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 17:32:43 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <20100106155352.GU18110@it.is.rice.edu>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
	<20100106155352.GU18110@it.is.rice.edu>
Message-ID: <201001061732.44115.marc.schiffbauer@mightycare.de>

Am Mittwoch, 6. Januar 2010 16:53:52 schrieb Kenneth Marshall:
> Would it be possible to send the poorly behaving loggers to
> a different port to allow it to be cleaned up properly? 

No, not in that case I am afraid.

An option in rsyslog that would allow it to skip/trash/log-to-a-file those bad 
messages would be a nice thing.

> Using
> SQL_ASCII does allow truly anything into the database, which
> means that all the output pieces need to process it appropriately
> too.

Yes but this is working nicely here with phplogcon.

-Marc

> 
> Regards,
> Ken
> 
> On Wed, Jan 06, 2010 at 04:48:02PM +0100, Marc Schiffbauer wrote:
> > Hi all again,
> >
> > replying to myself because I think I found the solution:
> >
> > With an db encoding of SQL_ASCII the postgres server will not do any
> > character conversion which seems to be the right thing for syslog
> > messages where the encoding cannot be determined reliably.
> >
> > Maybe this is an important piece for the rsyslog documentation as well.
> >
> > Now everthing is working again.
> >
> > To convert my existing database I switch to user postgres and used
> > "pg_dump -C syslog > syslog.sql" to dump the database. Then added a "DROP
> > DATABASE syslog" before the "CREATE DATABASE", changed any encodings from
> > "UTF-8" to "SQL_ASCII" (client_encoding and in the CREATE DATABASE
> > statement) and then loaded the data again with "psql < syslog.sql".
> >
> > -Marc
> >
> > Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> > > Hi all,
> > >
> > > which encoding should be chosen for the database when using postgres?
> > >
> > > My rsyslog version is 4.4.3.
> > >
> > > Which client_encoding does rsyslog use in ompgsql?
> > >
> > >
> > > I currently have set UTF-8 on the database. It worked for a while until
> > >  some special message arrived at the server where postgres denies the
> > >  INSERT:
> > >
> > > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > > encoding "UTF8": 0xd220
> > > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen
> > > if the byte sequence does not match the encoding expected by the
> > > server, which is controlled by "client_encoding".
> > >
> > > Now rsyslog is not able to log anything... it is currently spooling to
> > > disk because it "hangs" at this message not being accepted by postgres.
> > >
> > > Any hints?
> > > TIA
> > > -Marc
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Wed Jan  6 17:40:16 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Wed, 6 Jan 2010 10:40:16 -0600
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061732.44115.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
	<20100106155352.GU18110@it.is.rice.edu>
	<201001061732.44115.marc.schiffbauer@mightycare.de>
Message-ID: <20100106164016.GV18110@it.is.rice.edu>

On Wed, Jan 06, 2010 at 05:32:43PM +0100, Marc Schiffbauer wrote:
> Am Mittwoch, 6. Januar 2010 16:53:52 schrieb Kenneth Marshall:
> > Would it be possible to send the poorly behaving loggers to
> > a different port to allow it to be cleaned up properly? 
> 
> No, not in that case I am afraid.
> 
> An option in rsyslog that would allow it to skip/trash/log-to-a-file those bad 
> messages would be a nice thing.
> 
> > Using
> > SQL_ASCII does allow truly anything into the database, which
> > means that all the output pieces need to process it appropriately
> > too.
> 
> Yes but this is working nicely here with phplogcon.
> 
> -Marc
> 
I was more concerned about possible compromizes caused by the
ability to insert pretty arbitrary binary data into the system.
If we have this problem in the future, I will investigate other
options further. It might be possible to have the driver also
store them in a bad record table using such an option.

Cheers,
Ken


From a.smith at ukgrid.net  Thu Jan  7 15:53:36 2010
From: a.smith at ukgrid.net (Andy Smith)
Date: Thu, 07 Jan 2010 14:53:36 +0000
Subject: [rsyslog] help with config syntax
Message-ID: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>

Hi,

   Im having trouble getting the config setup how I need it. On a mail  
server I have a lot of data being written to the main messages file,  
thats because I have mail daemons writting data with a "notice"  
severity that is configured to be written to messages (so this is  
expected). How can I prevent just mail.notice going to the messages  
file while keeping all other *.notice stuff going there? I tried  
adding !mail.notice to the config for the messages file but this didnt  
seem to work...
Here is my config:

*.err;kern.warning;auth.notice;mail.crit;local7.none             
/dev/console;Tra
ditionalFormatWithPRI
mail.info;mail.notice                                    
-/var/log/maillog;Tradit
ionalFormatWithPRI
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local7.none       
  -/var/log/messages;TraditionalFormatWithPRI
security.*                                      /var/log/security
auth.info;authpriv.info                         /var/log/auth.log
lpr.info                                        /var/log/lpd-errs
ftp.info                                        /var/log/xferlog
cron.*                                          /var/log/cron
*.=debug;local7.none                                    /var/log/debug.log
*.emerg                                         *

thanks Andy.


From danson at rackspace.com  Thu Jan  7 19:43:54 2010
From: danson at rackspace.com (Daniel Anson)
Date: Thu, 7 Jan 2010 12:43:54 -0600
Subject: [rsyslog] RHEL5 rsyslog 4 rpms
Message-ID: <7616_1262890053_o07IlMci013462_8DFDF421C24C4B4883F75F4E81EF785627D32BEBD6@DFW1MXM01.RACKSPACE.CORP>

If anyone is interested, an RPM engineer I know has packaged RHEL5 rsyslog4 rpms.  These are available for public download and testing @ http://dl.iuscommunity.org/pub/ius  Any comments can be emailed directly to him at ius-coredev at lists.launchpad.net

rpms are regularly packaged by him so let him know what you think.  I believe you just have to add the yum repo.

--Daniel M. Anson
--Linux Systems Engineer



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From david at lang.hm  Fri Jan  8 17:41:33 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 8 Jan 2010 08:41:33 -0800 (PST)
Subject: [rsyslog] help with config syntax
In-Reply-To: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>
References: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>
Message-ID: <alpine.DEB.2.00.1001080840040.3341@asgard.lang.hm>

On Thu, 7 Jan 2010, Andy Smith wrote:

> Hi,
>
>   Im having trouble getting the config setup how I need it. On a mail
> server I have a lot of data being written to the main messages file,
> thats because I have mail daemons writting data with a "notice"
> severity that is configured to be written to messages (so this is
> expected). How can I prevent just mail.notice going to the messages
> file while keeping all other *.notice stuff going there? I tried
> adding !mail.notice to the config for the messages file but this didnt
> seem to work...
> Here is my config:
>
> *.err;kern.warning;auth.notice;mail.crit;local7.none  /dev/console;TraditionalFormatWithPRI
> mail.info;mail.notice -/var/log/maillog;TraditionalFormatWithPRI

at this point you can tell it to drop the message by adding the line

& ~

this tells it to use the same matchine rules as the line above, and drop 
the message (don't process it in any further rules)

David Lang

> *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local7.none
>  -/var/log/messages;TraditionalFormatWithPRI
> security.*                                      /var/log/security
> auth.info;authpriv.info                         /var/log/auth.log
> lpr.info                                        /var/log/lpd-errs
> ftp.info                                        /var/log/xferlog
> cron.*                                          /var/log/cron
> *.=debug;local7.none                                    /var/log/debug.log
> *.emerg                                         *
>
> thanks Andy.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Mon Jan 11 12:15:55 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jan 2010 12:15:55 +0100
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
References: <20100105195349.GO18110@it.is.rice.edu>
	<alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>

I think there is a patch (or a recommendation) regarding RELP in my mail
backlog. If I got it right, RELP does not necessarily detect a broken
connection, and thus no recovery action is initiated. I'll try to get to this
ASAP, but I am now the second day in office and there is still a pile of
things I need to look into ...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, January 05, 2010 10:13 PM
> To: rsyslog-users
> Cc: sandmant at rice.edu
> Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> 
> this sounds like rsyslog is failing to send the logs out to the RELP
> server, and so is building up a large queue. restarting rsyslog would
> clear the queued up log messages and make it fast again.
> 
> David Lang
> 
> 
>   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> 
> > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > From: Kenneth Marshall <ktm at rice.edu>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog at lists.adiscon.com
> > Cc: sandmant at rice.edu
> > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> >
> > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > and noticed slow logins to the box. The strace on the login
> > sshd shows the following:
> >
> > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> path="/dev/log"...}, 110) = 0
> > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be
> restarted)
> > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> 0}], WNOHANG, NULL) = 9844
> > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [],
> SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> system call)
> > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> ...>
> > 9937       8.001608 <... sendto resumed> ) = 90
> > 9937       0.000028 close(4)            = 0
> > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> > 9937       0.000104 close(6)            = 0
> > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> 0}], 0, NULL) = 9938
> > 9937       0.000032 alarm(0)            = 102
> > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [],
> SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > ...
> >
> > The problem seems to be caused by writing to /dev/log which should
> > be being managed by the rsyslog program. I see a similar problem
> > reported earlier on the forum:
> >
> > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> >
> > This was for version 3.18.4 but the symptom sounded very similar.
> > I restarted the rsyslog process and the login times returned to
> normal.
> > Let me know if there is something further I can do to help you debug
> > this matter.
> >
> > Regards,
> > Ken
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ktm at rice.edu  Mon Jan 11 14:52:19 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Mon, 11 Jan 2010 07:52:19 -0600
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5	machine
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
References: <20100105195349.GO18110@it.is.rice.edu>
	<alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
Message-ID: <20100111135218.GM1895@it.is.rice.edu>

It does seem to act like the RELP problem, but my use is only
with a regular TCP connection using @@logmachine. It had the
same symptom and restarting rsyslog cleared it up.

Regards,
Ken

On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote:
> I think there is a patch (or a recommendation) regarding RELP in my mail
> backlog. If I got it right, RELP does not necessarily detect a broken
> connection, and thus no recovery action is initiated. I'll try to get to this
> ASAP, but I am now the second day in office and there is still a pile of
> things I need to look into ...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Tuesday, January 05, 2010 10:13 PM
> > To: rsyslog-users
> > Cc: sandmant at rice.edu
> > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > machine
> > 
> > this sounds like rsyslog is failing to send the logs out to the RELP
> > server, and so is building up a large queue. restarting rsyslog would
> > clear the queued up log messages and make it fast again.
> > 
> > David Lang
> > 
> > 
> >   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> > 
> > > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > > From: Kenneth Marshall <ktm at rice.edu>
> > > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > > To: rsyslog at lists.adiscon.com
> > > Cc: sandmant at rice.edu
> > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > machine
> > >
> > > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > > and noticed slow logins to the box. The strace on the login
> > > sshd shows the following:
> > >
> > > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> > path="/dev/log"...}, 110) = 0
> > > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be
> > restarted)
> > > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > 0}], WNOHANG, NULL) = 9844
> > > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [],
> > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> > system call)
> > > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> > ...>
> > > 9937       8.001608 <... sendto resumed> ) = 90
> > > 9937       0.000028 close(4)            = 0
> > > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> > > 9937       0.000104 close(6)            = 0
> > > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > 0}], 0, NULL) = 9938
> > > 9937       0.000032 alarm(0)            = 102
> > > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [],
> > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > > ...
> > >
> > > The problem seems to be caused by writing to /dev/log which should
> > > be being managed by the rsyslog program. I see a similar problem
> > > reported earlier on the forum:
> > >
> > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> > >
> > > This was for version 3.18.4 but the symptom sounded very similar.
> > > I restarted the rsyslog process and the login times returned to
> > normal.
> > > Let me know if there is something further I can do to help you debug
> > > this matter.
> > >
> > > Regards,
> > > Ken
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From rgerhards at hq.adiscon.com  Mon Jan 11 16:39:01 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jan 2010 16:39:01 +0100
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5	machine
References: <20100105195349.GO18110@it.is.rice.edu><alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
	<20100111135218.GM1895@it.is.rice.edu>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036A0@GRFEXC.intern.adiscon.com>

A "problem" I am aware of is that a died peer (or connection dropped an
interim firewall) is not detected as broken, because no messages are
exchanged any longer. An often-used solution is KEEPALIVE, but this can also
take some time to timeout (and may have bad effects on slow connection or
those with outages of interim systems). I know that I wanted to implement the
capability to activate KEEPALIVE, but I am not sure if I found time to
actually do it. Will let you know once I can check that.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Kenneth Marshall
> Sent: Monday, January 11, 2010 2:52 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> 
> It does seem to act like the RELP problem, but my use is only
> with a regular TCP connection using @@logmachine. It had the
> same symptom and restarting rsyslog cleared it up.
> 
> Regards,
> Ken
> 
> On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote:
> > I think there is a patch (or a recommendation) regarding RELP in my
> mail
> > backlog. If I got it right, RELP does not necessarily detect a broken
> > connection, and thus no recovery action is initiated. I'll try to get
> to this
> > ASAP, but I am now the second day in office and there is still a pile
> of
> > things I need to look into ...
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > > Sent: Tuesday, January 05, 2010 10:13 PM
> > > To: rsyslog-users
> > > Cc: sandmant at rice.edu
> > > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a
> RHEL5
> > > machine
> > >
> > > this sounds like rsyslog is failing to send the logs out to the
> RELP
> > > server, and so is building up a large queue. restarting rsyslog
> would
> > > clear the queued up log messages and make it fast again.
> > >
> > > David Lang
> > >
> > >
> > >   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> > >
> > > > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > > > From: Kenneth Marshall <ktm at rice.edu>
> > > > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > > > To: rsyslog at lists.adiscon.com
> > > > Cc: sandmant at rice.edu
> > > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > > machine
> > > >
> > > > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > > > and noticed slow logins to the box. The strace on the login
> > > > sshd shows the following:
> > > >
> > > > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > > > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > > > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> > > path="/dev/log"...}, 110) = 0
> > > > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]:
> "...,
> > > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > > > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]:
> "...,
> > > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > > > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To
> be
> > > restarted)
> > > > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > > 0}], WNOHANG, NULL) = 9844
> > > > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > > > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740,
> [],
> > > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > > > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> > > system call)
> > > > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> > > ...>
> > > > 9937       8.001608 <... sendto resumed> ) = 90
> > > > 9937       0.000028 close(4)            = 0
> > > > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > > > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> > > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) =
> 1310
> > > > 9937       0.000104 close(6)            = 0
> > > > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> > > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > > > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > > > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s)
> ==
> > > 0}], 0, NULL) = 9938
> > > > 9937       0.000032 alarm(0)            = 102
> > > > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0,
> [],
> > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > > > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > > > ...
> > > >
> > > > The problem seems to be caused by writing to /dev/log which
> should
> > > > be being managed by the rsyslog program. I see a similar problem
> > > > reported earlier on the forum:
> > > >
> > > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> > > >
> > > > This was for version 3.18.4 but the symptom sounded very similar.
> > > > I restarted the rsyslog process and the login times returned to
> > > normal.
> > > > Let me know if there is something further I can do to help you
> debug
> > > > this matter.
> > > >
> > > > Regards,
> > > > Ken
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dirk.schulz at kinzesberg.de  Mon Jan 11 18:23:18 2010
From: dirk.schulz at kinzesberg.de (Dirk H. Schulz)
Date: Mon, 11 Jan 2010 18:23:18 +0100
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
Message-ID: <4B4B5E86.9050409@kinzesberg.de>

Hi folks,

I am running two central logservers using rsyslog that several dozen 
servers report to (mostly also rsyslog).

The  central logservers are writing everything into a  database and 
additionally into local logfiles.

I would like to change configuration in a way that only local messages 
are written to local logfiles, and all messages (local and received from 
remote servers) into the database.

Is this possible with Rsyslog? I have searched the documentation, but 
did not find anything helpful.

Any hint or help is appreciated.

Dirk



From david at lang.hm  Mon Jan 11 19:42:20 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jan 2010 10:42:20 -0800 (PST)
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
In-Reply-To: <4B4B5E86.9050409@kinzesberg.de>
References: <4B4B5E86.9050409@kinzesberg.de>
Message-ID: <alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>

On Mon, 11 Jan 2010, Dirk H. Schulz wrote:

> Hi folks,
>
> I am running two central logservers using rsyslog that several dozen
> servers report to (mostly also rsyslog).
>
> The  central logservers are writing everything into a  database and
> additionally into local logfiles.
>
> I would like to change configuration in a way that only local messages
> are written to local logfiles, and all messages (local and received from
> remote servers) into the database.

yes, I do something similar to this on my systems.

All logs except local logs get written to local files, all local logs get 
sent over the network (at which point they then get picked up as remote 
logs), and all logs (local or remote) get sent to a remote system.

:fromhost, !isequal, "127.0.0.1"        /var/log/messages;TraditionalFormat
:fromhost, isequal, "127.0.0.1"         @192.168.1.8;TraditionalForwardFormat
*.*             @192.168.1.2





From paul.ruiz at gmail.com  Mon Jan 11 21:46:10 2010
From: paul.ruiz at gmail.com (Paul Ruiz)
Date: Mon, 11 Jan 2010 12:46:10 -0800
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
In-Reply-To: <alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>
References: <4B4B5E86.9050409@kinzesberg.de>
	<alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>
Message-ID: <f210e0b1001111246v56fe45acnf00c241f319148e4@mail.gmail.com>

I do this by running 2 rsyslog processes, one for local logs just like
all other installations and one that does only log collection.  The
log collection one has it's own init, config and pid file.  This way I
can rely on packaged config for local logging being identical in
production and a secondary package for log collection that only
includes the conf and init script depending on the standard rsyslog
package.

/usr/sbin/rsyslogd -c4 -f /etc/rsyslog-collector.conf -i
/var/run/rsyslogd-collector.pid


On Mon, Jan 11, 2010 at 10:42 AM,  <david at lang.hm> wrote:
> On Mon, 11 Jan 2010, Dirk H. Schulz wrote:
>
>> Hi folks,
>>
>> I am running two central logservers using rsyslog that several dozen
>> servers report to (mostly also rsyslog).
>>
>> The ?central logservers are writing everything into a ?database and
>> additionally into local logfiles.
>>
>> I would like to change configuration in a way that only local messages
>> are written to local logfiles, and all messages (local and received from
>> remote servers) into the database.
>
> yes, I do something similar to this on my systems.
>
> All logs except local logs get written to local files, all local logs get
> sent over the network (at which point they then get picked up as remote
> logs), and all logs (local or remote) get sent to a remote system.
>
> :fromhost, !isequal, "127.0.0.1" ? ? ? ?/var/log/messages;TraditionalFormat
> :fromhost, isequal, "127.0.0.1" ? ? ? ? @192.168.1.8;TraditionalForwardFormat
> *.* ? ? ? ? ? ? @192.168.1.2
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From sepperlot at googlemail.com  Tue Jan 12 16:16:30 2010
From: sepperlot at googlemail.com (Sepperlot)
Date: Tue, 12 Jan 2010 16:16:30 +0100
Subject: [rsyslog] Only log from network devices to database
Message-ID: <4B4C924E.200@googlemail.com>

Hello.

I'm trying to log messages from various network devices to rsyslog and
write them into a database.
Therefore I use a setup as described in
http://www.rsyslog.com/doc-rsyslog_mysql.html

My (simple) rsyslog.conf contains the following:

$ModLoad imudp
$UDPServerAddress x.x.x.x
$UDPServerRun 1514 # standard port is used by syslog-ng

$ModLoad ommysql
*.*             :ommysql:localhost,DBNAME,DBUSER,DBPASS


This writes all arriving log messages to the database and I can watch
them with phplogcon. Up to here everything is ok and works.

Now I only want to log messages from specific network devices identified
by ip address but I'm totaly lost when it comes to combine filter
conditions and actions. I've tried

:fromhost-ip, isequal "IP.IP.IP.IP" \
:ommysql:localhost,DBNAME,DBUSER,DBPASS

*.* :fromhost-ip, isequal "IP.IP.IP.IP" \
:ommysql:localhost,DBNAME,DBUSER,DBPASS

but obvious this is BS ;)
Goal is to log only network devices and maybe later log different
devices to different databases.

The backslash is added by me only in this mail. The commands are all in
one line.

Any help is appreciated.

Best regards
Sebastian


From rgerhards at hq.adiscon.com  Tue Jan 12 17:37:09 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jan 2010 17:37:09 +0100
Subject: [rsyslog] Only log from network devices to database
References: <4B4C924E.200@googlemail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036B7@GRFEXC.intern.adiscon.com>

The config does not look obviously wrong to me (but I am bad at catching
errors...). A good suggestion is to write a debug log, it will tell you in
detail what happened during the filter evaluation.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Sepperlot
> Sent: Tuesday, January 12, 2010 4:17 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Only log from network devices to database
> 
> Hello.
> 
> I'm trying to log messages from various network devices to rsyslog and
> write them into a database.
> Therefore I use a setup as described in
> http://www.rsyslog.com/doc-rsyslog_mysql.html
> 
> My (simple) rsyslog.conf contains the following:
> 
> $ModLoad imudp
> $UDPServerAddress x.x.x.x
> $UDPServerRun 1514 # standard port is used by syslog-ng
> 
> $ModLoad ommysql
> *.*             :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> 
> This writes all arriving log messages to the database and I can watch
> them with phplogcon. Up to here everything is ok and works.
> 
> Now I only want to log messages from specific network devices
> identified
> by ip address but I'm totaly lost when it comes to combine filter
> conditions and actions. I've tried
> 
> :fromhost-ip, isequal "IP.IP.IP.IP" \
> :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> *.* :fromhost-ip, isequal "IP.IP.IP.IP" \
> :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> but obvious this is BS ;)
> Goal is to log only network devices and maybe later log different
> devices to different databases.
> 
> The backslash is added by me only in this mail. The commands are all in
> one line.
> 
> Any help is appreciated.
> 
> Best regards
> Sebastian
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From xkubina at fi.muni.cz  Wed Jan 13 12:16:06 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 12:16:06 +0100
Subject: [rsyslog] How to add new configuration option
Message-ID: <4B4DAB76.7070201@fi.muni.cz>

Hi,

I would appreciate any help with adding support for a new configuration 
directive. I have done some
code and I need now something like:
$AddClientCN [on/off].
I have read the sources to find out how rsyslog processes conf file. 
There is some linked list with
known commands. I think that it is enough to add new item to this list 
but I don't know how.
Is this my idea right?

Thanks for any help.

Regards,

Tomas


From rgerhards at hq.adiscon.com  Wed Jan 13 12:17:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 12:17:59 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>

Hi Tomas,

it's probably the simplest if you post your code so that I can give you the
relevant hints.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Wednesday, January 13, 2010 12:16 PM
> To: rsyslog-users
> Subject: [rsyslog] How to add new configuration option
> 
> Hi,
> 
> I would appreciate any help with adding support for a new configuration
> directive. I have done some
> code and I need now something like:
> $AddClientCN [on/off].
> I have read the sources to find out how rsyslog processes conf file.
> There is some linked list with
> known commands. I think that it is enough to add new item to this list
> but I don't know how.
> Is this my idea right?
> 
> Thanks for any help.
> 
> Regards,
> 
> Tomas
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 13 12:43:11 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 12:43:11 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>

Hi all,

I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
contains a number of bug fixes, some of them important for some environments.
As usual for a beta, it does not contain anything else but fixes. The full
lest can be seen in the change log.

Please note that it is my intent do replace the current (instable ;))
v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
get a few thumbs up, I may be able to accelerate promoting it to stable.

An update for the current master branch will happen soon.

ChangeLog:
http://www.rsyslog.com/Article435.phtml

Download:
http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml

I hope this release is useful.

Rainer 



From xkubina at fi.muni.cz  Wed Jan 13 14:02:31 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 14:02:31 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
Message-ID: <4B4DC467.5000903@fi.muni.cz>

Rainer Gerhards wrote:
> Hi Tomas,
>
> it's probably the simplest if you post your code so that I can give you the
> relevant hints.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
>> Sent: Wednesday, January 13, 2010 12:16 PM
>> To: rsyslog-users
>> Subject: [rsyslog] How to add new configuration option
>>
>> Hi,
>>
>> I would appreciate any help with adding support for a new configuration
>> directive. I have done some
>> code and I need now something like:
>> $AddClientCN [on/off].
>> I have read the sources to find out how rsyslog processes conf file.
>> There is some linked list with
>> known commands. I think that it is enough to add new item to this list
>> but I don't know how.
>> Is this my idea right?
>>
>> Thanks for any help.
>>
>> Regards,
>>
>> Tomas
>>     
>   
Hi Rainer,

the modified files are attached. The alternative code is marked by #if 
statement.
I had to try to do this modification because the project, I am 
interested in, needs
to verify client's authentication. I realize that the patch is something 
like a hack,
because the rsyslog's architecture doesn't provide this feature (adding 
client CN to
syslog message) and it is not proper solution, but for our needs it is 
enough.
BTW I use this templete:
template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME% 
%syslogtag%%msg%\n";

I have done a similar code for adding client principal for imgssapi.

Thanks for help.

Regards,

Tomas



From r.bhatia at ipax.at  Wed Jan 13 14:17:04 2010
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Jan 2010 14:17:04 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <4B4DC467.5000903@fi.muni.cz>
References: <4B4DAB76.7070201@fi.muni.cz>	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
	<4B4DC467.5000903@fi.muni.cz>
Message-ID: <4B4DC7D0.30300@ipax.at>

-ENOATTACHMENT

the mailinglist strips off this stuff :)

cheers,

On 01/13/2010 02:02 PM, Tomas Kubina wrote:
> the modified files are attached. The alternative code is marked by #if
> statement.
> I had to try to do this modification because the project, I am
> interested in, needs
> to verify client's authentication. I realize that the patch is something
> like a hack,
> because the rsyslog's architecture doesn't provide this feature (adding
> client CN to
> syslog message) and it is not proper solution, but for our needs it is
> enough.
> BTW I use this templete:
> template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
> %syslogtag%%msg%\n";
> 
> I have done a similar code for adding client principal for imgssapi.


-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From xkubina at fi.muni.cz  Wed Jan 13 14:48:33 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 14:48:33 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
Message-ID: <4B4DCF31.6090105@fi.muni.cz>

Rainer Gerhards wrote:
> Hi Tomas,
>
> it's probably the simplest if you post your code so that I can give you the
> relevant hints.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
>> Sent: Wednesday, January 13, 2010 12:16 PM
>> To: rsyslog-users
>> Subject: [rsyslog] How to add new configuration option
>>
>> Hi,
>>
>> I would appreciate any help with adding support for a new configuration
>> directive. I have done some
>> code and I need now something like:
>> $AddClientCN [on/off].
>> I have read the sources to find out how rsyslog processes conf file.
>> There is some linked list with
>> known commands. I think that it is enough to add new item to this list
>> but I don't know how.
>> Is this my idea right?
>>
>> Thanks for any help.
>>
>> Regards,
>>
>> Tomas
>>     
>   
Hi Rainer,

the modified files are attached. The alternative code is marked by #if
statement. I had to try to do this modification because the project,
I am interested in, needs to verify client's authentication. I realize 
that the patch is something  like a hack, because the rsyslog's 
architecture doesn't provide this feature (adding client CN to
syslog message) and it is not proper solution, but for our needs it is
enough.
BTW I use this templete:
template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
%syslogtag%%msg%\n";

I have done a similar code for adding client principal for imgssapi.

Thanks for help.

Regards,

Tomas

FILES:

nsd_gtls.c

static rsRetVal
AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew)
{
	DEFiRet;
	int gnuRet;
	nsd_gtls_t *pNew = NULL;
	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;

         ISOBJ_TYPE_assert((pThis), nsd_gtls);
	CHKiRet(nsd_gtlsConstruct(&pNew)); // TODO: prevent construct/destruct!
	CHKiRet(nsd_ptcp.Destruct(&pNew->pTcp));
	CHKiRet(nsd_ptcp.AcceptConnReq(pThis->pTcp, &pNew->pTcp));
	
	if(pThis->iMode == 0) {
		/* we are in non-TLS mode, so we are done */
		*ppNew = (nsd_t*) pNew;
		FINALIZE;
	}

	/* if we reach this point, we are in TLS mode */
	CHKiRet(gtlsInitSession(pNew));
	gtlsSetTransportPtr(pNew, ((nsd_ptcp_t*) (pNew->pTcp))->sock);
	pNew->authMode = pThis->authMode;
	pNew->pPermPeers = pThis->pPermPeers;

	/* we now do the handshake. This is a bit complicated, because we are
	 * on non-blocking sockets. Usually, the handshake will not complete
	 * immediately, so that we need to retry it some time later.
	 */
	gnuRet = gnutls_handshake(pNew->sess);
	if(gnuRet == GNUTLS_E_AGAIN || gnuRet == GNUTLS_E_INTERRUPTED) {
		pNew->rtryCall = gtlsRtry_handshake;
		dbgprintf("GnuTLS handshake does not complete immediately - setting to 
retry (this is OK and normal)\n");
	} else if(gnuRet == 0) {
		/* we got a handshake, now check authorization */
		CHKiRet(gtlsChkPeerAuth(pNew));
	} else {
		ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR);
	}

	pNew->iMode = 1; /* this session is now in TLS mode! */
#if 1
         pNew->clientCNValid = 0;
#endif
	*ppNew = (nsd_t*) pNew;

finalize_it:
	if(iRet != RS_RET_OK) {
		if(pNew != NULL)
			nsd_gtlsDestruct(&pNew);
	}
	RETiRet;
}

static rsRetVal
Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
{
	DEFiRet;
	ssize_t iBytesCopy; /* how many bytes are to be copied to the client 
buffer? */
	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
	ISOBJ_TYPE_assert(pThis, nsd_gtls);
#if 1
         cstr_t *pstrCN = NULL;
         const gnutls_datum *cert_list;
	unsigned int cert_list_size = 0;
         gnutls_x509_crt cert;
         int len = 0;
         char *buf_temp;
#endif
	if(pThis->bAbortConn)
		ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);

	if(pThis->iMode == 0) {
		CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
		FINALIZE;
	}

	/* --- in TLS mode now --- */

	/* Buffer logic applies only if we are in TLS mode. Here we
	 * assume that we will switch from plain to TLS, but never back. This
	 * assumption may be unsafe, but it is the model for the time being and I
	 * do not see any valid reason why we should switch back to plain TCP after
	 * we were in TLS mode. However, in that case we may lose something that
	 * is already in the receive buffer ... risk accepted. -- rgerhards, 
2008-06-23
	 */

	if(pThis->pszRcvBuf == NULL) {
		/* we have no buffer, so we need to malloc one */
		CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
		pThis->lenRcvBuf = -1;
	}

	/* now check if we have something in our buffer. If so, we satisfy
	 * the request from buffer contents.
	 */
	if(pThis->lenRcvBuf == -1) { /* no data present, must read */
		CHKiRet(gtlsRecordRecv(pThis));
	}

	if(pThis->lenRcvBuf == 0) { /* EOS */
		*pLenBuf = 0;
		/* in this case, we also need to free the receive buffer, if we
		 * allocated one. -- rgerhards, 2008-12-03
		 */
		if(pThis->pszRcvBuf != NULL) {
			free(pThis->pszRcvBuf);
			pThis->pszRcvBuf = NULL;
		}
		ABORT_FINALIZE(RS_RET_CLOSED);
	}

	/* if we reach this point, data is present in the buffer and must be 
copied */
	iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
	if(iBytesCopy > *pLenBuf) {
		iBytesCopy = *pLenBuf;
	} else {
		pThis->lenRcvBuf = -1; /* buffer will be emptied below */
	}
#if 1
         if (pThis->clientCNValid != 1)
         {
           cert_list = gnutls_certificate_get_peers(pThis->sess, 
&cert_list_size);

	  if(cert_list_size > 0)
           {
	    // we only print information about the first certificate
	    gnutls_x509_crt_init(&cert);
	    gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER);

             CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));

             len = snprintf(NULL, 0, "CN:%s ", (char*)cstrGetSzStr(pstrCN));
             if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char))) )
               return -1;

             snprintf(pThis->clientCN, len + 1, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
             pThis->clientCN[len] = '\0';
             pThis->clientCNLen = len + 1;

             pThis->clientCNValid = 1;
           }
         }

         iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ? 
iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;

         buf_temp = (char*)malloc(iBytesCopy);

         if (buf_temp)
         {
           memset(buf_temp, 0, iBytesCopy);
           strncpy(buf_temp, pThis->clientCN, pThis->clientCNLen);
           strncat(buf_temp, pThis->pszRcvBuf, pThis->lenRcvBuf);
           buf_temp[iBytesCopy] ='\0';
         }

         memset(pBuf, 0, *pLenBuf);
	memcpy(pBuf, buf_temp, iBytesCopy);

         if (buf_temp)
           free(buf_temp);
#else
         memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, iBytesCopy);
#endif	
         pThis->ptrRcvBuf += iBytesCopy;
	*pLenBuf = iBytesCopy;

finalize_it:
	dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf 
%d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
	RETiRet;
}

tcps_sess.c

static rsRetVal
Close(tcps_sess_t *pThis)
{
	DEFiRet;

	ISOBJ_TYPE_assert(pThis, tcps_sess);
	netstrm.Destruct(&pThis->pStrm);
	if(pThis->fromHost != NULL) {
		prop.Destruct(&pThis->fromHost);
	}
	if(pThis->fromHostIP != NULL)
		prop.Destruct(&pThis->fromHostIP);
#if 1
        if(pThis->clientPrincipal != NULL)
		free(pThis->clientPrincipal);
#endif
	RETiRet;
}

tcps_sess.h

/* the tcps_sess object */
struct tcps_sess_s {
	BEGINobjInstance;	/* Data to implement generic object - MUST be the 
first data element! */
	tcpsrv_t *pSrv;	/* pointer back to my server (e.g. for callbacks) */
	tcpLstnPortList_t *pLstnInfo;	/* pointer back to listener info */
	netstrm_t *pStrm;
	int iMsg;		 /* index of next char to store in msg */
	int bAtStrtOfFram;	/* are we at the very beginning of a new frame? */
	enum {
		eAtStrtFram,
		eInOctetCnt,
		eInMsg
	} inputState;		/* our current state */
	int iOctetsRemain;	/* Number of Octets remaining in message */
	TCPFRAMINGMODE eFraming;
	uchar *pMsg;		/* message (fragment) received */
	prop_t *fromHost;	/* host name we received messages from */
	prop_t *fromHostIP;
	void *pUsr;		/* a user-pointer */
#if 1
         char *clientPrincipal;    /* client principal */
         int clientPrincipalLen;
#endif
	rsRetVal (*DoSubmitMessage)(tcps_sess_t*, uchar*, int); /* submit 
message callback */
};




From rgerhards at hq.adiscon.com  Wed Jan 13 15:27:31 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 15:27:31 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
	<4B4DCF31.6090105@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>

Thanks for the code. Unfortunately, adding the config switch to it is not
quite easy in that case (good I asked for the actual code). I'd say that you
best do it similar to the other config directives, like the authentication
mode. They actual directives are in the upper level code (imtcp/omfwd).
There, they are shuffled over to the instance data, which goes along with
each of the configured listeners/sender. Then, when a new network stream is
created, the params are passed down to the generic stream interface and there
passed down to the selected stream driver, which finally stores and acts on
them. It's clumpsy and quite some work, but that is what is needed for the
old config system. You probably need to add around 50 to 100 lines of code
altogether to the various files. It's not complex, but easy to forget
something. Best start by a directive (like $..AuthMode), see how it is
handled (and passed down) in imtcp and work your way down the stack ;)

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Wednesday, January 13, 2010 2:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to add new configuration option
> 
> Rainer Gerhards wrote:
> > Hi Tomas,
> >
> > it's probably the simplest if you post your code so that I 
> can give you the
> > relevant hints.
> >
> > Rainer
> >
> >   
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> >> Sent: Wednesday, January 13, 2010 12:16 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] How to add new configuration option
> >>
> >> Hi,
> >>
> >> I would appreciate any help with adding support for a new 
> configuration
> >> directive. I have done some
> >> code and I need now something like:
> >> $AddClientCN [on/off].
> >> I have read the sources to find out how rsyslog processes 
> conf file.
> >> There is some linked list with
> >> known commands. I think that it is enough to add new item 
> to this list
> >> but I don't know how.
> >> Is this my idea right?
> >>
> >> Thanks for any help.
> >>
> >> Regards,
> >>
> >> Tomas
> >>     
> >   
> Hi Rainer,
> 
> the modified files are attached. The alternative code is marked by #if
> statement. I had to try to do this modification because the project,
> I am interested in, needs to verify client's authentication. 
> I realize 
> that the patch is something  like a hack, because the rsyslog's 
> architecture doesn't provide this feature (adding client CN to
> syslog message) and it is not proper solution, but for our needs it is
> enough.
> BTW I use this templete:
> template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
> %syslogtag%%msg%\n";
> 
> I have done a similar code for adding client principal for imgssapi.
> 
> Thanks for help.
> 
> Regards,
> 
> Tomas
> 
> FILES:
> 
> nsd_gtls.c
> 
> static rsRetVal
> AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew)
> {
> 	DEFiRet;
> 	int gnuRet;
> 	nsd_gtls_t *pNew = NULL;
> 	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 
>          ISOBJ_TYPE_assert((pThis), nsd_gtls);
> 	CHKiRet(nsd_gtlsConstruct(&pNew)); // TODO: prevent 
> construct/destruct!
> 	CHKiRet(nsd_ptcp.Destruct(&pNew->pTcp));
> 	CHKiRet(nsd_ptcp.AcceptConnReq(pThis->pTcp, &pNew->pTcp));
> 	
> 	if(pThis->iMode == 0) {
> 		/* we are in non-TLS mode, so we are done */
> 		*ppNew = (nsd_t*) pNew;
> 		FINALIZE;
> 	}
> 
> 	/* if we reach this point, we are in TLS mode */
> 	CHKiRet(gtlsInitSession(pNew));
> 	gtlsSetTransportPtr(pNew, ((nsd_ptcp_t*) (pNew->pTcp))->sock);
> 	pNew->authMode = pThis->authMode;
> 	pNew->pPermPeers = pThis->pPermPeers;
> 
> 	/* we now do the handshake. This is a bit complicated, 
> because we are
> 	 * on non-blocking sockets. Usually, the handshake will 
> not complete
> 	 * immediately, so that we need to retry it some time later.
> 	 */
> 	gnuRet = gnutls_handshake(pNew->sess);
> 	if(gnuRet == GNUTLS_E_AGAIN || gnuRet == GNUTLS_E_INTERRUPTED) {
> 		pNew->rtryCall = gtlsRtry_handshake;
> 		dbgprintf("GnuTLS handshake does not complete 
> immediately - setting to 
> retry (this is OK and normal)\n");
> 	} else if(gnuRet == 0) {
> 		/* we got a handshake, now check authorization */
> 		CHKiRet(gtlsChkPeerAuth(pNew));
> 	} else {
> 		ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR);
> 	}
> 
> 	pNew->iMode = 1; /* this session is now in TLS mode! */
> #if 1
>          pNew->clientCNValid = 0;
> #endif
> 	*ppNew = (nsd_t*) pNew;
> 
> finalize_it:
> 	if(iRet != RS_RET_OK) {
> 		if(pNew != NULL)
> 			nsd_gtlsDestruct(&pNew);
> 	}
> 	RETiRet;
> }
> 
> static rsRetVal
> Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
> {
> 	DEFiRet;
> 	ssize_t iBytesCopy; /* how many bytes are to be copied 
> to the client 
> buffer? */
> 	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 	ISOBJ_TYPE_assert(pThis, nsd_gtls);
> #if 1
>          cstr_t *pstrCN = NULL;
>          const gnutls_datum *cert_list;
> 	unsigned int cert_list_size = 0;
>          gnutls_x509_crt cert;
>          int len = 0;
>          char *buf_temp;
> #endif
> 	if(pThis->bAbortConn)
> 		ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);
> 
> 	if(pThis->iMode == 0) {
> 		CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
> 		FINALIZE;
> 	}
> 
> 	/* --- in TLS mode now --- */
> 
> 	/* Buffer logic applies only if we are in TLS mode. Here we
> 	 * assume that we will switch from plain to TLS, but 
> never back. This
> 	 * assumption may be unsafe, but it is the model for 
> the time being and I
> 	 * do not see any valid reason why we should switch 
> back to plain TCP after
> 	 * we were in TLS mode. However, in that case we may 
> lose something that
> 	 * is already in the receive buffer ... risk accepted. 
> -- rgerhards, 
> 2008-06-23
> 	 */
> 
> 	if(pThis->pszRcvBuf == NULL) {
> 		/* we have no buffer, so we need to malloc one */
> 		CHKmalloc(pThis->pszRcvBuf = 
> MALLOC(NSD_GTLS_MAX_RCVBUF));
> 		pThis->lenRcvBuf = -1;
> 	}
> 
> 	/* now check if we have something in our buffer. If so, 
> we satisfy
> 	 * the request from buffer contents.
> 	 */
> 	if(pThis->lenRcvBuf == -1) { /* no data present, must read */
> 		CHKiRet(gtlsRecordRecv(pThis));
> 	}
> 
> 	if(pThis->lenRcvBuf == 0) { /* EOS */
> 		*pLenBuf = 0;
> 		/* in this case, we also need to free the 
> receive buffer, if we
> 		 * allocated one. -- rgerhards, 2008-12-03
> 		 */
> 		if(pThis->pszRcvBuf != NULL) {
> 			free(pThis->pszRcvBuf);
> 			pThis->pszRcvBuf = NULL;
> 		}
> 		ABORT_FINALIZE(RS_RET_CLOSED);
> 	}
> 
> 	/* if we reach this point, data is present in the 
> buffer and must be 
> copied */
> 	iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
> 	if(iBytesCopy > *pLenBuf) {
> 		iBytesCopy = *pLenBuf;
> 	} else {
> 		pThis->lenRcvBuf = -1; /* buffer will be 
> emptied below */
> 	}
> #if 1
>          if (pThis->clientCNValid != 1)
>          {
>            cert_list = gnutls_certificate_get_peers(pThis->sess, 
> &cert_list_size);
> 
> 	  if(cert_list_size > 0)
>            {
> 	    // we only print information about the first certificate
> 	    gnutls_x509_crt_init(&cert);
> 	    gnutls_x509_crt_import(cert, &cert_list[0], 
> GNUTLS_X509_FMT_DER);
> 
>              CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));
> 
>              len = snprintf(NULL, 0, "CN:%s ", 
> (char*)cstrGetSzStr(pstrCN));
>              if ( !(pThis->clientCN = malloc((len + 
> 1)*sizeof(char))) )
>                return -1;
> 
>              snprintf(pThis->clientCN, len + 1, "CN:%s ", 
> (char*)cstrGetSzStr(pstrCN));
>              pThis->clientCN[len] = '\0';
>              pThis->clientCNLen = len + 1;
> 
>              pThis->clientCNValid = 1;
>            }
>          }
> 
>          iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < 
> *pLenBuf ? 
> iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;
> 
>          buf_temp = (char*)malloc(iBytesCopy);
> 
>          if (buf_temp)
>          {
>            memset(buf_temp, 0, iBytesCopy);
>            strncpy(buf_temp, pThis->clientCN, pThis->clientCNLen);
>            strncat(buf_temp, pThis->pszRcvBuf, pThis->lenRcvBuf);
>            buf_temp[iBytesCopy] ='\0';
>          }
> 
>          memset(pBuf, 0, *pLenBuf);
> 	memcpy(pBuf, buf_temp, iBytesCopy);
> 
>          if (buf_temp)
>            free(buf_temp);
> #else
>          memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, 
> iBytesCopy);
> #endif	
>          pThis->ptrRcvBuf += iBytesCopy;
> 	*pLenBuf = iBytesCopy;
> 
> finalize_it:
> 	dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf 
> %d, ptrRcvBuf 
> %d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
> 	RETiRet;
> }
> 
> tcps_sess.c
> 
> static rsRetVal
> Close(tcps_sess_t *pThis)
> {
> 	DEFiRet;
> 
> 	ISOBJ_TYPE_assert(pThis, tcps_sess);
> 	netstrm.Destruct(&pThis->pStrm);
> 	if(pThis->fromHost != NULL) {
> 		prop.Destruct(&pThis->fromHost);
> 	}
> 	if(pThis->fromHostIP != NULL)
> 		prop.Destruct(&pThis->fromHostIP);
> #if 1
>         if(pThis->clientPrincipal != NULL)
> 		free(pThis->clientPrincipal);
> #endif
> 	RETiRet;
> }
> 
> tcps_sess.h
> 
> /* the tcps_sess object */
> struct tcps_sess_s {
> 	BEGINobjInstance;	/* Data to implement generic 
> object - MUST be the 
> first data element! */
> 	tcpsrv_t *pSrv;	/* pointer back to my server (e.g. for 
> callbacks) */
> 	tcpLstnPortList_t *pLstnInfo;	/* pointer back to 
> listener info */
> 	netstrm_t *pStrm;
> 	int iMsg;		 /* index of next char to store 
> in msg */
> 	int bAtStrtOfFram;	/* are we at the very beginning 
> of a new frame? */
> 	enum {
> 		eAtStrtFram,
> 		eInOctetCnt,
> 		eInMsg
> 	} inputState;		/* our current state */
> 	int iOctetsRemain;	/* Number of Octets remaining 
> in message */
> 	TCPFRAMINGMODE eFraming;
> 	uchar *pMsg;		/* message (fragment) received */
> 	prop_t *fromHost;	/* host name we received 
> messages from */
> 	prop_t *fromHostIP;
> 	void *pUsr;		/* a user-pointer */
> #if 1
>          char *clientPrincipal;    /* client principal */
>          int clientPrincipalLen;
> #endif
> 	rsRetVal (*DoSubmitMessage)(tcps_sess_t*, uchar*, int); 
> /* submit 
> message callback */
> };
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Thu Jan 14 09:37:08 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 00:37:08 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001140036380.15983@asgard.lang.hm>

I am not seeing a tag for 5.3.6 in git. Am I missing something?

David Lang

On Wed, 13 Jan 2010, Rainer Gerhards wrote:

> Hi all,
>
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
>
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> An update for the current master branch will happen soon.
>
> ChangeLog:
> http://www.rsyslog.com/Article435.phtml
>
> Download:
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
>
> I hope this release is useful.
>
> Rainer
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Thu Jan 14 10:35:16 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jan 2010 10:35:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001140036380.15983@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036C6@GRFEXC.intern.adiscon.com>

Oh, thanks - I added the tag, but forgot to push it (looks like the vacation
was too good ;)).

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 14, 2010 9:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> I am not seeing a tag for 5.3.6 in git. Am I missing something?
> 
> David Lang
> 
> On Wed, 13 Jan 2010, Rainer Gerhards wrote:
> 
> > Hi all,
> >
> > I have just released rsyslog 5.3.6, a new v5-beta. Note 
> that this version
> > contains a number of bug fixes, some of them important for 
> some environments.
> > As usual for a beta, it does not contain anything else but 
> fixes. The full
> > lest can be seen in the change log.
> >
> > Please note that it is my intent do replace the current 
> (instable ;))
> > v5-stable by this beta soon. So I would appreciate feedback 
> on 5.3.6 - if I
> > get a few thumbs up, I may be able to accelerate promoting 
> it to stable.
> >
> > An update for the current master branch will happen soon.
> >
> > ChangeLog:
> > http://www.rsyslog.com/Article435.phtml
> >
> > Download:
> > 
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> >
> > I hope this release is useful.
> >
> > Rainer
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Thu Jan 14 14:58:51 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Thu, 14 Jan 2010 07:58:51 -0600
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <20100114135851.GF1895@it.is.rice.edu>

Hi Rainer,

I have been running 5.3.6 with a PostgreSQL 8.4 backend and
it has not exhibited the problems that I saw in 5.3.5 that
caused me to roll back to 4.4.2. Thank you for the fixes to
the PostgreSQL transaction interface. I will be doing some
more testing of the new functionality but it looks good.

Regards,
Ken

On Wed, Jan 13, 2010 at 12:43:11PM +0100, Rainer Gerhards wrote:
> Hi all,
> 
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
> 
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.
> 
> An update for the current master branch will happen soon.
> 
> ChangeLog:
> http://www.rsyslog.com/Article435.phtml
> 
> Download:
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> 
> I hope this release is useful.
> 
> Rainer 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From rgerhards at hq.adiscon.com  Thu Jan 14 16:05:12 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jan 2010 16:05:12 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<20100114135851.GF1895@it.is.rice.edu>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036CC@GRFEXC.intern.adiscon.com>

Hi Ken,

Thanks for the feedback, much appreciated. Please let my know anything more
of interest that you may find out.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
> Kenneth Marshall
> Sent: Thursday, January 14, 2010 2:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Hi Rainer,
> 
> I have been running 5.3.6 with a PostgreSQL 8.4 backend and
> it has not exhibited the problems that I saw in 5.3.5 that
> caused me to roll back to 4.4.2. Thank you for the fixes to
> the PostgreSQL transaction interface. I will be doing some
> more testing of the new functionality but it looks good.
> 
> Regards,
> Ken
> 
> On Wed, Jan 13, 2010 at 12:43:11PM +0100, Rainer Gerhards wrote:
> > Hi all,
> > 
> > I have just released rsyslog 5.3.6, a new v5-beta. Note 
> that this version
> > contains a number of bug fixes, some of them important for 
> some environments.
> > As usual for a beta, it does not contain anything else but 
> fixes. The full
> > lest can be seen in the change log.
> > 
> > Please note that it is my intent do replace the current 
> (instable ;))
> > v5-stable by this beta soon. So I would appreciate feedback 
> on 5.3.6 - if I
> > get a few thumbs up, I may be able to accelerate promoting 
> it to stable.
> > 
> > An update for the current master branch will happen soon.
> > 
> > ChangeLog:
> > http://www.rsyslog.com/Article435.phtml
> > 
> > Download:
> > 
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> > 
> > I hope this release is useful.
> > 
> > Rainer 
> > 
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ryan.b.lynch at gmail.com  Thu Jan 14 16:16:43 2010
From: ryan.b.lynch at gmail.com (Ryan Lynch)
Date: Thu, 14 Jan 2010 10:16:43 -0500
Subject: [rsyslog] MySQL output module: General questions.
Message-ID: <115906d11001140716o3c45a659ndddb7a5851fc7d35@mail.gmail.com>

Hi,

I was hoping that someone with experience using the MySQL output
module, or maybe someone familiar with the source, could help me
understand a few details about the module.

1) Can ommysql use SSL connections to the database server? If not, are
there any future plans to add SSL support?

2) Do failover destinations ('$ActionExecOnlyWhenPreviousIsSuspended',
http://wiki.rsyslog.com/index.php/FailoverSyslogServer) work correctly
with ommysql? If so, how and when do connection failures
register--does the failover happen when the MySQL client fails to
execute an INSERT statement, or when the TCP socket dies, or what?

3) Does ommysql support periodically re-connection to the database server?

4) Is the retry limit for ommysql's INSERT process configurable? The
HOWTO (http://www.rsyslog.com/doc-rsyslog_mysql.html), in the section
'On Reliability...', says "If rsyslogd is unable to store a message,
it performs one retry." I assume this means the retry limit is
hard-coded--is that right?

5) How efficient is ommysql in comparison to omtcp or omrelp? I
imagine there's more overhead for the MySQL protocol, but I don't know
whether are other considerations, too. I would love to hear what
levels of load people experience, with ommysql in production, and what
kind of log volumes they handle.

Thanks!

Ryan B. Lynch
ryan.b.lynch at gmail.com


From mbiebl at gmail.com  Fri Jan 15 07:48:46 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 07:48:46 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>

2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
>
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.

So I gave 5.3.6 a try and stumbled over some rather important regressions.

Compilation and installation went fine and rsyslog started up without
an error message.
I got one log message in the syslog
Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.

But silence afterwards.

When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice

Running rsyslogd -d -c4 I got
pluto:~# rsyslogd -d -c4
rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
rsyslogd: the last error occured in
/etc/rsyslog.d/network-manager.conf, line 2:"~"
rsyslogd: warning: selector line without actions will be discarded
rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
rsyslogd: the last error occured in
/etc/rsyslog.d/network-manager.conf, line 4:"~"
rsyslogd: warning: selector line without actions will be discarded
rsyslogd: the last error occured in /etc/rsyslog.conf, line
46:"$IncludeConfig /etc/rsyslog.d/*.conf"
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

The file in question (which worked fine with 4.4.2) contains
=====
:programname, contains, "NetworkManager" /var/log/NetworkManager.log
~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
~
=====


several issues here:
first of all, the above statements no longer work.
second, rsyslog can't be killed anymore with a single SIGTERM
third, it shouldn't just silently fail. I neither got an error message
on stdout/stderr, nor in the log file.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 07:52:17 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 07:52:17 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
Message-ID: <aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Hi all,
>>
>> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
>> contains a number of bug fixes, some of them important for some environments.
>> As usual for a beta, it does not contain anything else but fixes. The full
>> lest can be seen in the change log.
>>
>> Please note that it is my intent do replace the current (instable ;))
>> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
>> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> So I gave 5.3.6 a try and stumbled over some rather important regressions.
>
> Compilation and installation went fine and rsyslog started up without
> an error message.
> I got one log message in the syslog
> Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.
>
> But silence afterwards.
>
> When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice
>
> Running rsyslogd -d -c4 I got
> pluto:~# rsyslogd -d -c4
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 2:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 4:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 46:"$IncludeConfig /etc/rsyslog.d/*.conf"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> The file in question (which worked fine with 4.4.2) contains
> =====
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> =====
>
>
> several issues here:
> first of all, the above statements no longer work.
> second, rsyslog can't be killed anymore with a single SIGTERM
> third, it shouldn't just silently fail. I neither got an error message
> on stdout/stderr, nor in the log file.

fwiw, changing -c4 to -c5 and removing network-manager.conf didn't help.

rsyslog still logs nothing and rsyslog -d is suspicously silent (i.e.
no output).

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 07:53:07 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 22:53:07 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142252150.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Hi all,
>>
>> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
>> contains a number of bug fixes, some of them important for some environments.
>> As usual for a beta, it does not contain anything else but fixes. The full
>> lest can be seen in the change log.
>>
>> Please note that it is my intent do replace the current (instable ;))
>> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
>> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> So I gave 5.3.6 a try and stumbled over some rather important regressions.
>
> Compilation and installation went fine and rsyslog started up without
> an error message.
> I got one log message in the syslog
> Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.
>
> But silence afterwards.
>
> When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice
>
> Running rsyslogd -d -c4 I got

it may not matter, but I think you need to do -c5 with 5.x

David Lang

> pluto:~# rsyslogd -d -c4
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 2:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 4:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 46:"$IncludeConfig /etc/rsyslog.d/*.conf"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> The file in question (which worked fine with 4.4.2) contains
> =====
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> =====
>
>
> several issues here:
> first of all, the above statements no longer work.
> second, rsyslog can't be killed anymore with a single SIGTERM
> third, it shouldn't just silently fail. I neither got an error message
> on stdout/stderr, nor in the log file.
>
> Cheers,
> Michael
>
>
>


From pgollucci at p6m7g8.com  Fri Jan 15 07:59:45 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Fri, 15 Jan 2010 06:59:45 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
Message-ID: <4B501261.1070901@p6m7g8.com>

Michael Biebl wrote:
>> =====
>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> ~
>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> ~
>> =====

1) rsyslogd always pukes on itself if the config file doesn't parse.
    not new.

2) Its documented that selectors syntax is not a stable API / config 
file syntax, though, the maintainer should have noted it in UPDATING.
I would have hit this tomorrow myself.

http://www.rsyslog.com/doc-rsyslog_conf_filter.html
Expression-Based Filters
Expression based filters allow filtering on arbitrary complex 
expressions, which can include boolean, arithmetic and string 
operations. Expression filters will evolve into a full configuration 
scripting language. Unfortunately, their syntax will slightly change 
during that process. So if you use them now, you need to be prepared to 
change your configuration files some time later. However, we try to 
implement the scripting facility as soon as possible (also in respect to 
stage work needed). So the window of exposure is probably not too long.

-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From mbiebl at gmail.com  Fri Jan 15 08:15:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:15:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <4B501261.1070901@p6m7g8.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
Message-ID: <aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>

2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
> Michael Biebl wrote:
>>> =====
>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>> ~
>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>> ~
>>> =====
>
> 1) rsyslogd always pukes on itself if the config file doesn't parse.
> ? ?not new.
>
> 2) Its documented that selectors syntax is not a stable API / config
> file syntax, though, the maintainer should have noted it in UPDATING.
> I would have hit this tomorrow myself.
>

You forgot the part, where I said that I remove those lines and
rsyslog still doesn't log anything.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 08:19:09 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:19:09 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
Message-ID: <aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>> Michael Biebl wrote:
>>>> =====
>>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>>> ~
>>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>>> ~
>>>> =====
>>
>> 1) rsyslogd always pukes on itself if the config file doesn't parse.
>> ? ?not new.
>>
>> 2) Its documented that selectors syntax is not a stable API / config
>> file syntax, though, the maintainer should have noted it in UPDATING.
>> I would have hit this tomorrow myself.
>>
>
> You forgot the part, where I said that I remove those lines and
> rsyslog still doesn't log anything.

And the fact that if it fails to parse, it should complain loudly and
not fail silently.

Anyway, let's see what Rainer has to say.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:23:51 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:23:51 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>>> Michael Biebl wrote:
>>>>> =====
>>>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>>>> ~
>>>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>>>> ~
>>>>> =====
>>>
>>> 1) rsyslogd always pukes on itself if the config file doesn't parse.
>>> ? ?not new.
>>>
>>> 2) Its documented that selectors syntax is not a stable API / config
>>> file syntax, though, the maintainer should have noted it in UPDATING.
>>> I would have hit this tomorrow myself.
>>>
>>
>> You forgot the part, where I said that I remove those lines and
>> rsyslog still doesn't log anything.
>
> And the fact that if it fails to parse, it should complain loudly and
> not fail silently.

unfortunantly in my experiance it doesn't complain loudly :-(

in V5 there is a new option to tell it to exit if it can't read the 
config.

I suspect that the actual config error is significantly earlier in the 
config. One thing that I frequently tripped over when switching back and 
forth was th HUPisRestart option. In V5 that's not a valid option anymore 
and needs to be removed.

can you post your full config?

David Lang

> Anyway, let's see what Rainer has to say.
>
> Michael
>
>
>

From mbiebl at gmail.com  Fri Jan 15 08:27:16 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:27:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> in V5 there is a new option to tell it to exit if it can't read the config.
>
> I suspect that the actual config error is significantly earlier in the
> config. One thing that I frequently tripped over when switching back and
> forth was th HUPisRestart option. In V5 that's not a valid option anymore
> and needs to be removed.
>
> can you post your full config?

Here is the rsyslog.conf (default Debian install)
http://paste.debian.net/56723/
and the included network-manager.conf file
http://paste.debian.net/56724/

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:34:03 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:34:03 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>> in V5 there is a new option to tell it to exit if it can't read the config.
>>
>> I suspect that the actual config error is significantly earlier in the
>> config. One thing that I frequently tripped over when switching back and
>> forth was th HUPisRestart option. In V5 that's not a valid option anymore
>> and needs to be removed.
>>
>> can you post your full config?
>
> Here is the rsyslog.conf (default Debian install)
> http://paste.debian.net/56723/
> and the included network-manager.conf file
> http://paste.debian.net/56724/

I think I just realized the problem

you have

:programname, contains, "NetworkManager" /var/log/NetworkManager.log
~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
~

when you should have

:programname, contains, "NetworkManager" /var/log/NetworkManager.log
& ~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
& ~

give that a shot.

David Lang


From pgollucci at p6m7g8.com  Fri Jan 15 08:37:41 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Fri, 15 Jan 2010 07:37:41 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
Message-ID: <4B501B45.2080006@p6m7g8.com>

david at lang.hm wrote:
  > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> 
> when you should have
> 
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> & ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> & ~
Sounds reasonable.  I typically do --

# MySQL
:programname, contains, "mysql"  ?by_prog
& :omrelp:cl.tld:2514
& ~

# REST
*.* :omrelp:cl.tld:2514



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From mbiebl at gmail.com  Fri Jan 15 08:42:10 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:42:10 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>

2010/1/15  <david at lang.hm>:

> I think I just realized the problem
>
> you have
>
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
>
> when you should have
>
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> & ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> & ~
>
> give that a shot.

The error message goes away but rsyslog still logs nothing.

Interesting fact is, that the above syntax worked fine with 4.4.2



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:50:19 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:50:19 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>
>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> & ~
>>
>> give that a shot.
>
> The error message goes away but rsyslog still logs nothing.
>
> Interesting fact is, that the above syntax worked fine with 4.4.2

You can wait for Rainer to weigh in, but if you want to test more I would 
start by commenting out everything you can and see if it works, then 
putting more stuff back until it fails.

I have noticed that V5 tends to be a bit more sensitive to invalid lines 
than v4 was, v4 seemed to just ignore what it couldn't understand and 
continue, v5 just goes nuts (very similar to what you re reporting)

you may also try adding '$AboortOnUncleanConfig yes' to the config. I 
found that gave me an error in some cases where it just wouldn't do what I 
expected without it.

David Lang


From mbiebl at gmail.com  Fri Jan 15 08:55:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:55:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>

2010/1/15  <david at lang.hm>:
>
> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
> found that gave me an error in some cases where it just wouldn't do what I
> expected without it.

Oh, the irony :-)

rsyslogd: Option value must be on or off, but is 'yes'
rsyslogd: the last error occured in /etc/rsyslog.conf, line
11:"$AbortOnUncleanConfig yes"
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]


Unfortunately, no further clues. Will try the undocument-everything
approach now.
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:01:49 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:01:49 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>>
>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>> found that gave me an error in some cases where it just wouldn't do what I
>> expected without it.
>
> Oh, the irony :-)
>
> rsyslogd: Option value must be on or off, but is 'yes'
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 11:"$AbortOnUncleanConfig yes"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
>
> Unfortunately, no further clues. Will try the undocument-everything
> approach now.

anything different if you use 'on' instead of 'yes'?

David Lang


From mbiebl at gmail.com  Fri Jan 15 09:07:08 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:07:08 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> 2010/1/15 ?<david at lang.hm>:
>>>
>>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>>> found that gave me an error in some cases where it just wouldn't do what I
>>> expected without it.
>>
>> Oh, the irony :-)
>>
>> rsyslogd: Option value must be on or off, but is 'yes'
>> rsyslogd: the last error occured in /etc/rsyslog.conf, line
>> 11:"$AbortOnUncleanConfig yes"
>> rsyslogd: CONFIG ERROR: could not interpret master config file
>> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>
>>
>> Unfortunately, no further clues. Will try the undocument-everything
>> approach now.
>
> anything different if you use 'on' instead of 'yes'?

Tried that of course. There is no relevant error message.

Further testing revealed:
-d no longer gives me the debug messages on stdout.
I had to run kill -USR1 $(cat /var/run/rsyslogd.pid) to get a verbose output.

With a tiny rsyslog.conf like
$ModLoad imuxsock
*.* /var/log/debug-rsyslog

I finally get log messages again.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 09:08:23 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:08:23 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
Message-ID: <aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/15 ?<david at lang.hm>:
>> On Fri, 15 Jan 2010, Michael Biebl wrote:
>>
>>> 2010/1/15 ?<david at lang.hm>:
>>>>
>>>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>>>> found that gave me an error in some cases where it just wouldn't do what I
>>>> expected without it.
>>>
>>> Oh, the irony :-)
>>>
>>> rsyslogd: Option value must be on or off, but is 'yes'
>>> rsyslogd: the last error occured in /etc/rsyslog.conf, line
>>> 11:"$AbortOnUncleanConfig yes"
>>> rsyslogd: CONFIG ERROR: could not interpret master config file
>>> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>>
>>>
>>> Unfortunately, no further clues. Will try the undocument-everything
>>> approach now.
>>
>> anything different if you use 'on' instead of 'yes'?
>
> Tried that of course. There is no relevant error message.
>
> Further testing revealed:
> -d no longer gives me the debug messages on stdout.
> I had to run kill -USR1 $(cat /var/run/rsyslogd.pid) to get a verbose output.
>
> With a tiny rsyslog.conf like
> $ModLoad imuxsock
> *.* /var/log/debug-rsyslog
>
> I finally get log messages again.

BTW, I'm actually surprised that you don't encounter those problems yourself.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:11:08 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:11:08 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
	<aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> BTW, I'm actually surprised that you don't encounter those problems yourself.

I'm running 5.3.5 still, I haven't had time to build a new version 
(hopefully tomorrow)

David Lang


From mrdemeanour at jackpot.uk.net  Fri Jan 15 09:19:54 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 08:19:54 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <4B50252A.1000106@jackpot.uk.net>

david at lang.hm wrote:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
> 
>> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
>>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>>>> Michael Biebl wrote:
>>>>>> ===== :programname, contains, "NetworkManager"
>>>>>> /var/log/NetworkManager.log ~ :programname, contains,
>>>>>> "wpa_supplicant" /var/log/NetworkManager.log ~ =====
>>>> 
>>>> 1) rsyslogd always pukes on itself if the config file doesn't
>>>> parse. not new.
>>>> 
>>>> 2) Its documented that selectors syntax is not a stable API /
>>>> config file syntax, though, the maintainer should have noted it
>>>> in UPDATING. I would have hit this tomorrow myself.
>>>> 
>>> 
>>> You forgot the part, where I said that I remove those lines and 
>>> rsyslog still doesn't log anything.
>> 
>> And the fact that if it fails to parse, it should complain loudly
>> and not fail silently.
> 
> unfortunantly in my experiance it doesn't complain loudly :-(
> 
> in V5 there is a new option to tell it to exit if it can't read the
> config.

Regarding failure to parse the config:

If you have a config entry of this form:

*.*	-/var/log/syslog	# Send everything else to syslog

(i.e. with a trailing comment appended using hash), it doesn't work (on
4.5.6, at least - I've observed this with other versions, but I don't
have a list). The config line is silently ignored.

The manpage says:
     "Lines  starting with a hash mark ('#') and empty lines are
      ignored."

That's fair enough; but it doesn't mention that lines containing a
trailing comment will also be ignored (silently).

Incorrect config lines should elicit a complaint in rsyslogd's output.

-- 
Jack.


From mbiebl at gmail.com  Fri Jan 15 09:44:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:44:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <4B50252A.1000106@jackpot.uk.net>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
Message-ID: <aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>

Apparently it is that line in my config file, that make rsyslog unhappy:

daemon.*;mail.*;\
       news.err;\
       *.=debug;*.=info;\
       *.=notice;*.=warn       |/dev/xconsole


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:48:35 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:48:35 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> Apparently it is that line in my config file, that make rsyslog unhappy:
>
> daemon.*;mail.*;\
>       news.err;\
>       *.=debug;*.=info;\
>       *.=notice;*.=warn       |/dev/xconsole

my ubuntu laptop doesn't have /dev/xconsole

also, I thought that | was used to execute a program and send the 
logmessage to stdin on that program

David Lang


From mbiebl at gmail.com  Fri Jan 15 09:59:13 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:59:13 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
	<alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001150059rd187152w95901c02dd37b6d9@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> Apparently it is that line in my config file, that make rsyslog unhappy:
>>
>> daemon.*;mail.*;\
>> ? ? ? news.err;\
>> ? ? ? *.=debug;*.=info;\
>> ? ? ? *.=notice;*.=warn ? ? ? |/dev/xconsole
>
> my ubuntu laptop doesn't have /dev/xconsole

That's most likely because of the switch to native upstart jobs. The
old SysV init script had an explicit
mknod -m 640 /dev/xconsole p
line. The new upstart job apparently not anymore. That is arguably a
bug in the upstart job.
(I've CC Michael Vogt, as he is responsible for rsyslog during the lucid cycle)

> also, I thought that | was used to execute a program and send the
> logmessage to stdin on that program

We might argue about the usefulness of this, but it's mostly for
historical reasons. The original syslog.conf had this entry for ages.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:25:38 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:25:38 +0100
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6 (v5-beta)
	released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>

Hi folks,

jumping right in the middle and looking at one issue at the other ;)

Please note that nothing is silently ignored. Whenever rsyslog encounters a
problem, a message is generated. HOWEVER, almost nobody ever looks at the
messages emitted from the syslog facility and so the error messages are
"lost". See also:

http://blog.gerhards.net/2009/11/rsyslog-internal-messages.html

For this, the $AbortOnUnleanConfig directive has been introduced, which will
prevent rsyslog from starting if there is any problem. As the doc for that
directive

http://www.rsyslog.com/doc-rsconf1_abortonuncleanconfig.html

says, enabling it can have harsh consequences. There is a reason that rsyslog
by default does not abort - but rather emit an error message - and continue
to function for that part of the config that is OK. This usually is much
better than aborting.

Please note that this is a long-term issue. For example, see this blog post:

http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.html

Since I have written this post, rsyslog now has a config check action and
also emits error messages (if not disabled) to stderr during startup. I have
to admit I have no further clue how I can make sure people actually look at
the error messages... (it's quite frustrating for me). Any suggestions are
very welcome.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 9:20 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> david at lang.hm wrote:
> > On Fri, 15 Jan 2010, Michael Biebl wrote:
> >
> >> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> >>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
> >>>> Michael Biebl wrote:
> >>>>>> ===== :programname, contains, "NetworkManager"
> >>>>>> /var/log/NetworkManager.log ~ :programname, contains,
> >>>>>> "wpa_supplicant" /var/log/NetworkManager.log ~ =====
> >>>>
> >>>> 1) rsyslogd always pukes on itself if the config file doesn't
> >>>> parse. not new.
> >>>>
> >>>> 2) Its documented that selectors syntax is not a stable API /
> >>>> config file syntax, though, the maintainer should have noted it
> >>>> in UPDATING. I would have hit this tomorrow myself.
> >>>>
> >>>
> >>> You forgot the part, where I said that I remove those lines and
> >>> rsyslog still doesn't log anything.
> >>
> >> And the fact that if it fails to parse, it should complain loudly
> >> and not fail silently.
> >
> > unfortunantly in my experiance it doesn't complain loudly :-(
> >
> > in V5 there is a new option to tell it to exit if it can't read the
> > config.
> 
> Regarding failure to parse the config:
> 
> If you have a config entry of this form:
> 
> *.*	-/var/log/syslog	# Send everything else to syslog
> 
> (i.e. with a trailing comment appended using hash), it doesn't work (on
> 4.5.6, at least - I've observed this with other versions, but I don't
> have a list). The config line is silently ignored.
> 
> The manpage says:
>      "Lines  starting with a hash mark ('#') and empty lines are
>       ignored."
> 
> That's fair enough; but it doesn't mention that lines containing a
> trailing comment will also be ignored (silently).
> 
> Incorrect config lines should elicit a complaint in rsyslogd's output.
> 
> --
> Jack.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 14:28:03 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:28:03 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<alpine.DEB.2.00.1001142252150.4649@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D0@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 15, 2010 7:53 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> > Running rsyslogd -d -c4 I got
> 
> it may not matter, but I think you need to do -c5 with 5.x

No - if you specify -c4, it will start up with the v4 defaults, if you do
-c5, it will start up with the v5 defaults. Nothing else. That's what -c is
for (just the defaults). Note that currently -c4 and -c5, I think, are
aquivalent.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 14:30:14 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:30:14 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D1@GRFEXC.intern.adiscon.com>

> config. One thing that I frequently tripped over when switching back
> and
> forth was th HUPisRestart option. In V5 that's not a valid option
> anymore
> and needs to be removed.

Just FYI: if $HUPisRestart is present in a v5 config, it will generate an
error message, but that's it. No harsh effects (except, of course, if you set
rsyslog to abort on error ;)).




From rgerhards at hq.adiscon.com  Fri Jan 15 14:32:43 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:32:43 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 8:42 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15  <david at lang.hm>:
> 
> > I think I just realized the problem
> >
> > you have
> >
> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> > ~
> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> > ~
> >
> > when you should have
> >
> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> > & ~
> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> > & ~
> >
> > give that a shot.
> 
> The error message goes away but rsyslog still logs nothing.
> 
> Interesting fact is, that the above syntax worked fine with 4.4.2

I don't think so, you probably ignored (did not record?) the error message.
The tilde character is an action, and an action needs to be placed after a
filter. So a tilde character just on its own in a single line is definitely a
syntax error. The engine would not know what to do with such a line.

If it generated no error in v4.4.2, *that* was a bug (will verify later).

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 14:36:54 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:36:54 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D3@GRFEXC.intern.adiscon.com>

ah, that's interesting. The code for pipes (and file output in general) has
been considerably changed, and there was a problem with pipes. I assume that
/dev/xconsole exists? If so, it may fill up and block further processing.

Just to verify, could you try the latest version from the master branch?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 9:45 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Apparently it is that line in my config file, that make rsyslog
> unhappy:
> 
> daemon.*;mail.*;\
>        news.err;\
>        *.=debug;*.=info;\
>        *.=notice;*.=warn       |/dev/xconsole
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 14:39:11 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:39:11 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 8:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15  <david at lang.hm>:
> > in V5 there is a new option to tell it to exit if it can't read the
> config.
> >
> > I suspect that the actual config error is significantly earlier in
> the
> > config. One thing that I frequently tripped over when switching back
> and
> > forth was th HUPisRestart option. In V5 that's not a valid option
> anymore
> > and needs to be removed.
> >
> > can you post your full config?
> 
> Here is the rsyslog.conf (default Debian install)
> http://paste.debian.net/56723/

If I am not mistaken, the default Debian config discards rsyslog error
messages - at least I have not spotted any rule that records syslog.err
messages anywhere...

Rainer


From mbiebl at gmail.com  Fri Jan 15 14:45:20 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 14:45:20 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> Sent: Friday, January 15, 2010 8:42 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> 2010/1/15 ?<david at lang.hm>:
>>
>> > I think I just realized the problem
>> >
>> > you have
>> >
>> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> > ~
>> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> > ~
>> >
>> > when you should have
>> >
>> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> > & ~
>> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> > & ~
>> >
>> > give that a shot.
>>
>> The error message goes away but rsyslog still logs nothing.
>>
>> Interesting fact is, that the above syntax worked fine with 4.4.2
>
> I don't think so, you probably ignored (did not record?) the error message.
> The tilde character is an action, and an action needs to be placed after a
> filter. So a tilde character just on its own in a single line is definitely a
> syntax error. The engine would not know what to do with such a line.
>
> If it generated no error in v4.4.2, *that* was a bug (will verify later).

It definitely worked with 4.4.2, i.e. the
NetworkManager/wpa_supplicant messages were discarded.

Will have to check if rsyslog wrote any error message in the syslog.

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:47:48 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:47:48 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D6@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 2:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> Sent: Friday, January 15, 2010 8:42 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> 2010/1/15 ?<david at lang.hm>:
> >>
> >> > I think I just realized the problem
> >> >
> >> > you have
> >> >
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > ~
> >> > :programname, contains, "wpa_supplicant"
> /var/log/NetworkManager.log
> >> > ~
> >> >
> >> > when you should have
> >> >
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > & ~
> >> > :programname, contains, "wpa_supplicant"
> /var/log/NetworkManager.log
> >> > & ~
> >> >
> >> > give that a shot.
> >>
> >> The error message goes away but rsyslog still logs nothing.
> >>
> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >
> > I don't think so, you probably ignored (did not record?) the error
> message.
> > The tilde character is an action, and an action needs to be placed
> after a
> > filter. So a tilde character just on its own in a single line is
> definitely a
> > syntax error. The engine would not know what to do with such a line.
> >
> > If it generated no error in v4.4.2, *that* was a bug (will verify
> later).
> 
> It definitely worked with 4.4.2, i.e. the
> NetworkManager/wpa_supplicant messages were discarded.
> 
> Will have to check if rsyslog wrote any error message in the syslog.

OK, thanks, will see where the bug in v4 is. I am right now setting up a new
Debian test env, it's probaly easiest to find the issues using the same
platform as you :)

Rainer


From mbiebl at gmail.com  Fri Jan 15 14:43:09 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 14:43:09 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150543i47e9a0b2o7ebdc0e0a47dc22d@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> Here is the rsyslog.conf (default Debian install)
>> http://paste.debian.net/56723/
>
> If I am not mistaken, the default Debian config discards rsyslog error
> messages - at least I have not spotted any rule that records syslog.err
> messages anywhere...

*.*;auth,authpriv.none		-/var/log/syslog

should catch syslog errors, right?


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:52:56 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:52:56 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150543i47e9a0b2o7ebdc0e0a47dc22d@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D7@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 2:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>
> >> Here is the rsyslog.conf (default Debian install)
> >> http://paste.debian.net/56723/
> >
> > If I am not mistaken, the default Debian config discards rsyslog
> error
> > messages - at least I have not spotted any rule that records
> syslog.err
> > messages anywhere...
> 
> *.*;auth,authpriv.none		-/var/log/syslog
> 
> should catch syslog errors, right?

Oops, I overlooked "*.*". And, indeed, it should catch them.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 15:23:34 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 15:23:34 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>

Michael,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > ~
> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >
> > I don't think so, you probably ignored (did not record?) the error
> message.
> > The tilde character is an action, and an action needs to be placed
> after a
> > filter. So a tilde character just on its own in a single line is
> definitely a
> > syntax error. The engine would not know what to do with such a line.
> >
> > If it generated no error in v4.4.2, *that* was a bug (will verify
> later).
> 
> It definitely worked with 4.4.2, i.e. the
> NetworkManager/wpa_supplicant messages were discarded.

I used a Debian 5 I had available here, ran apt-get update/upgrade and
compiled rsyslog 4.4.2 from scratch. Then I entered the first line into the
config and restarted rsyslog.

After doing so, I had the relevant errors in /var/log/syslog.

Two observations: 

a) the commands were flagged as invalid by 4.4.2
b) error messages are logged (at least up to 4.4.2)

Note that I had the statements directly in my main config. Can you verify you
get the error messages, too, when you have them directly in the main config?

I'll now see if v5 does not emit the messages...

Rainer




From rgerhards at hq.adiscon.com  Fri Jan 15 15:33:10 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 15:33:10 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D9@GRFEXC.intern.adiscon.com>

Michael, 

I could reproduce the original bug report, now a bugzilla entry:

http://bugzilla.adiscon.com/show_bug.cgi?id=169

I guess you don't see any entries in /var/log/syslog simply because rsyslog
hangs and so is unable to process any further message. I suggest you
subscribe to the bug in bugzilla.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 3:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Michael,
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > >> > :programname, contains, "NetworkManager"
> > /var/log/NetworkManager.log
> > >> > ~
> > >> Interesting fact is, that the above syntax worked fine with 4.4.2
> > >
> > > I don't think so, you probably ignored (did not record?) the error
> > message.
> > > The tilde character is an action, and an action needs to be placed
> > after a
> > > filter. So a tilde character just on its own in a single line is
> > definitely a
> > > syntax error. The engine would not know what to do with such a
> line.
> > >
> > > If it generated no error in v4.4.2, *that* was a bug (will verify
> > later).
> >
> > It definitely worked with 4.4.2, i.e. the
> > NetworkManager/wpa_supplicant messages were discarded.
> 
> I used a Debian 5 I had available here, ran apt-get update/upgrade and
> compiled rsyslog 4.4.2 from scratch. Then I entered the first line into
> the
> config and restarted rsyslog.
> 
> After doing so, I had the relevant errors in /var/log/syslog.
> 
> Two observations:
> 
> a) the commands were flagged as invalid by 4.4.2
> b) error messages are logged (at least up to 4.4.2)
> 
> Note that I had the statements directly in my main config. Can you
> verify you
> get the error messages, too, when you have them directly in the main
> config?
> 
> I'll now see if v5 does not emit the messages...
> 
> Rainer
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Fri Jan 15 16:02:30 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 16:02:30 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> >> > :programname, contains, "NetworkManager"
>> /var/log/NetworkManager.log
>> >> > ~
>> >> Interesting fact is, that the above syntax worked fine with 4.4.2
>> >
>> > I don't think so, you probably ignored (did not record?) the error
>> message.
>> > The tilde character is an action, and an action needs to be placed
>> after a
>> > filter. So a tilde character just on its own in a single line is
>> definitely a
>> > syntax error. The engine would not know what to do with such a line.
>> >
>> > If it generated no error in v4.4.2, *that* was a bug (will verify
>> later).
>>
>> It definitely worked with 4.4.2, i.e. the
>> NetworkManager/wpa_supplicant messages were discarded.
>
> I used a Debian 5 I had available here, ran apt-get update/upgrade and
> compiled rsyslog 4.4.2 from scratch. Then I entered the first line into the
> config and restarted rsyslog.
>
> After doing so, I had the relevant errors in /var/log/syslog.
>
> Two observations:
>
> a) the commands were flagged as invalid by 4.4.2
> b) error messages are logged (at least up to 4.4.2)

Yeah, false alarm from my side, sorry.

4.4.2 writes an error message about using incorrect syntax and the log
messages are not dropped when using a simple "~". Everything as it
should be :-)
So this was all a red herring.

The real problem, as you already noticed, the non-working pipe which
causes 5.3.6 to hang and not process any further message.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 16:37:15 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 16:37:15 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>

> Yeah, false alarm from my side, sorry.

No problem - much better a false alarm here and there than no alarm at all.
Thankfully, we could avoid propagating the pipe error into v4-stable, which I
consider very useful :)

Rainer

> 
> 4.4.2 writes an error message about using incorrect syntax and the log
> messages are not dropped when using a simple "~". Everything as it
> should be :-)
> So this was all a red herring.
> 
> The real problem, as you already noticed, the non-working pipe which
> causes 5.3.6 to hang and not process any further message.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Fri Jan 15 16:43:14 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 16:43:14 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150743p70182910t144d10a068c8358b@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Yeah, false alarm from my side, sorry.
>
> No problem - much better a false alarm here and there than no alarm at all.
> Thankfully, we could avoid propagating the pipe error into v4-stable, which I
> consider very useful :)

BTW, can you reproduce the problem, that -d no longer produces a
verbose output with 5.3.6?

Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 16:45:02 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 16:45:02 +0100
Subject: [rsyslog] -d doesn't work - RE:  rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150743p70182910t144d10a068c8358b@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DB@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 4:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> Yeah, false alarm from my side, sorry.
> >
> > No problem - much better a false alarm here and there than no alarm
> at all.
> > Thankfully, we could avoid propagating the pipe error into v4-stable,
> which I
> > consider very useful :)
> 
> BTW, can you reproduce the problem, that -d no longer produces a
> verbose output with 5.3.6?

Will look into that after the fix. I remember I had this issue and I think it
is fixed. Maybe I forgot to merge some change into v5-beta. This most
probably is a result of the improved runtime debugging support. Sorry I
forgot to comment on this one.

Rainer


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:04:58 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:04:58 +0000
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
Message-ID: <4B50922A.8090900@jackpot.uk.net>

Rainer Gerhards wrote:
> Hi folks,
> 
> jumping right in the middle and looking at one issue at the other ;)
> 
> Please note that nothing is silently ignored. Whenever rsyslog encounters a
> problem, a message is generated. HOWEVER, almost nobody ever looks at the
> messages emitted from the syslog facility and so the error messages are
> "lost". See also:

Rainer,

Consider please this single action line from a simple config:
   *.*                             /var/log/syslog

If that is modified as follows:
   *.*                             /var/log/syslog   # Comment goes here

then (1) no message goes to stdout; (2) nothing gets logged to
/var/log/syslog, because the action line specifying that action is
faulty. The service starts; but given that the defective action line is
the only one in the config, it might as well have failed to start,
because no log output will ever be produced. In particular, messages for 
the syslog facility will not be sent anywhere.

I call that "silent"; as far as I can see, there is absolutely no
message anywhere indicating that the service had any problems with the
config.

# rsyslogd -v
rsyslogd 4.5.6, compiled with:
         FEATURE_REGEXP:                         Yes
         FEATURE_LARGEFILE:                      Yes
         FEATURE_NETZIP (message compression):   Yes
         GSSAPI Kerberos 5 support:              No
         FEATURE_DEBUG (debug build, slow code): No
         Atomic operations supported:            Yes
         Runtime Instrumentation (slow code):    No

I think config parsing problems should be output unconditionally to
stdout; but what do I know :-) Anyway, relying on the logging service to
tell you about a problem with the logging service seems - umm -
over-confident.
-- 
Jack.



From rgerhards at hq.adiscon.com  Fri Jan 15 17:08:36 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:08:36 +0100
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
	(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
	<4B50922A.8090900@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 5:05 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
> (v5-beta) released
> 
> Rainer Gerhards wrote:
> > Hi folks,
> >
> > jumping right in the middle and looking at one issue at the other ;)
> >
> > Please note that nothing is silently ignored. Whenever rsyslog
> encounters a
> > problem, a message is generated. HOWEVER, almost nobody ever looks at
> the
> > messages emitted from the syslog facility and so the error messages
> are
> > "lost". See also:
> 
> Rainer,
> 
> Consider please this single action line from a simple config:
>    *.*                             /var/log/syslog
> 
> If that is modified as follows:
>    *.*                             /var/log/syslog   # Comment goes
> here
> 
> then (1) no message goes to stdout; (2) nothing gets logged to
> /var/log/syslog, because the action line specifying that action is
> faulty. The service starts; but given that the defective action line is
> the only one in the config, it might as well have failed to start,
> because no log output will ever be produced. In particular, messages
> for
> the syslog facility will not be sent anywhere.
> 
> I call that "silent"; as far as I can see, there is absolutely no
> message anywhere indicating that the service had any problems with the
> config.

That's a problem with the current config syntax. Interestingly hard to fix.

> 
> # rsyslogd -v
> rsyslogd 4.5.6, compiled with:
>          FEATURE_REGEXP:                         Yes
>          FEATURE_LARGEFILE:                      Yes
>          FEATURE_NETZIP (message compression):   Yes
>          GSSAPI Kerberos 5 support:              No
>          FEATURE_DEBUG (debug build, slow code): No
>          Atomic operations supported:            Yes
>          Runtime Instrumentation (slow code):    No
> 
> I think config parsing problems should be output unconditionally to
> stdout; but what do I know :-) Anyway, relying on the logging service
> to
> tell you about a problem with the logging service seems - umm -
> over-confident.

Well, that's the meat of it. So what shall I do? I am asking this question
for roughly 20 months now, and so far obviously did not get a good answer,
nor do I have one. As I wrote, we can already output error messages to
stderr. Would it really help to add another option to send them to stdout as
well? 

All suggestions on how to handle error notifications are *very* welcome.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 17:18:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:18:24 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>

Michael,

Fix now in git, links at the bug tracker:

http://bugzilla.adiscon.com/show_bug.cgi?id=169

Please let me know if it works for you (the patch is a bit trickier than it
looks, so confirmations would be good).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 4:03 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> >> > :programname, contains, "NetworkManager"
> >> /var/log/NetworkManager.log
> >> >> > ~
> >> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >> >
> >> > I don't think so, you probably ignored (did not record?) the error
> >> message.
> >> > The tilde character is an action, and an action needs to be placed
> >> after a
> >> > filter. So a tilde character just on its own in a single line is
> >> definitely a
> >> > syntax error. The engine would not know what to do with such a
> line.
> >> >
> >> > If it generated no error in v4.4.2, *that* was a bug (will verify
> >> later).
> >>
> >> It definitely worked with 4.4.2, i.e. the
> >> NetworkManager/wpa_supplicant messages were discarded.
> >
> > I used a Debian 5 I had available here, ran apt-get update/upgrade
> and
> > compiled rsyslog 4.4.2 from scratch. Then I entered the first line
> into the
> > config and restarted rsyslog.
> >
> > After doing so, I had the relevant errors in /var/log/syslog.
> >
> > Two observations:
> >
> > a) the commands were flagged as invalid by 4.4.2
> > b) error messages are logged (at least up to 4.4.2)
> 
> Yeah, false alarm from my side, sorry.
> 
> 4.4.2 writes an error message about using incorrect syntax and the log
> messages are not dropped when using a simple "~". Everything as it
> should be :-)
> So this was all a red herring.
> 
> The real problem, as you already noticed, the non-working pipe which
> causes 5.3.6 to hang and not process any further message.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:23:10 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:23:10 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
	<4B50922A.8090900@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>

 -----Original Message-----
> If that is modified as follows:
>    *.*                             /var/log/syslog   # Comment goes
> here
> 

It's even worse: data is written to the 
file "/var/log/syslog   # Comment goes here"!

Looks like I need to find a solution at least for omfile. Thanks for bringing
this issue up in this context ;)

Rainer


> then (1) no message goes to stdout; (2) nothing gets logged to
> /var/log/syslog, because the action line specifying that action is
> faulty. The service starts; but given that the defective action line is
> the only one in the config, it might as well have failed to start,
> because no log output will ever be produced. In particular, messages
> for
> the syslog facility will not be sent anywhere.
> 
> I call that "silent"; as far as I can see, there is absolutely no
> message anywhere indicating that the service had any problems with the
> config.
> 
> # rsyslogd -v
> rsyslogd 4.5.6, compiled with:
>          FEATURE_REGEXP:                         Yes
>          FEATURE_LARGEFILE:                      Yes
>          FEATURE_NETZIP (message compression):   Yes
>          GSSAPI Kerberos 5 support:              No
>          FEATURE_DEBUG (debug build, slow code): No
>          Atomic operations supported:            Yes
>          Runtime Instrumentation (slow code):    No
> 
> I think config parsing problems should be output unconditionally to
> stdout; but what do I know :-) Anyway, relying on the logging service
> to
> tell you about a problem with the logging service seems - umm -
> over-confident.
> --
> Jack.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:27:00 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:27:00 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com><4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DF@GRFEXC.intern.adiscon.com>

bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=170

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 5:23 PM
> To: rsyslog-users
> Subject: [rsyslog] comments in file actions
> 
>  -----Original Message-----
> > If that is modified as follows:
> >    *.*                             /var/log/syslog   # Comment goes
> > here
> >
> 
> It's even worse: data is written to the
> file "/var/log/syslog   # Comment goes here"!
> 
> Looks like I need to find a solution at least for omfile. Thanks for
> bringing
> this issue up in this context ;)
> 
> Rainer
> 
> 
> > then (1) no message goes to stdout; (2) nothing gets logged to
> > /var/log/syslog, because the action line specifying that action is
> > faulty. The service starts; but given that the defective action line
> is
> > the only one in the config, it might as well have failed to start,
> > because no log output will ever be produced. In particular, messages
> > for
> > the syslog facility will not be sent anywhere.
> >
> > I call that "silent"; as far as I can see, there is absolutely no
> > message anywhere indicating that the service had any problems with
> the
> > config.
> >
> > # rsyslogd -v
> > rsyslogd 4.5.6, compiled with:
> >          FEATURE_REGEXP:                         Yes
> >          FEATURE_LARGEFILE:                      Yes
> >          FEATURE_NETZIP (message compression):   Yes
> >          GSSAPI Kerberos 5 support:              No
> >          FEATURE_DEBUG (debug build, slow code): No
> >          Atomic operations supported:            Yes
> >          Runtime Instrumentation (slow code):    No
> >
> > I think config parsing problems should be output unconditionally to
> > stdout; but what do I know :-) Anyway, relying on the logging service
> > to
> > tell you about a problem with the logging service seems - umm -
> > over-confident.
> > --
> > Jack.
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:36:28 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:36:28 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com><4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DF@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E0@GRFEXC.intern.adiscon.com>

mhh... general question: would anybody object if I would not permit spaces
inside file names? (one could introduce them by using dynafiles with a clever
template if absolutely needed...).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 5:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=170
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Friday, January 15, 2010 5:23 PM
> > To: rsyslog-users
> > Subject: [rsyslog] comments in file actions
> >
> >  -----Original Message-----
> > > If that is modified as follows:
> > >    *.*                             /var/log/syslog   # Comment goes
> > > here
> > >
> >
> > It's even worse: data is written to the
> > file "/var/log/syslog   # Comment goes here"!
> >
> > Looks like I need to find a solution at least for omfile. Thanks for
> > bringing
> > this issue up in this context ;)
> >
> > Rainer
> >
> >
> > > then (1) no message goes to stdout; (2) nothing gets logged to
> > > /var/log/syslog, because the action line specifying that action is
> > > faulty. The service starts; but given that the defective action
> line
> > is
> > > the only one in the config, it might as well have failed to start,
> > > because no log output will ever be produced. In particular,
> messages
> > > for
> > > the syslog facility will not be sent anywhere.
> > >
> > > I call that "silent"; as far as I can see, there is absolutely no
> > > message anywhere indicating that the service had any problems with
> > the
> > > config.
> > >
> > > # rsyslogd -v
> > > rsyslogd 4.5.6, compiled with:
> > >          FEATURE_REGEXP:                         Yes
> > >          FEATURE_LARGEFILE:                      Yes
> > >          FEATURE_NETZIP (message compression):   Yes
> > >          GSSAPI Kerberos 5 support:              No
> > >          FEATURE_DEBUG (debug build, slow code): No
> > >          Atomic operations supported:            Yes
> > >          Runtime Instrumentation (slow code):    No
> > >
> > > I think config parsing problems should be output unconditionally to
> > > stdout; but what do I know :-) Anyway, relying on the logging
> service
> > > to
> > > tell you about a problem with the logging service seems - umm -
> > > over-confident.
> > > --
> > > Jack.
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:54:50 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:54:50 +0000
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog
 5.3.6	(v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>
Message-ID: <4B509DDA.9070207@jackpot.uk.net>

Rainer Gerhards wrote:
>> 
>> I think config parsing problems should be output unconditionally to
>>  stdout; but what do I know :-) Anyway, relying on the logging
>> service to tell you about a problem with the logging service seems
>> - umm - over-confident.
> 
> Well, that's the meat of it. So what shall I do? I am asking this
> question for roughly 20 months now, and so far obviously did not get
> a good answer, nor do I have one. As I wrote, we can already output
> error messages to stderr. Would it really help to add another option
> to send them to stdout as well?

It outputs to stderr? I don't seem to be able to make it do that (as far
as I know, both stderr and stdout should be going to the console).

With the config file containing the invalid action line, I tried this:

# rsyslogd -c4 -N1
rsyslogd: version 4.5.6, config validation run (level 1), master config
/etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

Shouldn't it at least THEN have told me there was something wrong with
the conf? I would expect anything that tells me it's a "config
validation run" to output any errors in the config to the same channel
that message gets printed on.

> 
> All suggestions on how to handle error notifications are *very*
> welcome.
> 

Output to stderr would be fine with me; but I'm not convinced it does that.


-- 
Jack.


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:58:40 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:58:40 +0000
Subject: [rsyslog] comments in file actions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
Message-ID: <4B509EC0.3050504@jackpot.uk.net>

Rainer Gerhards wrote:
> -----Original Message-----
>> If that is modified as follows: *.*
>> /var/log/syslog   # Comment goes here
>> 
> 
> It's even worse: data is written to the file "/var/log/syslog   #
> Comment goes here"!

Aaaaahhh - that finally explains some odd files lurking in /var/log!

That also explains why there's no error message on stdout/stdderr -
there's no error, as far as rsyslog is concerned. [/me feels stupid]

I would really like to be able to put comments on the ends of config
lines, as I can with many other packages. But now I know what's going on
here. Thanks!

-- 
Jack.


From mbiebl at gmail.com  Fri Jan 15 23:57:08 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 23:57:08 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> Fix now in git, links at the bug tracker:
>
> http://bugzilla.adiscon.com/show_bug.cgi?id=169
>
> Please let me know if it works for you (the patch is a bit trickier than it
> looks, so confirmations would be good).

I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
But now I'm getting a crash when rsyslog encounters the xconsole pipe
config.

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From sanelson at gmail.com  Sun Jan 17 00:46:59 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sat, 16 Jan 2010 23:46:59 +0000
Subject: [rsyslog] Difference between versions
Message-ID: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>

Hi there,

I've spent a long time picking through changelogs, but I'm afraid I
don't have a clear understanding of what to choose between versions of
rsyslog.

My platform is RHEL 5 - the distro ships with 2.0 which seems to be
both ancient and deprecated.

If I have to build a new package, it would be good to understand which
version I should choose.  For example. Rawhide has a 4.x package,
which would be the obvious starting point.

My overall objective is to be able to aggregate syslog and also Drupal
watchdog logs in a central location, and index them with Solr, to
produce a data mart/wharehouse.

I look forward to enlightenment!

TIA,

S,

-- 
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com


From david at lang.hm  Sun Jan 17 01:19:01 2010
From: david at lang.hm (david at lang.hm)
Date: Sat, 16 Jan 2010 16:19:01 -0800 (PST)
Subject: [rsyslog] Difference between versions
In-Reply-To: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>

On Sat, 16 Jan 2010, Stephen Nelson-Smith wrote:

> Hi there,
>
> I've spent a long time picking through changelogs, but I'm afraid I
> don't have a clear understanding of what to choose between versions of
> rsyslog.

2 is ancient, it's only in RHEL because that is the version that was out 
when RHEL5 was released and they never upgrade software (by policy)

3 was the stable about a year ago. This is in Debian 5

4 is after a bunch of rapid developement, it's starting to appear in some 
distros

5 is the current version, it is _much_ faster than previous version.

unfortunantly the current 5.2 'stable' release is known to be very buggy. 
5.3.6 was released a week ago, and it is believed to be the best version. 
several of us are testing it (I put in it production on a couple dozen 
machines in friday, so I should find anything that affects my environment 
by monday). The expectation is that this will replace the broken 5.2 very 
shortly.

so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you 
find anything that doesn't work, post here and you will probably get a fix 
quickly (note that the main developer is in germany, so you do have the 
time zone lag to deal with)

David Lang

> My platform is RHEL 5 - the distro ships with 2.0 which seems to be
> both ancient and deprecated.
>
> If I have to build a new package, it would be good to understand which
> version I should choose.  For example. Rawhide has a 4.x package,
> which would be the obvious starting point.
>
> My overall objective is to be able to aggregate syslog and also Drupal
> watchdog logs in a central location, and index them with Solr, to
> produce a data mart/wharehouse.
>
> I look forward to enlightenment!
>
> TIA,
>
> S,
>
>


From sanelson at gmail.com  Sun Jan 17 09:56:51 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sun, 17 Jan 2010 08:56:51 +0000
Subject: [rsyslog] Difference between versions
In-Reply-To: <alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
Message-ID: <b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>

Hi there,

> 4 is after a bunch of rapid developement, it's starting to appear in some
> distros

So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?

> 5 is the current version, it is _much_ faster than previous version.

Fast is good!

> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you
> find anything that doesn't work, post here and you will probably get a fix
> quickly (note that the main developer is in germany, so you do have the
> time zone lag to deal with)

Right - I'll get to work on a spec file.  Anyone got any gotchas to share?

S.


From david at lang.hm  Sun Jan 17 10:07:37 2010
From: david at lang.hm (david at lang.hm)
Date: Sun, 17 Jan 2010 01:07:37 -0800 (PST)
Subject: [rsyslog] Difference between versions
In-Reply-To: <b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
	<b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>

On Sun, 17 Jan 2010, Stephen Nelson-Smith wrote:

> Hi there,
>
>> 4 is after a bunch of rapid developement, it's starting to appear in some
>> distros
>
> So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?

4 will go into stable mode like 3 is now. 4 and 5 were under development 
at the same time, but the changes for 5 were so drastic that Rainer didn't 
feel comfortable doing them in the normal development version. 4 settled 
down a few months ago, it looks like 5 is settling down now. I think we 
have already hit one bug that may not end up getting fixed in 4 as the fix 
would be too invasive (when Rainer declares a version stable he is _very_ 
careful about changes to it, even if that means leaving something broken 
to avoid a substantial risk of breaking other things)

people are finding more bugs in rsyslog in recent months, most of the bugs 
that they have been finding are not new bugs, but are instead the result 
of more people useing rsyslog in more different ways (the fact that most 
distros have switched to rsyslog for their next release, if not their last 
release, has drasticly increased it's use)

>> 5 is the current version, it is _much_ faster than previous version.
>
> Fast is good!
>
>> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you
>> find anything that doesn't work, post here and you will probably get a fix
>> quickly (note that the main developer is in germany, so you do have the
>> time zone lag to deal with)
>
> Right - I'll get to work on a spec file.  Anyone got any gotchas to share?

the big thing is that it is sensitive to config file errors, make sure 
that your startup script doesn't hide such errors from the user.

David Lang


From rgerhards at hq.adiscon.com  Sun Jan 17 11:43:18 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:43:18 +0100
Subject: [rsyslog] FW:  RHEL5 rsyslog 4 rpms
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E2@GRFEXC.intern.adiscon.com>

Hi Stephen,

this message (below) may be useful for you. Maybe you can join forces...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Daniel Anson
> Sent: Thursday, January 07, 2010 7:44 PM
> To: rsyslog-users
> Subject: [rsyslog] RHEL5 rsyslog 4 rpms
> 
> If anyone is interested, an RPM engineer I know has packaged RHEL5
> rsyslog4 rpms.  These are available for public download and testing @
> http://dl.iuscommunity.org/pub/ius  Any comments can be emailed
> directly to him at ius-coredev at lists.launchpad.net
> 
> rpms are regularly packaged by him so let him know what you think.  I
> believe you just have to add the yum repo.
> 
> --Daniel M. Anson
> --Linux Systems Engineer
> 
> 
> 
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use
> of the
> individual or entity to which this message is addressed, and unless
> otherwise
> expressly indicated, is confidential and privileged information of
> Rackspace.
> Any dissemination, distribution or copying of the enclosed material is
> prohibited.
> If you receive this transmission in error, please notify us immediately
> by e-mail
> at abuse at rackspace.com, and delete the original message.
> Your cooperation is appreciated.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Sun Jan 17 11:43:36 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:43:36 +0100
Subject: [rsyslog] Difference between versions
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E3@GRFEXC.intern.adiscon.com>

David,

thanks, that's an excellent summary :)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Sunday, January 17, 2010 1:19 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Difference between versions
> 
> On Sat, 16 Jan 2010, Stephen Nelson-Smith wrote:
> 
> > Hi there,
> >
> > I've spent a long time picking through changelogs, but I'm afraid I
> > don't have a clear understanding of what to choose between versions
> of
> > rsyslog.
> 
> 2 is ancient, it's only in RHEL because that is the version that was
> out
> when RHEL5 was released and they never upgrade software (by policy)
> 
> 3 was the stable about a year ago. This is in Debian 5
> 
> 4 is after a bunch of rapid developement, it's starting to appear in
> some
> distros
> 
> 5 is the current version, it is _much_ faster than previous version.
> 
> unfortunantly the current 5.2 'stable' release is known to be very
> buggy.
> 5.3.6 was released a week ago, and it is believed to be the best
> version.
> several of us are testing it (I put in it production on a couple dozen
> machines in friday, so I should find anything that affects my
> environment
> by monday). The expectation is that this will replace the broken 5.2
> very
> shortly.
> 
> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if
> you
> find anything that doesn't work, post here and you will probably get a
> fix
> quickly (note that the main developer is in germany, so you do have the
> time zone lag to deal with)
> 
> David Lang
> 
> > My platform is RHEL 5 - the distro ships with 2.0 which seems to be
> > both ancient and deprecated.
> >
> > If I have to build a new package, it would be good to understand
> which
> > version I should choose.  For example. Rawhide has a 4.x package,
> > which would be the obvious starting point.
> >
> > My overall objective is to be able to aggregate syslog and also
> Drupal
> > watchdog logs in a central location, and index them with Solr, to
> > produce a data mart/wharehouse.
> >
> > I look forward to enlightenment!
> >
> > TIA,
> >
> > S,
> >
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Sun Jan 17 11:47:25 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:47:25 +0100
Subject: [rsyslog] Difference between versions
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com><alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm><b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
	<alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Sunday, January 17, 2010 10:08 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Difference between versions
> 
> On Sun, 17 Jan 2010, Stephen Nelson-Smith wrote:
> 
> > Hi there,
> >
> >> 4 is after a bunch of rapid developement, it's starting to appear in
> some
> >> distros
> >
> > So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?
> 
> 4 will go into stable mode like 3 is now. 4 and 5 were under
> development
> at the same time, but the changes for 5 were so drastic that Rainer
> didn't
> feel comfortable doing them in the normal development version. 4
> settled
> down a few months ago, it looks like 5 is settling down now. I think we
> have already hit one bug that may not end up getting fixed in 4 as the
> fix
> would be too invasive (when Rainer declares a version stable he is
> _very_
> careful about changes to it, even if that means leaving something
> broken
> to avoid a substantial risk of breaking other things)

Let me elaborate a bit on the v4 bug. There are some situations in the v4
queue engine, that will lead to an unclean shutdown, maybe even a hang
condition (based on the configuration). To fix this, I would need to rewrite
the v4 queue engine very much in the same way as the v5 engine is (minus some
things, but it is a *very* substantial change). Rather than spending time on
that, I accept this issue as it is, and recommend to move to v5 for those few
that are affected. Thankfully, with 5.3.6 we will have a real stable v5 soon.
Note that the v4 bug is *very unlikely* to show up - you need many queues,
various queing params (I don't know all of them out of my head) and it will
happen only very occasionally.

Rainer


From rgerhards at hq.adiscon.com  Sun Jan 17 11:51:16 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:51:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 11:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > Fix now in git, links at the bug tracker:
> >
> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >
> > Please let me know if it works for you (the patch is a bit trickier
> than it
> > looks, so confirmations would be good).
> 
> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> But now I'm getting a crash when rsyslog encounters the xconsole pipe
> config.

I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
stock Debian config?

Rainer

> 
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From sanelson at gmail.com  Sun Jan 17 12:26:42 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sun, 17 Jan 2010 11:26:42 +0000
Subject: [rsyslog] Difference between versions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
	<b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
	<alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>
Message-ID: <b6131fdc1001170326g64142ed9ta2395883695f45d4@mail.gmail.com>

Hi,
> Thankfully, with 5.3.6 we will have a real stable v5 soon.

OK - you'll have to excuse my weak git fu:

git clone git://git.adiscon.com/git/rsyslog.git
git checkout -b v5.3.6rpm v5.3.6

Is this the place to start from?  I also want to make sure I can pull
in any patches that emerge while I'm working on the package.  Do I do
that with rebase?  But will that rebase from origin/master?  Do I need
to have checked out 5.3.6-devel first?

S.


From mbiebl at gmail.com  Sun Jan 17 12:48:59 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 17 Jan 2010 12:48:59 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>

2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> Sent: Friday, January 15, 2010 11:57 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> > Michael,
>> >
>> > Fix now in git, links at the bug tracker:
>> >
>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
>> >
>> > Please let me know if it works for you (the patch is a bit trickier
>> than it
>> > looks, so confirmations would be good).
>>
>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
>> But now I'm getting a crash when rsyslog encounters the xconsole pipe
>> config.
>
> I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
> stock Debian config?

Yes. As said, I just downloaded the 5.3.6 tarball applied the
5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
got the crash. I use the default rsyslog.conf from the official debian
package.
I attached a backtrace. Hope that helps

(gdb) run -c4 -d
Starting program: /usr/sbin/rsyslogd -c4 -d
[Thread debugging using libthread_db enabled]
[New Thread 0xb7df2b70 (LWP 5162)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7df2b70 (LWP 5162)]
qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
pUsr=0x80b72c0) at queue.c:2256
2256            if(pThis->qType != QUEUETYPE_DIRECT) {
(gdb) bt full
#0  qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
pUsr=0x80b72c0) at queue.c:2256
        iRet = <value optimized out>
        iCancelStateSave = <value optimized out>
#1  0x0807cd5b in actionWriteToAction (pAction=0x80ac8d8) at
../action.c:1169
        pMsgSave = 0x0
        iRet = <value optimized out>
#2  0x0807d4fe in doActionCallAction (pAction=0x80ac8d8,
pMsg=0x80b72c0) at ../action.c:1244
No locals.
#3  actionCallAction (pAction=0x80ac8d8, pMsg=0x80b72c0) at ../action.c:1274
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113864, 0, -1210114056, 1464119912, -422385385},
              __mask_was_saved = 0}}, __pad = {0xb7df2240, 0x0,
0xb7df20d0, 0xb7feb27f}}
        __cancel_arg = 0x80ad0a0
        not_first_call = <value optimized out>
        iRet = -1210113864
#4  0x080794d7 in processMsgDoActions (pData=0xfffff815,
pParam=0xb7df20b8) at rule.c:113
        iRet = <value optimized out>
        iRetMod = <value optimized out>
#5  0x080627ba in llExecFunc (pThis=0x80ac9a0, pFunc=0x8079480
<processMsgDoActions>, pParam=0xb7df20b8) at linkedlist.c:391
        iRet = <value optimized out>
        iRetLL = <value optimized out>
        pData = 0x80ac8d8
        llCookie = 0x80ac508
        llCookiePrev = 0x0
#6  0x08079007 in processMsg (pThis=0x80ac968, pMsg=0x80b72c0) at rule.c:299
        bProcessMsg = 1
        DoActData = {bPrevWasSuspended = 0, pMsg = 0x80b72c0}
        iRet = RS_RET_OK
#7  0x080781ba in processMsgDoRules (pData=0x80ac968,
pParam=0x80b72c0) at ruleset.c:145
        iRet = <value optimized out>
#8  0x080627ba in llExecFunc (pThis=0x809eb68, pFunc=0x8078190
<processMsgDoRules>, pParam=0x80b72c0) at linkedlist.c:391
        iRet = <value optimized out>
        iRetLL = <value optimized out>
        pData = 0x80ac968
        llCookie = 0x80ac478
        llCookiePrev = 0x80ac4f8
#9  0x0807876d in processMsg (pMsg=0x80b72c0) at ruleset.c:164
        pThis = <value optimized out>
        iRet = <value optimized out>
#10 0x080506ab in msgConsumer (notNeeded=0x0, pBatch=0x809ecc8,
pbShutdownImmediate=0x80ad348) at syslogd.c:614
        i = 0
        pMsg = 0x80b72c0
        localRet = RS_RET_IO_ERROR
#11 0x08077ca4 in ConsumerReg (pThis=0x80ad338, pWti=0x809ecb0) at queue.c:1638
        iCancelStateSave = 1
        iRet = <value optimized out>
#12 0x08070526 in wtiWorker (pThis=0x809ecb0) at wti.c:286
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113200, 0, -1210113384, 1463800424, -411898089},
              __mask_was_saved = 0}}, __pad = {0xb7df2350, 0x0, 0x0, 0x809002c}}
        not_first_call = <value optimized out>
        pWtp = 0x809ebd8
        bInactivityTOOccured = <value optimized out>
        localRet = <value optimized out>
        terminateRet = RS_RET_OK
        iCancelStateSave = <value optimized out>
#13 0x08070074 in wtpWorker (arg=0x809ecb0) at wtp.c:356
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113152, 0, -1210113096, 1463726696, -411345641},
              __mask_was_saved = 0}}, __pad = {0xb7df2460, 0x0,
0xb7feff7b, 0xb7fe0cb0}}
        not_first_call = <value optimized out>
        pszDbgHdr = <value optimized out>
        thrdName = "rs:main Q:Reg", '\000' <repeats 18 times>
---Type <return> to continue, or q <return> to quit---
        pThis = 0x809ebd8
        sigSet = {__val = {2147483647, 4294967294, 4294967295 <repeats
30 times>}}
#14 0xb7f9c585 in start_thread () from /lib/i686/cmov/libpthread.so.0
No symbol table info available.
#15 0xb7f1026e in clone () from /lib/i686/cmov/libc.so.6
No symbol table info available.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Sun Jan 17 12:51:37 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 17 Jan 2010 12:51:37 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
Message-ID: <aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>

2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>>> Sent: Friday, January 15, 2010 11:57 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>>
>>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>> > Michael,
>>> >
>>> > Fix now in git, links at the bug tracker:
>>> >
>>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
>>> >
>>> > Please let me know if it works for you (the patch is a bit trickier
>>> than it
>>> > looks, so confirmations would be good).
>>>
>>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
>>> But now I'm getting a crash when rsyslog encounters the xconsole pipe
>>> config.
>>
>> I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
>> stock Debian config?
>
> Yes. As said, I just downloaded the 5.3.6 tarball applied the
> 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> got the crash. I use the default rsyslog.conf from the official debian
> package.

As an additonal hint: If I start xconsole (a process reading from
/dev/xconsole) before I start rsyslogd, then the crash does not occur.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From pgollucci at p6m7g8.com  Sun Jan 17 12:56:07 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Sun, 17 Jan 2010 06:56:07 -0500
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
Message-ID: <4B52FAD7.305@p6m7g8.com>

On 1/17/2010 6:51 AM, Michael Biebl wrote:
> As an additonal hint: If I start xconsole (a process reading from
> /dev/xconsole) before I start rsyslogd, then the crash does not occur.
Possibly related, on FreeBSD in a jail if rsyslog ever tries to write to
/dev/console, it loops in a extremely tightly loop consuming 100% of the
core its on.  [see -dn output, possibly with ktrace/kdump]

Eventually it will consume all the memory on the box and it will go boom.


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Sr. System Admin,                 Ridecharge Inc.
Consultant,                       P6M7G8 Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From ralph at crongeyer.com  Sun Jan 17 23:50:10 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Sun, 17 Jan 2010 17:50:10 -0500
Subject: [rsyslog] fromhost-ip
Message-ID: <4B539422.3020709@crongeyer.com>

Hello list,
I'm trying to send my IPOCop Firewall logs to my rsyslog server like this:

# Firewall logs #
$template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
*.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall

But I just getting this error in /var/log/syslog:

Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
Jan 17 16:49:47 log rsyslogd: the last error occured in 
/etc/rsyslog.d/remote-logs.conf, line 10
Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
will be discarded
Jan 17 16:49:47 log rsyslogd: the last error occured in 
/etc/rsyslog.conf, line 48
Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

I'm trying to log all logs from my IPCop host to 
"/var/log/server-logs/firewall/%HOSTNAME%.log" .

Can someone help me out with this?

Thanks,
Ralph

-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From pgollucci at p6m7g8.com  Mon Jan 18 00:09:22 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Sun, 17 Jan 2010 18:09:22 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B539422.3020709@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>
Message-ID: <4B5398A2.4020604@p6m7g8.com>

On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
> # Firewall logs #
> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
> 
> But I just getting this error in /var/log/syslog:
> 
> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
> swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> /etc/rsyslog.d/remote-logs.conf, line 10
> Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
> will be discarded
> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> /etc/rsyslog.conf, line 48
> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
> master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
> 
> I'm trying to log all logs from my IPCop host to 
> "/var/log/server-logs/firewall/%HOSTNAME%.log" .

I tried for 1.5 days to figure this out cutting and pasting examples
left and right.  Finally I came up with the following with works well
for me, you should be able to tweak it slightly for yourself.


$template by_prog,"/var/log/rws/%programname%.log"

:programname, regex, "^pxy.*rc\."  ?by_prog
& :omrelp:cl.dca1.rws:2514
& ~

Just sub out %programname% for %HOSTNAME%



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Sr. System Admin,                 Ridecharge Inc.
Consultant,                       P6M7G8 Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From ralph at crongeyer.com  Mon Jan 18 16:37:22 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 10:37:22 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B5398A2.4020604@p6m7g8.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
Message-ID: <4B548032.60807@crongeyer.com>

Hi Phillip,
Thanks for the response.
The %HOSTNAME% part works fine here if I do this:
$template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
*.*    -?DynFwall

However if I try to filter by IP using the "fromhost-ip" like this:
*.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

It fails to capture logs in the DynFwall template file.

I've tried to do this with the "fromhost" and the "fromhost-ip" and 
neither seem to work?

I want to have it so that a specific host IP uses a specific template.

It looks like the fromhost and the fromhost-ip arn't working at all? Or 
my config is wrong.

Dose anyone on the list have "fromhost-ip" working?

Thanks,
Ralph

Philip M. Gollucci wrote:
> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>   
>> # Firewall logs #
>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>
>> But I just getting this error in /var/log/syslog:
>>
>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
>> swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>> /etc/rsyslog.d/remote-logs.conf, line 10
>> Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
>> will be discarded
>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>> /etc/rsyslog.conf, line 48
>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
>> master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>
>> I'm trying to log all logs from my IPCop host to 
>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>     
>
> I tried for 1.5 days to figure this out cutting and pasting examples
> left and right.  Finally I came up with the following with works well
> for me, you should be able to tweak it slightly for yourself.
>
>
> $template by_prog,"/var/log/rws/%programname%.log"
>
> :programname, regex, "^pxy.*rc\."  ?by_prog
> & :omrelp:cl.dca1.rws:2514
> & ~
>
> Just sub out %programname% for %HOSTNAME%
>
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Mon Jan 18 17:24:20 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 17:24:20 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph 
> Crongeyer
> Sent: Monday, January 18, 2010 4:37 PM
> To: Philip M. Gollucci
> Cc: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> Hi Phillip,
> Thanks for the response.
> The %HOSTNAME% part works fine here if I do this:
> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> *.*    -?DynFwall

Phillip suggested the rigth thing. 
> 
> However if I try to filter by IP using the "fromhost-ip" like this:
> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

The issue is that the config is wrong. "*.*" and ":fromhost..." are both
filters. There can only be one filter in front of an action. As *.* maeans
all messages, I assume ou actually wanted to do this:

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

Which filters alls messages based on fromhost-ip.

The config format is clumpsy. I am currently talking with some folks at
Adiscon, and we will probably create a cookbook-type doc that provides
samples for some common scenarios. I guess that would be useful. Any feedback
on that effort would be welcome.

Rainer

> 
> It fails to capture logs in the DynFwall template file.
> 
> I've tried to do this with the "fromhost" and the "fromhost-ip" and 
> neither seem to work?
> 
> I want to have it so that a specific host IP uses a specific template.
> 
> It looks like the fromhost and the fromhost-ip arn't working 
> at all? Or 
> my config is wrong.
> 
> Dose anyone on the list have "fromhost-ip" working?
> 
> Thanks,
> Ralph
> 
> Philip M. Gollucci wrote:
> > On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
> >   
> >> # Firewall logs #
> >> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> >> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
> >>
> >> But I just getting this error in /var/log/syslog:
> >>
> >> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
> >> swVersion="4.4.2" x-pid="12540" 
> x-info="http://www.rsyslog.com"] (re)start
> >> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> >> /etc/rsyslog.d/remote-logs.conf, line 10
> >> Jan 17 16:49:47 log rsyslogd: warning: selector line 
> without actions 
> >> will be discarded
> >> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> >> /etc/rsyslog.conf, line 48
> >> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not 
> interpret 
> >> master config file '/etc/rsyslog.conf'. [try 
> http://www.rsyslog.com/e/2124 ]
> >>
> >> I'm trying to log all logs from my IPCop host to 
> >> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
> >>     
> >
> > I tried for 1.5 days to figure this out cutting and pasting examples
> > left and right.  Finally I came up with the following with 
> works well
> > for me, you should be able to tweak it slightly for yourself.
> >
> >
> > $template by_prog,"/var/log/rws/%programname%.log"
> >
> > :programname, regex, "^pxy.*rc\."  ?by_prog
> > & :omrelp:cl.dca1.rws:2514
> > & ~
> >
> > Just sub out %programname% for %HOSTNAME%
> >
> >
> >
> >   
> 
> 
> -- 
> Reminds me of my expedition into the wilds of Afghanistan. We 
> lost our 
> corkscrew and were compelled to live on food and water for 
> several days. - 
> WC Fields
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ralph at crongeyer.com  Mon Jan 18 18:18:18 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 12:18:18 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
Message-ID: <4B5497DA.9070405@crongeyer.com>

Hi Rainer,
Thanks for the explanation, that helps me understand how it's working.

That works, the logs are going to the correct file, however they are 
also being sent to /var/log/syslog? How can I make all the logs from my 
host "192.168.1.1" go only to the "-?DynFwall" template file?

I would like to give feedback on the cookbook let me know how I can help.

Thanks all, for your help with this.
Ralph

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com 
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph 
>> Crongeyer
>> Sent: Monday, January 18, 2010 4:37 PM
>> To: Philip M. Gollucci
>> Cc: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> Hi Phillip,
>> Thanks for the response.
>> The %HOSTNAME% part works fine here if I do this:
>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>> *.*    -?DynFwall
>>     
>
> Phillip suggested the rigth thing. 
>   
>> However if I try to filter by IP using the "fromhost-ip" like this:
>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>     
>
> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
> filters. There can only be one filter in front of an action. As *.* maeans
> all messages, I assume ou actually wanted to do this:
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>
> Which filters alls messages based on fromhost-ip.
>
> The config format is clumpsy. I am currently talking with some folks at
> Adiscon, and we will probably create a cookbook-type doc that provides
> samples for some common scenarios. I guess that would be useful. Any feedback
> on that effort would be welcome.
>
> Rainer
>
>   
>> It fails to capture logs in the DynFwall template file.
>>
>> I've tried to do this with the "fromhost" and the "fromhost-ip" and 
>> neither seem to work?
>>
>> I want to have it so that a specific host IP uses a specific template.
>>
>> It looks like the fromhost and the fromhost-ip arn't working 
>> at all? Or 
>> my config is wrong.
>>
>> Dose anyone on the list have "fromhost-ip" working?
>>
>> Thanks,
>> Ralph
>>
>> Philip M. Gollucci wrote:
>>     
>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>   
>>>       
>>>> # Firewall logs #
>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>
>>>> But I just getting this error in /var/log/syslog:
>>>>
>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
>>>> swVersion="4.4.2" x-pid="12540" 
>>>>         
>> x-info="http://www.rsyslog.com"] (re)start
>>     
>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line 
>>>>         
>> without actions 
>>     
>>>> will be discarded
>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>>>> /etc/rsyslog.conf, line 48
>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not 
>>>>         
>> interpret 
>>     
>>>> master config file '/etc/rsyslog.conf'. [try 
>>>>         
>> http://www.rsyslog.com/e/2124 ]
>>     
>>>> I'm trying to log all logs from my IPCop host to 
>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>     
>>>>         
>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>> left and right.  Finally I came up with the following with 
>>>       
>> works well
>>     
>>> for me, you should be able to tweak it slightly for yourself.
>>>
>>>
>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>
>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>> & :omrelp:cl.dca1.rws:2514
>>> & ~
>>>
>>> Just sub out %programname% for %HOSTNAME%
>>>
>>>
>>>
>>>   
>>>       
>> -- 
>> Reminds me of my expedition into the wilds of Afghanistan. We 
>> lost our 
>> corkscrew and were compelled to live on food and water for 
>> several days. - 
>> WC Fields
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 18:29:02 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 09:29:02 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B5497DA.9070405@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Hi Rainer,
> Thanks for the explanation, that helps me understand how it's working.
>
> That works, the logs are going to the correct file, however they are
> also being sent to /var/log/syslog? How can I make all the logs from my
> host "192.168.1.1" go only to the "-?DynFwall" template file?

after you tell rsyslog to put the logs in that file, you then need to tell 
rsyslog to throw the log away.

so you would do something like

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
& ~

which is logicly the same as

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
:fromhost-ip,isequal,"192.168.1.1"    ~

David Lang


> I would like to give feedback on the cookbook let me know how I can help.
>
> Thanks all, for your help with this.
> Ralph
>
> Rainer Gerhards wrote:
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>> Crongeyer
>>> Sent: Monday, January 18, 2010 4:37 PM
>>> To: Philip M. Gollucci
>>> Cc: rsyslog-users
>>> Subject: Re: [rsyslog] fromhost-ip
>>>
>>> Hi Phillip,
>>> Thanks for the response.
>>> The %HOSTNAME% part works fine here if I do this:
>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>> *.*    -?DynFwall
>>>
>>
>> Phillip suggested the rigth thing.
>>
>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>
>>
>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>> filters. There can only be one filter in front of an action. As *.* maeans
>> all messages, I assume ou actually wanted to do this:
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>
>> Which filters alls messages based on fromhost-ip.
>>
>> The config format is clumpsy. I am currently talking with some folks at
>> Adiscon, and we will probably create a cookbook-type doc that provides
>> samples for some common scenarios. I guess that would be useful. Any feedback
>> on that effort would be welcome.
>>
>> Rainer
>>
>>
>>> It fails to capture logs in the DynFwall template file.
>>>
>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>> neither seem to work?
>>>
>>> I want to have it so that a specific host IP uses a specific template.
>>>
>>> It looks like the fromhost and the fromhost-ip arn't working
>>> at all? Or
>>> my config is wrong.
>>>
>>> Dose anyone on the list have "fromhost-ip" working?
>>>
>>> Thanks,
>>> Ralph
>>>
>>> Philip M. Gollucci wrote:
>>>
>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>
>>>>
>>>>> # Firewall logs #
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>
>>>>> But I just getting this error in /var/log/syslog:
>>>>>
>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>
>>> x-info="http://www.rsyslog.com"] (re)start
>>>
>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>
>>> without actions
>>>
>>>>> will be discarded
>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>> /etc/rsyslog.conf, line 48
>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>
>>> interpret
>>>
>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>
>>> http://www.rsyslog.com/e/2124 ]
>>>
>>>>> I'm trying to log all logs from my IPCop host to
>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>
>>>>>
>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>> left and right.  Finally I came up with the following with
>>>>
>>> works well
>>>
>>>> for me, you should be able to tweak it slightly for yourself.
>>>>
>>>>
>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>
>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>> & :omrelp:cl.dca1.rws:2514
>>>> & ~
>>>>
>>>> Just sub out %programname% for %HOSTNAME%
>>>>
>>>>
>>>>
>>>>
>>>>
>>> --
>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>> lost our
>>> corkscrew and were compelled to live on food and water for
>>> several days. -
>>> WC Fields
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 18:47:03 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 12:47:03 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
Message-ID: <4B549E97.8030108@crongeyer.com>

Oh,
I tried that but I had it on the same line. So that has to be on a 
separate line?

Thanks again for the explanation that really helps me understand how 
it's working.

Thanks again for all your help with this.

Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Hi Rainer,
>> Thanks for the explanation, that helps me understand how it's working.
>>
>> That works, the logs are going to the correct file, however they are
>> also being sent to /var/log/syslog? How can I make all the logs from my
>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>     
>
> after you tell rsyslog to put the logs in that file, you then need to tell 
> rsyslog to throw the log away.
>
> so you would do something like
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
> & ~
>
> which is logicly the same as
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
> :fromhost-ip,isequal,"192.168.1.1"    ~
>
> David Lang
>
>
>   
>> I would like to give feedback on the cookbook let me know how I can help.
>>
>> Thanks all, for your help with this.
>> Ralph
>>
>> Rainer Gerhards wrote:
>>     
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>> Crongeyer
>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>> To: Philip M. Gollucci
>>>> Cc: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> Hi Phillip,
>>>> Thanks for the response.
>>>> The %HOSTNAME% part works fine here if I do this:
>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>> *.*    -?DynFwall
>>>>
>>>>         
>>> Phillip suggested the rigth thing.
>>>
>>>       
>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>>         
>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>> filters. There can only be one filter in front of an action. As *.* maeans
>>> all messages, I assume ou actually wanted to do this:
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>
>>> Which filters alls messages based on fromhost-ip.
>>>
>>> The config format is clumpsy. I am currently talking with some folks at
>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>> on that effort would be welcome.
>>>
>>> Rainer
>>>
>>>
>>>       
>>>> It fails to capture logs in the DynFwall template file.
>>>>
>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>> neither seem to work?
>>>>
>>>> I want to have it so that a specific host IP uses a specific template.
>>>>
>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>> at all? Or
>>>> my config is wrong.
>>>>
>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>
>>>> Thanks,
>>>> Ralph
>>>>
>>>> Philip M. Gollucci wrote:
>>>>
>>>>         
>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> # Firewall logs #
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>
>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>
>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>
>>>>>>             
>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>
>>>>         
>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>
>>>>>>             
>>>> without actions
>>>>
>>>>         
>>>>>> will be discarded
>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>> /etc/rsyslog.conf, line 48
>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>
>>>>>>             
>>>> interpret
>>>>
>>>>         
>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>
>>>>>>             
>>>> http://www.rsyslog.com/e/2124 ]
>>>>
>>>>         
>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>
>>>>>>
>>>>>>             
>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>> left and right.  Finally I came up with the following with
>>>>>
>>>>>           
>>>> works well
>>>>
>>>>         
>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>
>>>>>
>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>
>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>> & :omrelp:cl.dca1.rws:2514
>>>>> & ~
>>>>>
>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> --
>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>> lost our
>>>> corkscrew and were compelled to live on food and water for
>>>> several days. -
>>>> WC Fields
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 19:15:49 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 13:15:49 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B549E97.8030108@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com>
Message-ID: <4B54A555.9010007@crongeyer.com>

Ok one more question.
I have:
$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
mail.*  -?DynMail

Which logs all mail to the %HOSTNAME%.mail.log.

My guess would be:
$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail

But as Rainer explained these are both filters which won't work.

So how do I use "fromhost-ip" to send only "mail.*" logs from a 
specified host IP to the "DynMail" template?

Thanks,
Ralph

Ralph Crongeyer wrote:
> Oh,
> I tried that but I had it on the same line. So that has to be on a 
> separate line?
>
> Thanks again for the explanation that really helps me understand how 
> it's working.
>
> Thanks again for all your help with this.
>
> Ralph
>
> david at lang.hm wrote:
>   
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>   
>>     
>>> Hi Rainer,
>>> Thanks for the explanation, that helps me understand how it's working.
>>>
>>> That works, the logs are going to the correct file, however they are
>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>     
>>>       
>> after you tell rsyslog to put the logs in that file, you then need to tell 
>> rsyslog to throw the log away.
>>
>> so you would do something like
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> & ~
>>
>> which is logicly the same as
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>
>> David Lang
>>
>>
>>   
>>     
>>> I would like to give feedback on the cookbook let me know how I can help.
>>>
>>> Thanks all, for your help with this.
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>     
>>>       
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>> Crongeyer
>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>> To: Philip M. Gollucci
>>>>> Cc: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> Hi Phillip,
>>>>> Thanks for the response.
>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.*    -?DynFwall
>>>>>
>>>>>         
>>>>>           
>>>> Phillip suggested the rigth thing.
>>>>
>>>>       
>>>>         
>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>>         
>>>>>           
>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>> all messages, I assume ou actually wanted to do this:
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>> Which filters alls messages based on fromhost-ip.
>>>>
>>>> The config format is clumpsy. I am currently talking with some folks at
>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>> on that effort would be welcome.
>>>>
>>>> Rainer
>>>>
>>>>
>>>>       
>>>>         
>>>>> It fails to capture logs in the DynFwall template file.
>>>>>
>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>> neither seem to work?
>>>>>
>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>
>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>> at all? Or
>>>>> my config is wrong.
>>>>>
>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>
>>>>> Thanks,
>>>>> Ralph
>>>>>
>>>>> Philip M. Gollucci wrote:
>>>>>
>>>>>         
>>>>>           
>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>           
>>>>>>             
>>>>>>> # Firewall logs #
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>
>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>
>>>>>         
>>>>>           
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> without actions
>>>>>
>>>>>         
>>>>>           
>>>>>>> will be discarded
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> interpret
>>>>>
>>>>>         
>>>>>           
>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>
>>>>>         
>>>>>           
>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>> left and right.  Finally I came up with the following with
>>>>>>
>>>>>>           
>>>>>>             
>>>>> works well
>>>>>
>>>>>         
>>>>>           
>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>
>>>>>>
>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>
>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>> & ~
>>>>>>
>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>           
>>>>>>             
>>>>> --
>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>> lost our
>>>>> corkscrew and were compelled to live on food and water for
>>>>> several days. -
>>>>> WC Fields
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>       
>>>>         
>>>     
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 20:14:36 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 14:14:36 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54A555.9010007@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>
Message-ID: <4B54B31C.80109@crongeyer.com>

Is it possible to use "fromhost-ip" to send only "mail.*" logs from a 
specified host IP to the "DynMail" template?

I tried this:
:fromhost-ip,isequal,"192.168.1.1" & mail.* -?DynMail

But that didn't work.

How can I accomplish this?

Thanks,
Ralph
 
Ralph Crongeyer wrote:
> Ok one more question.
> I have:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  -?DynMail
>
> Which logs all mail to the %HOSTNAME%.mail.log.
>
> My guess would be:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>
> But as Rainer explained these are both filters which won't work.
>
> So how do I use "fromhost-ip" to send only "mail.*" logs from a 
> specified host IP to the "DynMail" template?
>
> Thanks,
> Ralph
>
> Ralph Crongeyer wrote:
>   
>> Oh,
>> I tried that but I had it on the same line. So that has to be on a 
>> separate line?
>>
>> Thanks again for the explanation that really helps me understand how 
>> it's working.
>>
>> Thanks again for all your help with this.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>   
>>     
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>   
>>>     
>>>       
>>>> Hi Rainer,
>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>
>>>> That works, the logs are going to the correct file, however they are
>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>     
>>>>       
>>>>         
>>> after you tell rsyslog to put the logs in that file, you then need to tell 
>>> rsyslog to throw the log away.
>>>
>>> so you would do something like
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> & ~
>>>
>>> which is logicly the same as
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>
>>> David Lang
>>>
>>>
>>>   
>>>     
>>>       
>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>
>>>> Thanks all, for your help with this.
>>>> Ralph
>>>>
>>>> Rainer Gerhards wrote:
>>>>     
>>>>       
>>>>         
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>> Crongeyer
>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>> To: Philip M. Gollucci
>>>>>> Cc: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> Hi Phillip,
>>>>>> Thanks for the response.
>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.*    -?DynFwall
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> Phillip suggested the rigth thing.
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>> all messages, I assume ou actually wanted to do this:
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>> Which filters alls messages based on fromhost-ip.
>>>>>
>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>> on that effort would be welcome.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>
>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>> neither seem to work?
>>>>>>
>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>
>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>> at all? Or
>>>>>> my config is wrong.
>>>>>>
>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>
>>>>>> Thanks,
>>>>>> Ralph
>>>>>>
>>>>>> Philip M. Gollucci wrote:
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>>>> # Firewall logs #
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>
>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> without actions
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> will be discarded
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> interpret
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> works well
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>
>>>>>>>
>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>
>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>> & ~
>>>>>>>
>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> --
>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>> lost our
>>>>>> corkscrew and were compelled to live on food and water for
>>>>>> several days. -
>>>>>> WC Fields
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>     
>>>>       
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>   
>>>     
>>>       
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 20:49:39 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 11:49:39 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B549E97.8030108@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181149010.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Oh,
> I tried that but I had it on the same line. So that has to be on a
> separate line?

yes, one line is a filter plus an action

haveing two filters on a line (like you initially tried) doesn't work, 
neither does having two actions on a line.

David Lang

> Thanks again for the explanation that really helps me understand how
> it's working.
>
> Thanks again for all your help with this.
>
> Ralph
>
> david at lang.hm wrote:
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>
>>> Hi Rainer,
>>> Thanks for the explanation, that helps me understand how it's working.
>>>
>>> That works, the logs are going to the correct file, however they are
>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>
>>
>> after you tell rsyslog to put the logs in that file, you then need to tell
>> rsyslog to throw the log away.
>>
>> so you would do something like
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> & ~
>>
>> which is logicly the same as
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>
>> David Lang
>>
>>
>>
>>> I would like to give feedback on the cookbook let me know how I can help.
>>>
>>> Thanks all, for your help with this.
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>> Crongeyer
>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>> To: Philip M. Gollucci
>>>>> Cc: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> Hi Phillip,
>>>>> Thanks for the response.
>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.*    -?DynFwall
>>>>>
>>>>>
>>>> Phillip suggested the rigth thing.
>>>>
>>>>
>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>>
>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>> all messages, I assume ou actually wanted to do this:
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>> Which filters alls messages based on fromhost-ip.
>>>>
>>>> The config format is clumpsy. I am currently talking with some folks at
>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>> on that effort would be welcome.
>>>>
>>>> Rainer
>>>>
>>>>
>>>>
>>>>> It fails to capture logs in the DynFwall template file.
>>>>>
>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>> neither seem to work?
>>>>>
>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>
>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>> at all? Or
>>>>> my config is wrong.
>>>>>
>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>
>>>>> Thanks,
>>>>> Ralph
>>>>>
>>>>> Philip M. Gollucci wrote:
>>>>>
>>>>>
>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> # Firewall logs #
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>
>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>
>>>>>>>
>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>
>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>
>>>>>>>
>>>>> without actions
>>>>>
>>>>>
>>>>>>> will be discarded
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>
>>>>>>>
>>>>> interpret
>>>>>
>>>>>
>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>
>>>>>>>
>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>
>>>>>
>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>> left and right.  Finally I came up with the following with
>>>>>>
>>>>>>
>>>>> works well
>>>>>
>>>>>
>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>
>>>>>>
>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>
>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>> & ~
>>>>>>
>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>> lost our
>>>>> corkscrew and were compelled to live on food and water for
>>>>> several days. -
>>>>> WC Fields
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From david at lang.hm  Mon Jan 18 20:57:41 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 11:57:41 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54A555.9010007@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Ok one more question.
> I have:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  -?DynMail
>
> Which logs all mail to the %HOSTNAME%.mail.log.
>
> My guess would be:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>
> But as Rainer explained these are both filters which won't work.
>
> So how do I use "fromhost-ip" to send only "mail.*" logs from a
> specified host IP to the "DynMail" template?

you need to use the more powerful/complex

if ((condition) and (condition)) action

line format

David Lang

> Thanks,
> Ralph
>
> Ralph Crongeyer wrote:
>> Oh,
>> I tried that but I had it on the same line. So that has to be on a
>> separate line?
>>
>> Thanks again for the explanation that really helps me understand how
>> it's working.
>>
>> Thanks again for all your help with this.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>
>>>
>>>> Hi Rainer,
>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>
>>>> That works, the logs are going to the correct file, however they are
>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>
>>>>
>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>> rsyslog to throw the log away.
>>>
>>> so you would do something like
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> & ~
>>>
>>> which is logicly the same as
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>
>>> David Lang
>>>
>>>
>>>
>>>
>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>
>>>> Thanks all, for your help with this.
>>>> Ralph
>>>>
>>>> Rainer Gerhards wrote:
>>>>
>>>>
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>> Crongeyer
>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>> To: Philip M. Gollucci
>>>>>> Cc: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> Hi Phillip,
>>>>>> Thanks for the response.
>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.*    -?DynFwall
>>>>>>
>>>>>>
>>>>>>
>>>>> Phillip suggested the rigth thing.
>>>>>
>>>>>
>>>>>
>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>>
>>>>>>
>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>> all messages, I assume ou actually wanted to do this:
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>> Which filters alls messages based on fromhost-ip.
>>>>>
>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>> on that effort would be welcome.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>
>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>> neither seem to work?
>>>>>>
>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>
>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>> at all? Or
>>>>>> my config is wrong.
>>>>>>
>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>
>>>>>> Thanks,
>>>>>> Ralph
>>>>>>
>>>>>> Philip M. Gollucci wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> # Firewall logs #
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>
>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> without actions
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> will be discarded
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> interpret
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> works well
>>>>>>
>>>>>>
>>>>>>
>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>
>>>>>>>
>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>
>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>> & ~
>>>>>>>
>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>> lost our
>>>>>> corkscrew and were compelled to live on food and water for
>>>>>> several days. -
>>>>>> WC Fields
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 21:37:03 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 15:37:03 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
Message-ID: <4B54C66F.80506@crongeyer.com>

Thanks David,
Ok so now I'm trying this:

$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then 
?DynMail

After a restart of rsyslog there are no errors in /var/log/syslog 
however no logs are being collected?

Thanks for your help with this David.

Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Ok one more question.
>> I have:
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> mail.*  -?DynMail
>>
>> Which logs all mail to the %HOSTNAME%.mail.log.
>>
>> My guess would be:
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>
>> But as Rainer explained these are both filters which won't work.
>>
>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>> specified host IP to the "DynMail" template?
>>     
>
> you need to use the more powerful/complex
>
> if ((condition) and (condition)) action
>
> line format
>
> David Lang
>
>   
>> Thanks,
>> Ralph
>>
>> Ralph Crongeyer wrote:
>>     
>>> Oh,
>>> I tried that but I had it on the same line. So that has to be on a
>>> separate line?
>>>
>>> Thanks again for the explanation that really helps me understand how
>>> it's working.
>>>
>>> Thanks again for all your help with this.
>>>
>>> Ralph
>>>
>>> david at lang.hm wrote:
>>>
>>>       
>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Hi Rainer,
>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>
>>>>> That works, the logs are going to the correct file, however they are
>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>
>>>>>
>>>>>           
>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>> rsyslog to throw the log away.
>>>>
>>>> so you would do something like
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>> & ~
>>>>
>>>> which is logicly the same as
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>>
>>>>         
>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>
>>>>> Thanks all, for your help with this.
>>>>> Ralph
>>>>>
>>>>> Rainer Gerhards wrote:
>>>>>
>>>>>
>>>>>           
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>> Crongeyer
>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>> To: Philip M. Gollucci
>>>>>>> Cc: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>
>>>>>>> Hi Phillip,
>>>>>>> Thanks for the response.
>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.*    -?DynFwall
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Phillip suggested the rigth thing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>
>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>> on that effort would be welcome.
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>
>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>> neither seem to work?
>>>>>>>
>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>
>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>> at all? Or
>>>>>>> my config is wrong.
>>>>>>>
>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ralph
>>>>>>>
>>>>>>> Philip M. Gollucci wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> # Firewall logs #
>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>
>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> without actions
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> will be discarded
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> interpret
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> works well
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>
>>>>>>>>
>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>
>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>> & ~
>>>>>>>>
>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> --
>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>> lost our
>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>> several days. -
>>>>>>> WC Fields
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 21:41:26 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 12:41:26 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54C66F.80506@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
	<4B54C66F.80506@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Thanks David,
> Ok so now I'm trying this:
>
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then
> ?DynMail

you can't use single quotes, you must use double quotes (apparently the 
config language uses single quotes for something else, I don't know what)

I've tripped over this several times now.

David Lang

> After a restart of rsyslog there are no errors in /var/log/syslog
> however no logs are being collected?
>
> Thanks for your help with this David.
>
> Ralph
>
> david at lang.hm wrote:
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>
>>> Ok one more question.
>>> I have:
>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>> mail.*  -?DynMail
>>>
>>> Which logs all mail to the %HOSTNAME%.mail.log.
>>>
>>> My guess would be:
>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>>
>>> But as Rainer explained these are both filters which won't work.
>>>
>>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>>> specified host IP to the "DynMail" template?
>>>
>>
>> you need to use the more powerful/complex
>>
>> if ((condition) and (condition)) action
>>
>> line format
>>
>> David Lang
>>
>>
>>> Thanks,
>>> Ralph
>>>
>>> Ralph Crongeyer wrote:
>>>
>>>> Oh,
>>>> I tried that but I had it on the same line. So that has to be on a
>>>> separate line?
>>>>
>>>> Thanks again for the explanation that really helps me understand how
>>>> it's working.
>>>>
>>>> Thanks again for all your help with this.
>>>>
>>>> Ralph
>>>>
>>>> david at lang.hm wrote:
>>>>
>>>>
>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi Rainer,
>>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>>
>>>>>> That works, the logs are going to the correct file, however they are
>>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>>
>>>>>>
>>>>>>
>>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>>> rsyslog to throw the log away.
>>>>>
>>>>> so you would do something like
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>> & ~
>>>>>
>>>>> which is logicly the same as
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>>
>>>>>> Thanks all, for your help with this.
>>>>>> Ralph
>>>>>>
>>>>>> Rainer Gerhards wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>>> Crongeyer
>>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>>> To: Philip M. Gollucci
>>>>>>>> Cc: rsyslog-users
>>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>>
>>>>>>>> Hi Phillip,
>>>>>>>> Thanks for the response.
>>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.*    -?DynFwall
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Phillip suggested the rigth thing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>>
>>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>
>>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>>
>>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>>> on that effort would be welcome.
>>>>>>>
>>>>>>> Rainer
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>>
>>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>>> neither seem to work?
>>>>>>>>
>>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>>
>>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>>> at all? Or
>>>>>>>> my config is wrong.
>>>>>>>>
>>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ralph
>>>>>>>>
>>>>>>>> Philip M. Gollucci wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> # Firewall logs #
>>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>>
>>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>>
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> without actions
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> will be discarded
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> interpret
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> works well
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>>
>>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>>> & ~
>>>>>>>>>
>>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>>> lost our
>>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>>> several days. -
>>>>>>>> WC Fields
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 21:52:32 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 15:52:32 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>	<4B54C66F.80506@crongeyer.com>
	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
Message-ID: <4B54CA10.2060103@crongeyer.com>

When I switched to double quotes I get the error in /var/log/syslog and 
no logs are collected?
I switched back to single quots and restart and no error but still no logs?

What else may I be doing wrong?

Thanks,
Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Thanks David,
>> Ok so now I'm trying this:
>>
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then
>> ?DynMail
>>     
>
> you can't use single quotes, you must use double quotes (apparently the 
> config language uses single quotes for something else, I don't know what)
>
> I've tripped over this several times now.
>
> David Lang
>
>   
>> After a restart of rsyslog there are no errors in /var/log/syslog
>> however no logs are being collected?
>>
>> Thanks for your help with this David.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>     
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>
>>>       
>>>> Ok one more question.
>>>> I have:
>>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>>> mail.*  -?DynMail
>>>>
>>>> Which logs all mail to the %HOSTNAME%.mail.log.
>>>>
>>>> My guess would be:
>>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>>>
>>>> But as Rainer explained these are both filters which won't work.
>>>>
>>>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>>>> specified host IP to the "DynMail" template?
>>>>
>>>>         
>>> you need to use the more powerful/complex
>>>
>>> if ((condition) and (condition)) action
>>>
>>> line format
>>>
>>> David Lang
>>>
>>>
>>>       
>>>> Thanks,
>>>> Ralph
>>>>
>>>> Ralph Crongeyer wrote:
>>>>
>>>>         
>>>>> Oh,
>>>>> I tried that but I had it on the same line. So that has to be on a
>>>>> separate line?
>>>>>
>>>>> Thanks again for the explanation that really helps me understand how
>>>>> it's working.
>>>>>
>>>>> Thanks again for all your help with this.
>>>>>
>>>>> Ralph
>>>>>
>>>>> david at lang.hm wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Hi Rainer,
>>>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>>>
>>>>>>> That works, the logs are going to the correct file, however they are
>>>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>>>> rsyslog to throw the log away.
>>>>>>
>>>>>> so you would do something like
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>> & ~
>>>>>>
>>>>>> which is logicly the same as
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>>>
>>>>>>> Thanks all, for your help with this.
>>>>>>> Ralph
>>>>>>>
>>>>>>> Rainer Gerhards wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>>>> Crongeyer
>>>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>>>> To: Philip M. Gollucci
>>>>>>>>> Cc: rsyslog-users
>>>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>>>
>>>>>>>>> Hi Phillip,
>>>>>>>>> Thanks for the response.
>>>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>> *.*    -?DynFwall
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Phillip suggested the rigth thing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>>>
>>>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>
>>>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>>>
>>>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>>>> on that effort would be welcome.
>>>>>>>>
>>>>>>>> Rainer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>>>
>>>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>>>> neither seem to work?
>>>>>>>>>
>>>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>>>
>>>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>>>> at all? Or
>>>>>>>>> my config is wrong.
>>>>>>>>>
>>>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ralph
>>>>>>>>>
>>>>>>>>> Philip M. Gollucci wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> # Firewall logs #
>>>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>>>
>>>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>>>
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> without actions
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> will be discarded
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> interpret
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> works well
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>>>
>>>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>>>> & ~
>>>>>>>>>>
>>>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> --
>>>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>>>> lost our
>>>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>>>> several days. -
>>>>>>>>> WC Fields
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rsyslog mailing list
>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>>> http://www.rsyslog.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>           
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 21:56:30 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 12:56:30 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54CA10.2060103@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
	<4B54C66F.80506@crongeyer.com>
	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
	<4B54CA10.2060103@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> When I switched to double quotes I get the error in /var/log/syslog and
> no logs are collected?

what was the error you got this time?

David Lang



From rgerhards at hq.adiscon.com  Mon Jan 18 21:59:54 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 21:59:54 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>

David,

Single quotes are right in the scripting engine (double quotes are reserved
for future use - they shall provide the capability to extend macros, e.g.
$A="BC" => '$A' is the string "$A", while "$A" is supposed to be the string
"BC").

I don't have an idea what may be wrong, but running rsyslog in debug mode
will most probably pinpoint it.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 18, 2010 9:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> 
> > When I switched to double quotes I get the error in 
> /var/log/syslog and
> > no logs are collected?
> 
> what was the error you got this time?
> 
> David Lang
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Mon Jan 18 22:02:04 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 13:02:04 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Rainer Gerhards wrote:

> David,
>
> Single quotes are right in the scripting engine (double quotes are reserved
> for future use - they shall provide the capability to extend macros, e.g.
> $A="BC" => '$A' is the string "$A", while "$A" is supposed to be the string
> "BC").

that is the normal behavior of single vs double quotes, but in such 
situations it's normal for 'ABC' and "ABC" to be equivalent, it's only 
when you have variables involved that there would be a difference.

David Lang

> I don't have an idea what may be wrong, but running rsyslog in debug mode
> will most probably pinpoint it.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 18, 2010 9:57 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>> When I switched to double quotes I get the error in
>> /var/log/syslog and
>>> no logs are collected?
>>
>> what was the error you got this time?
>>
>> David Lang
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From ralph at crongeyer.com  Mon Jan 18 22:02:27 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:02:27 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>	<4B54C66F.80506@crongeyer.com>	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>	<4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
Message-ID: <4B54CC63.3010103@crongeyer.com>

With double quots I get this in /var/log/syslog:

Jan 18 16:00:22 log rsyslogd: [origin software="rsyslogd" 
swVersion="4.4.2" x-pid="15703" x-info="http://www.rsyslog.com"] (re)start
Jan 18 16:00:22 log rsyslogd: the last error occured in 
/etc/rsyslog.d/remote-logs.conf, line 6
Jan 18 16:00:22 log rsyslogd: warning: selector line without actions 
will be discarded
Jan 18 16:00:22 log rsyslogd: the last error occured in 
/etc/rsyslog.conf, line 48
Jan 18 16:00:22 log rsyslogd-2124: CONFIG ERROR: could not interpret 
master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> When I switched to double quotes I get the error in /var/log/syslog and
>> no logs are collected?
>>     
>
> what was the error you got this time?
>
> David Lang
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Mon Jan 18 22:03:50 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 22:03:50 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 18, 2010 10:02 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> 
> > David,
> >
> > Single quotes are right in the scripting engine (double 
> quotes are reserved
> > for future use - they shall provide the capability to 
> extend macros, e.g.
> > $A="BC" => '$A' is the string "$A", while "$A" is supposed 
> to be the string
> > "BC").
> 
> that is the normal behavior of single vs double quotes, but in such 
> situations it's normal for 'ABC' and "ABC" to be equivalent, 
> it's only 
> when you have variables involved that there would be a difference.

Jup, that's right - but double quotes are not yet implemented ;)

Rainer
> 
> David Lang
> 
> > I don't have an idea what may be wrong, but running rsyslog 
> in debug mode
> > will most probably pinpoint it.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com
> >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
> david at lang.hm
> >> Sent: Monday, January 18, 2010 9:57 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] fromhost-ip
> >>
> >> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >>
> >>> When I switched to double quotes I get the error in
> >> /var/log/syslog and
> >>> no logs are collected?
> >>
> >> what was the error you got this time?
> >>
> >> David Lang
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ralph at crongeyer.com  Mon Jan 18 22:27:49 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:27:49 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
Message-ID: <4B54D255.30505@crongeyer.com>

Here's the debug output when configured with single quotes.
I'm sending this off the list to Rainer.
David, let me know if you want this also.

Thanks guys,
Ralph

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com 
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 18, 2010 10:02 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>
>>     
>>> David,
>>>
>>> Single quotes are right in the scripting engine (double 
>>>       
>> quotes are reserved
>>     
>>> for future use - they shall provide the capability to 
>>>       
>> extend macros, e.g.
>>     
>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed 
>>>       
>> to be the string
>>     
>>> "BC").
>>>       
>> that is the normal behavior of single vs double quotes, but in such 
>> situations it's normal for 'ABC' and "ABC" to be equivalent, 
>> it's only 
>> when you have variables involved that there would be a difference.
>>     
>
> Jup, that's right - but double quotes are not yet implemented ;)
>
> Rainer
>   
>> David Lang
>>
>>     
>>> I don't have an idea what may be wrong, but running rsyslog 
>>>       
>> in debug mode
>>     
>>> will most probably pinpoint it.
>>>
>>> Rainer
>>>
>>>       
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
>>>>         
>> david at lang.hm
>>     
>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>
>>>>         
>>>>> When I switched to double quotes I get the error in
>>>>>           
>>>> /var/log/syslog and
>>>>         
>>>>> no logs are collected?
>>>>>           
>>>> what was the error you got this time?
>>>>
>>>> David Lang
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 22:47:53 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:47:53 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54D255.30505@crongeyer.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
	<4B54D255.30505@crongeyer.com>
Message-ID: <4B54D709.4050408@crongeyer.com>

This ma be of help:

0928.085091536:imrelp.c: Message has legacy syslog format.
0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
0928.085443830:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
0928.085812887:imrelp.c: tcpSend returns 17
0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
0928.086029125:imrelp.c: relp engine is dispatching frame with command 
'syslog'
0928.086053430:imrelp.c: in 'syslog' command handler
0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg 
2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost 
connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
0928.086124392:imrelp.c: Message has legacy syslog format.
0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
0928.086514402:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
0928.087044659:imrelp.c: tcpSend returns 17
0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
0928.087110313:imrelp.c: relp engine is dispatching frame with command 
'syslog'
0928.087131545:imrelp.c: in 'syslog' command handler
0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg 
2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect 
from 81-64-60-151.rev.numericable.fr[81.64.60.151]
0928.087200552:imrelp.c: Message has legacy syslog format.
0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
0928.087609280:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
0928.088020802:imrelp.c: tcpSend returns 17
0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
0928.088099586:imrelp.c: ***<librelp> calling select, active file 
descriptors (max 23): 6 7 23
0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout, 
worker terminating...
0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08, 
terminated, num workers now 0
0988.088339377:main queue:Reg/w0: destructor for debug call stack 
0x9bd1260 called


Ralph Crongeyer wrote:
> Here's the debug output when configured with single quotes.
> I'm sending this off the list to Rainer.
> David, let me know if you want this also.
>
> Thanks guys,
> Ralph
>
> Rainer Gerhards wrote:
>   
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com 
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Monday, January 18, 2010 10:02 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] fromhost-ip
>>>
>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>
>>>     
>>>       
>>>> David,
>>>>
>>>> Single quotes are right in the scripting engine (double 
>>>>       
>>>>         
>>> quotes are reserved
>>>     
>>>       
>>>> for future use - they shall provide the capability to 
>>>>       
>>>>         
>>> extend macros, e.g.
>>>     
>>>       
>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed 
>>>>       
>>>>         
>>> to be the string
>>>     
>>>       
>>>> "BC").
>>>>       
>>>>         
>>> that is the normal behavior of single vs double quotes, but in such 
>>> situations it's normal for 'ABC' and "ABC" to be equivalent, 
>>> it's only 
>>> when you have variables involved that there would be a difference.
>>>     
>>>       
>> Jup, that's right - but double quotes are not yet implemented ;)
>>
>> Rainer
>>   
>>     
>>> David Lang
>>>
>>>     
>>>       
>>>> I don't have an idea what may be wrong, but running rsyslog 
>>>>       
>>>>         
>>> in debug mode
>>>     
>>>       
>>>> will most probably pinpoint it.
>>>>
>>>> Rainer
>>>>
>>>>       
>>>>         
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
>>>>>         
>>>>>           
>>> david at lang.hm
>>>     
>>>       
>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>
>>>>>         
>>>>>           
>>>>>> When I switched to double quotes I get the error in
>>>>>>           
>>>>>>             
>>>>> /var/log/syslog and
>>>>>         
>>>>>           
>>>>>> no logs are collected?
>>>>>>           
>>>>>>             
>>>>> what was the error you got this time?
>>>>>
>>>>> David Lang
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>       
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>     
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 22:52:32 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 13:52:32 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54D709.4050408@crongeyer.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
	<4B54D255.30505@crongeyer.com> <4B54D709.4050408@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>

Ok, this says that fromhost-ip is not being set in your case.

I think I ran into a similar problem before, are you starting with -x to 
disable name lookups?

try changing from fromhost-ip to fromhost

David Lang

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> This ma be of help:
>
> 0928.085091536:imrelp.c: Message has legacy syslog format.
> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
> 0928.085443830:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.085812887:imrelp.c: tcpSend returns 17
> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> 0928.086029125:imrelp.c: relp engine is dispatching frame with command
> 'syslog'
> 0928.086053430:imrelp.c: in 'syslog' command handler
> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
> connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> 0928.086124392:imrelp.c: Message has legacy syslog format.
> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
> 0928.086514402:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.087044659:imrelp.c: tcpSend returns 17
> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> 0928.087110313:imrelp.c: relp engine is dispatching frame with command
> 'syslog'
> 0928.087131545:imrelp.c: in 'syslog' command handler
> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect
> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> 0928.087200552:imrelp.c: Message has legacy syslog format.
> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
> 0928.087609280:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.088020802:imrelp.c: tcpSend returns 17
> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
> descriptors (max 23): 6 7 23
> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout,
> worker terminating...
> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08,
> terminated, num workers now 0
> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> 0x9bd1260 called
>
>
> Ralph Crongeyer wrote:
>> Here's the debug output when configured with single quotes.
>> I'm sending this off the list to Rainer.
>> David, let me know if you want this also.
>>
>> Thanks guys,
>> Ralph
>>
>> Rainer Gerhards wrote:
>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>> Sent: Monday, January 18, 2010 10:02 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>>
>>>>
>>>>
>>>>> David,
>>>>>
>>>>> Single quotes are right in the scripting engine (double
>>>>>
>>>>>
>>>> quotes are reserved
>>>>
>>>>
>>>>> for future use - they shall provide the capability to
>>>>>
>>>>>
>>>> extend macros, e.g.
>>>>
>>>>
>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>>>>>
>>>>>
>>>> to be the string
>>>>
>>>>
>>>>> "BC").
>>>>>
>>>>>
>>>> that is the normal behavior of single vs double quotes, but in such
>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>>>> it's only
>>>> when you have variables involved that there would be a difference.
>>>>
>>>>
>>> Jup, that's right - but double quotes are not yet implemented ;)
>>>
>>> Rainer
>>>
>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>>> I don't have an idea what may be wrong, but running rsyslog
>>>>>
>>>>>
>>>> in debug mode
>>>>
>>>>
>>>>> will most probably pinpoint it.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>>>>>>
>>>>>>
>>>> david at lang.hm
>>>>
>>>>
>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> When I switched to double quotes I get the error in
>>>>>>>
>>>>>>>
>>>>>> /var/log/syslog and
>>>>>>
>>>>>>
>>>>>>> no logs are collected?
>>>>>>>
>>>>>>>
>>>>>> what was the error you got this time?
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 23:12:24 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 17:12:24 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>	<4B54D255.30505@crongeyer.com>
	<4B54D709.4050408@crongeyer.com>
	<alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
Message-ID: <4B54DCC8.3020504@crongeyer.com>

No, I'm starting with -c4.

I'll give it a try but ultimately  I need to filter in IP.

I'll try it when I get back from dinner......

Thanks again for your help with this guys.

david at lang.hm wrote:
> Ok, this says that fromhost-ip is not being set in your case.
>
> I think I ran into a similar problem before, are you starting with -x to 
> disable name lookups?
>
> try changing from fromhost-ip to fromhost
>
> David Lang
>
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> This ma be of help:
>>
>> 0928.085091536:imrelp.c: Message has legacy syslog format.
>> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.085443830:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.085812887:imrelp.c: tcpSend returns 17
>> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
>> 0928.086029125:imrelp.c: relp engine is dispatching frame with command
>> 'syslog'
>> 0928.086053430:imrelp.c: in 'syslog' command handler
>> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
>> connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> 0928.086124392:imrelp.c: Message has legacy syslog format.
>> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.086514402:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.087044659:imrelp.c: tcpSend returns 17
>> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
>> 0928.087110313:imrelp.c: relp engine is dispatching frame with command
>> 'syslog'
>> 0928.087131545:imrelp.c: in 'syslog' command handler
>> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect
>> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> 0928.087200552:imrelp.c: Message has legacy syslog format.
>> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.087609280:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.088020802:imrelp.c: tcpSend returns 17
>> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
>> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
>> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
>> descriptors (max 23): 6 7 23
>> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout,
>> worker terminating...
>> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
>> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
>> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08,
>> terminated, num workers now 0
>> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
>> 0x9bd1260 called
>>
>>
>> Ralph Crongeyer wrote:
>>     
>>> Here's the debug output when configured with single quotes.
>>> I'm sending this off the list to Rainer.
>>> David, let me know if you want this also.
>>>
>>> Thanks guys,
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>
>>>       
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>>> Sent: Monday, January 18, 2010 10:02 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> David,
>>>>>>
>>>>>> Single quotes are right in the scripting engine (double
>>>>>>
>>>>>>
>>>>>>             
>>>>> quotes are reserved
>>>>>
>>>>>
>>>>>           
>>>>>> for future use - they shall provide the capability to
>>>>>>
>>>>>>
>>>>>>             
>>>>> extend macros, e.g.
>>>>>
>>>>>
>>>>>           
>>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>>>>>>
>>>>>>
>>>>>>             
>>>>> to be the string
>>>>>
>>>>>
>>>>>           
>>>>>> "BC").
>>>>>>
>>>>>>
>>>>>>             
>>>>> that is the normal behavior of single vs double quotes, but in such
>>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>>>>> it's only
>>>>> when you have variables involved that there would be a difference.
>>>>>
>>>>>
>>>>>           
>>>> Jup, that's right - but double quotes are not yet implemented ;)
>>>>
>>>> Rainer
>>>>
>>>>
>>>>         
>>>>> David Lang
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I don't have an idea what may be wrong, but running rsyslog
>>>>>>
>>>>>>
>>>>>>             
>>>>> in debug mode
>>>>>
>>>>>
>>>>>           
>>>>>> will most probably pinpoint it.
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>> david at lang.hm
>>>>>
>>>>>
>>>>>           
>>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>>>> To: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>
>>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> When I switched to double quotes I get the error in
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> /var/log/syslog and
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> no logs are collected?
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> what was the error you got this time?
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Tue Jan 19 10:44:04 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 10:44:04 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>	<4B54D255.30505@crongeyer.com><4B54D709.4050408@crongeyer.com><alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
	<4B54DCC8.3020504@crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F9@GRFEXC.intern.adiscon.com>

RELP did not provide fromhost-ip until recently. You need to use the most
recent development version of the git master branch (to be released soon)
TOGETHER with the most recent version of librelp to get that information.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> Sent: Monday, January 18, 2010 11:12 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> No, I'm starting with -c4.
> 
> I'll give it a try but ultimately  I need to filter in IP.
> 
> I'll try it when I get back from dinner......
> 
> Thanks again for your help with this guys.
> 
> david at lang.hm wrote:
> > Ok, this says that fromhost-ip is not being set in your case.
> >
> > I think I ran into a similar problem before, are you starting with -x
> to
> > disable name lookups?
> >
> > try changing from fromhost-ip to fromhost
> >
> > David Lang
> >
> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >
> >
> >> This ma be of help:
> >>
> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.085812887:imrelp.c: tcpSend returns 17
> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
> command
> >> 'syslog'
> >> 0928.086053430:imrelp.c: in 'syslog' command handler
> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
> >> connection after RCPT from 81-64-60-
> 151.rev.numericable.fr[81.64.60.151]
> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.087044659:imrelp.c: tcpSend returns 17
> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
> command
> >> 'syslog'
> >> 0928.087131545:imrelp.c: in 'syslog' command handler
> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
> disconnect
> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.088020802:imrelp.c: tcpSend returns 17
> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> >> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
> >> descriptors (max 23): 6 7 23
> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
> timeout,
> >> worker terminating...
> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
> command 1
> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
> terminating
> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
> 9bb5a08,
> >> terminated, num workers now 0
> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> >> 0x9bd1260 called
> >>
> >>
> >> Ralph Crongeyer wrote:
> >>
> >>> Here's the debug output when configured with single quotes.
> >>> I'm sending this off the list to Rainer.
> >>> David, let me know if you want this also.
> >>>
> >>> Thanks guys,
> >>> Ralph
> >>>
> >>> Rainer Gerhards wrote:
> >>>
> >>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces at lists.adiscon.com
> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> david at lang.hm
> >>>>> Sent: Monday, January 18, 2010 10:02 PM
> >>>>> To: rsyslog-users
> >>>>> Subject: Re: [rsyslog] fromhost-ip
> >>>>>
> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> David,
> >>>>>>
> >>>>>> Single quotes are right in the scripting engine (double
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> quotes are reserved
> >>>>>
> >>>>>
> >>>>>
> >>>>>> for future use - they shall provide the capability to
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> extend macros, e.g.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> to be the string
> >>>>>
> >>>>>
> >>>>>
> >>>>>> "BC").
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> that is the normal behavior of single vs double quotes, but in
> such
> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
> >>>>> it's only
> >>>>> when you have variables involved that there would be a
> difference.
> >>>>>
> >>>>>
> >>>>>
> >>>> Jup, that's right - but double quotes are not yet implemented ;)
> >>>>
> >>>> Rainer
> >>>>
> >>>>
> >>>>
> >>>>> David Lang
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> I don't have an idea what may be wrong, but running rsyslog
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> in debug mode
> >>>>>
> >>>>>
> >>>>>
> >>>>>> will most probably pinpoint it.
> >>>>>>
> >>>>>> Rainer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>> david at lang.hm
> >>>>>
> >>>>>
> >>>>>
> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
> >>>>>>> To: rsyslog-users
> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
> >>>>>>>
> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> When I switched to double quotes I get the error in
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> /var/log/syslog and
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> no logs are collected?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> what was the error you got this time?
> >>>>>>>
> >>>>>>> David Lang
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> rsyslog mailing list
> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>>> http://www.rsyslog.com
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> rsyslog mailing list
> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>> http://www.rsyslog.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> 
> --
> Reminds me of my expedition into the wilds of Afghanistan. We lost our
> corkscrew and were compelled to live on food and water for several
> days. -
> WC Fields
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 14:53:44 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 14:53:44 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>

Michael,

I tried to reproduce, but I can not get to this error. Could you provide me a
debug log of the failed startup?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, January 17, 2010 12:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> Sent: Friday, January 15, 2010 11:57 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> > Michael,
> >> >
> >> > Fix now in git, links at the bug tracker:
> >> >
> >> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >> >
> >> > Please let me know if it works for you (the patch is a bit
> trickier
> >> than it
> >> > looks, so confirmations would be good).
> >>
> >> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> >> But now I'm getting a crash when rsyslog encounters the xconsole
> pipe
> >> config.
> >
> > I am a bit puzzled, but will try to reproduce that on my Debian box.
> I assume
> > stock Debian config?
> 
> Yes. As said, I just downloaded the 5.3.6 tarball applied the
> 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> got the crash. I use the default rsyslog.conf from the official debian
> package.
> I attached a backtrace. Hope that helps
> 
> (gdb) run -c4 -d
> Starting program: /usr/sbin/rsyslogd -c4 -d
> [Thread debugging using libthread_db enabled]
> [New Thread 0xb7df2b70 (LWP 5162)]
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xb7df2b70 (LWP 5162)]
> qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
> pUsr=0x80b72c0) at queue.c:2256
> 2256            if(pThis->qType != QUEUETYPE_DIRECT) {
> (gdb) bt full
> #0  qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
> pUsr=0x80b72c0) at queue.c:2256
>         iRet = <value optimized out>
>         iCancelStateSave = <value optimized out>
> #1  0x0807cd5b in actionWriteToAction (pAction=0x80ac8d8) at
> ../action.c:1169
>         pMsgSave = 0x0
>         iRet = <value optimized out>
> #2  0x0807d4fe in doActionCallAction (pAction=0x80ac8d8,
> pMsg=0x80b72c0) at ../action.c:1244
> No locals.
> #3  actionCallAction (pAction=0x80ac8d8, pMsg=0x80b72c0) at
> ../action.c:1274
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113864, 0, -1210114056, 1464119912, -422385385},
>               __mask_was_saved = 0}}, __pad = {0xb7df2240, 0x0,
> 0xb7df20d0, 0xb7feb27f}}
>         __cancel_arg = 0x80ad0a0
>         not_first_call = <value optimized out>
>         iRet = -1210113864
> #4  0x080794d7 in processMsgDoActions (pData=0xfffff815,
> pParam=0xb7df20b8) at rule.c:113
>         iRet = <value optimized out>
>         iRetMod = <value optimized out>
> #5  0x080627ba in llExecFunc (pThis=0x80ac9a0, pFunc=0x8079480
> <processMsgDoActions>, pParam=0xb7df20b8) at linkedlist.c:391
>         iRet = <value optimized out>
>         iRetLL = <value optimized out>
>         pData = 0x80ac8d8
>         llCookie = 0x80ac508
>         llCookiePrev = 0x0
> #6  0x08079007 in processMsg (pThis=0x80ac968, pMsg=0x80b72c0) at
> rule.c:299
>         bProcessMsg = 1
>         DoActData = {bPrevWasSuspended = 0, pMsg = 0x80b72c0}
>         iRet = RS_RET_OK
> #7  0x080781ba in processMsgDoRules (pData=0x80ac968,
> pParam=0x80b72c0) at ruleset.c:145
>         iRet = <value optimized out>
> #8  0x080627ba in llExecFunc (pThis=0x809eb68, pFunc=0x8078190
> <processMsgDoRules>, pParam=0x80b72c0) at linkedlist.c:391
>         iRet = <value optimized out>
>         iRetLL = <value optimized out>
>         pData = 0x80ac968
>         llCookie = 0x80ac478
>         llCookiePrev = 0x80ac4f8
> #9  0x0807876d in processMsg (pMsg=0x80b72c0) at ruleset.c:164
>         pThis = <value optimized out>
>         iRet = <value optimized out>
> #10 0x080506ab in msgConsumer (notNeeded=0x0, pBatch=0x809ecc8,
> pbShutdownImmediate=0x80ad348) at syslogd.c:614
>         i = 0
>         pMsg = 0x80b72c0
>         localRet = RS_RET_IO_ERROR
> #11 0x08077ca4 in ConsumerReg (pThis=0x80ad338, pWti=0x809ecb0) at
> queue.c:1638
>         iCancelStateSave = 1
>         iRet = <value optimized out>
> #12 0x08070526 in wtiWorker (pThis=0x809ecb0) at wti.c:286
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113200, 0, -1210113384, 1463800424, -411898089},
>               __mask_was_saved = 0}}, __pad = {0xb7df2350, 0x0, 0x0,
> 0x809002c}}
>         not_first_call = <value optimized out>
>         pWtp = 0x809ebd8
>         bInactivityTOOccured = <value optimized out>
>         localRet = <value optimized out>
>         terminateRet = RS_RET_OK
>         iCancelStateSave = <value optimized out>
> #13 0x08070074 in wtpWorker (arg=0x809ecb0) at wtp.c:356
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113152, 0, -1210113096, 1463726696, -411345641},
>               __mask_was_saved = 0}}, __pad = {0xb7df2460, 0x0,
> 0xb7feff7b, 0xb7fe0cb0}}
>         not_first_call = <value optimized out>
>         pszDbgHdr = <value optimized out>
>         thrdName = "rs:main Q:Reg", '\000' <repeats 18 times>
> ---Type <return> to continue, or q <return> to quit---
>         pThis = 0x809ebd8
>         sigSet = {__val = {2147483647, 4294967294, 4294967295 <repeats
> 30 times>}}
> #14 0xb7f9c585 in start_thread () from /lib/i686/cmov/libpthread.so.0
> No symbol table info available.
> #15 0xb7f1026e in clone () from /lib/i686/cmov/libc.so.6
> No symbol table info available.
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 14:55:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 14:55:59 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<4B52FAD7.305@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FD@GRFEXC.intern.adiscon.com>

Philip,

I wil try to set up this as well. In the mean time, could you tell me if it
happens with the plain 5.3.6 or with the newer git tree (with the patch).
Without the patch, I can already see why it can happen, with it, I do not yet
have a clear understanding of the issue.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Sunday, January 17, 2010 12:56 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On 1/17/2010 6:51 AM, Michael Biebl wrote:
> > As an additonal hint: If I start xconsole (a process reading from
> > /dev/xconsole) before I start rsyslogd, then the crash does not
> occur.
> Possibly related, on FreeBSD in a jail if rsyslog ever tries to write
> to
> /dev/console, it loops in a extremely tightly loop consuming 100% of
> the
> core its on.  [see -dn output, possibly with ktrace/kdump]
> 
> Eventually it will consume all the memory on the box and it will go
> boom.
> 
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Sr. System Admin,                 Ridecharge Inc.
> Consultant,                       P6M7G8 Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Tue Jan 19 14:58:50 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 19 Jan 2010 14:58:50 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001190558k138e1868y2ec8763e8a574a00@mail.gmail.com>

2010/1/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> I tried to reproduce, but I can not get to this error. Could you provide me a
> debug log of the failed startup?

There is no debug output of rsyslog before it crashes. All I can get
is the gdb output I already attached

The missing debug output when using -d is another bug I already
mentioned in this thread.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Tue Jan 19 15:20:03 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:20:03 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
	<4B509EC0.3050504@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>

Jack,

I have written this small patch for v4:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c89ae6b
41d1c5838b9a42b0

I assume it will apply without problems in v5 as well, but I have not yet
tried as I am doing some more work on v4 first (hoping to be able to save a
merge or two, which clutter up the git history...).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 5:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> Rainer Gerhards wrote:
> > -----Original Message-----
> >> If that is modified as follows: *.*
> >> /var/log/syslog   # Comment goes here
> >>
> >
> > It's even worse: data is written to the file "/var/log/syslog   #
> > Comment goes here"!
> 
> Aaaaahhh - that finally explains some odd files lurking in /var/log!
> 
> That also explains why there's no error message on stdout/stdderr -
> there's no error, as far as rsyslog is concerned. [/me feels stupid]
> 
> I would really like to be able to put comments on the ends of config
> lines, as I can with many other packages. But now I know what's going
> on
> here. Thanks!
> 
> --
> Jack.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 15:50:42 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:50:42 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
	<aa76cdac1001190558k138e1868y2ec8763e8a574a00@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FF@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Tuesday, January 19, 2010 2:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > I tried to reproduce, but I can not get to this error. Could you
> provide me a
> > debug log of the failed startup?
> 
> There is no debug output of rsyslog before it crashes. All I can get
> is the gdb output I already attached
> 
> The missing debug output when using -d is another bug I already
> mentioned in this thread.

slipped my mind, I should have opened a bug tracker. As I thought, it was a
regression from "debug on demand" mode. Patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bd03b86c6322c82fc9f66712
2f4365e339f28ccc

Rainer

> 
> Cheers,
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mrdemeanour at jackpot.uk.net  Tue Jan 19 15:50:17 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Tue, 19 Jan 2010 14:50:17 +0000
Subject: [rsyslog] comments in file actions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>	<4B509EC0.3050504@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
Message-ID: <4B55C6A9.5010008@jackpot.uk.net>

Rainer Gerhards wrote:
> Jack,
> 
> I have written this small patch for v4:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c89ae6b
>  41d1c5838b9a42b0
> 
> I assume it will apply without problems in v5 as well, but I have not
> yet tried as I am doing some more work on v4 first (hoping to be able
> to save a merge or two, which clutter up the git history...).

OK - have to go out now, but I will try this tomorrow and report back.

-- 
Jack.



From rgerhards at hq.adiscon.com  Tue Jan 19 15:51:33 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:51:33 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>	<4B509EC0.3050504@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
	<4B55C6A9.5010008@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103700@GRFEXC.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Tuesday, January 19, 2010 3:50 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> Rainer Gerhards wrote:
> > Jack,
> >
> > I have written this small patch for v4:
> >
> >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c
> 89ae6b
> >  41d1c5838b9a42b0
> >
> > I assume it will apply without problems in v5 as well, but I have not
> > yet tried as I am doing some more work on v4 first (hoping to be able
> > to save a merge or two, which clutter up the git history...).
> 
> OK - have to go out now, but I will try this tomorrow and report back.

excellent - thanks!

Rainer

> 
> --
> Jack.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ralph at crongeyer.com  Tue Jan 19 16:22:14 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Tue, 19 Jan 2010 10:22:14 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F9@GRFEXC.intern.adiscon.com>
Message-ID: <5fa6e0144d8003c7c72edff17f9f1675@webmail.crongeyer.com>

 Ok.
I'll try it with TCP (@@).
This weekend I'll build a deb of the latest rsyslog and relp and check it
out.

Would I ned the latest on both the rsyslog server and the client or just the
server?

Thanks,
Ralph
 
----------------original message-----------------
From: "Rainer Gerhards" rgerhards at hq.adiscon.com
To: "rsyslog-users" rsyslog at lists.adiscon.com
Date: Tue, 19 Jan 2010 10:44:04 +0100
-------------------------------------------------
 
 
> RELP did not provide fromhost-ip until recently. You need to use the most
> recent development version of the git master branch (to be released soon)
> TOGETHER with the most recent version of librelp to get that information.
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
>> Sent: Monday, January 18, 2010 11:12 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>> 
>> No, I'm starting with -c4.
>> 
>> I'll give it a try but ultimately I need to filter in IP.
>> 
>> I'll try it when I get back from dinner......
>> 
>> Thanks again for your help with this guys.
>> 
>> david at lang.hm wrote:
>> > Ok, this says that fromhost-ip is not being set in your case.
>> >
>> > I think I ran into a similar problem before, are you starting with -x
>> to
>> > disable name lookups?
>> >
>> > try changing from fromhost-ip to fromhost
>> >
>> > David Lang
>> >
>> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>> >
>> >
>> >> This ma be of help:
>> >>
>> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
>> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.085812887:imrelp.c: tcpSend returns 17
>> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
>> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
>> command
>> >> 'syslog'
>> >> 0928.086053430:imrelp.c: in 'syslog' command handler
>> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: 
>> lost
>> >> connection after RCPT from 81-64-60-
>> 151.rev.numericable.fr[81.64.60.151]
>> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
>> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.087044659:imrelp.c: tcpSend returns 17
>> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
>> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
>> command
>> >> 'syslog'
>> >> 0928.087131545:imrelp.c: in 'syslog' command handler
>> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
>> disconnect
>> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
>> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.088020802:imrelp.c: tcpSend returns 17
>> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
>> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
>> >> 0928.088099586:imrelp.c: ***
 calling select, active file
>> >> descriptors (max 23): 6 7 23
>> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
>> timeout,
>> >> worker terminating...
>> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
>> command 1
>> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
>> terminating
>> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
>> 9bb5a08,
>> >> terminated, num workers now 0
>> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
>> >> 0x9bd1260 called
>> >>
>> >>
>> >> Ralph Crongeyer wrote:
>> >>
>> >>> Here's the debug output when configured with single quotes.
>> >>> I'm sending this off the list to Rainer.
>> >>> David, let me know if you want this also.
>> >>>
>> >>> Thanks guys,
>> >>> Ralph
>> >>>
>> >>> Rainer Gerhards wrote:
>> >>>
>> >>>
>> >>>>> -----Original Message-----
>> >>>>> From: rsyslog-bounces at lists.adiscon.com
>> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>> david at lang.hm
>> >>>>> Sent: Monday, January 18, 2010 10:02 PM
>> >>>>> To: rsyslog-users
>> >>>>> Subject: Re: [rsyslog] fromhost-ip
>> >>>>>
>> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> David,
>> >>>>>>
>> >>>>>> Single quotes are right in the scripting engine (double
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> quotes are reserved
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> for future use - they shall provide the capability to
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> extend macros, e.g.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> to be the string
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> "BC").
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> that is the normal behavior of single vs double quotes, but in
>> such
>> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>> >>>>> it's only
>> >>>>> when you have variables involved that there would be a
>> difference.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>> Jup, that's right - but double quotes are not yet implemented ;)
>> >>>>
>> >>>> Rainer
>> >>>>
>> >>>>
>> >>>>
>> >>>>> David Lang
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> I don't have an idea what may be wrong, but running rsyslog
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> in debug mode
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> will most probably pinpoint it.
>> >>>>>>
>> >>>>>> Rainer
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> -----Original Message-----
>> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
>> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>> david at lang.hm
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>> >>>>>>> To: rsyslog-users
>> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
>> >>>>>>>
>> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> When I switched to double quotes I get the error in
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>> /var/log/syslog and
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> no logs are collected?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>> what was the error you got this time?
>> >>>>>>>
>> >>>>>>> David Lang
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> rsyslog mailing list
>> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>>>> http://www.rsyslog.com
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> rsyslog mailing list
>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>>> http://www.rsyslog.com
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> _______________________________________________
>> >>>>> rsyslog mailing list
>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>> http://www.rsyslog.com
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>> _______________________________________________
>> >>>> rsyslog mailing list
>> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>> http://www.rsyslog.com
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >>
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> 
>> 
>> --
>> Reminds me of my expedition into the wilds of Afghanistan. We lost our
>> corkscrew and were compelled to live on food and water for several
>> days. -
>> WC Fields
>> 
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 




From rgerhards at hq.adiscon.com  Tue Jan 19 16:28:01 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 16:28:01 +0100
Subject: [rsyslog] fromhost-ip
References: <5fa6e0144d8003c7c72edff17f9f1675@webmail.crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103702@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> Sent: Tuesday, January 19, 2010 4:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
>  Ok.
> I'll try it with TCP (@@).
> This weekend I'll build a deb of the latest rsyslog and relp and check
> it
> out.
> 
> Would I ned the latest on both the rsyslog server and the client or
> just the
> server?

The server should be sufficient. The issue is that librelp < 1.0.0 has the
information, but does not pass it down to the call (imrelp in rsyslog case).
So imrelp decided to use "[unset]" instead of anything else (librelp actually
passes down the hostname twice). In librelp >= 1.0.0 this is corrected, it
now provides the ip address. However, you also need the new imrelp, as it now
needs to use that property. All of this, however, is done on the server, so
no dependency on the client should exist.

I have done these changes in early december 2009 as a side-activity for
something else relp related. My memory has a bit vanished since them, but I
think I conveyed the right information (but you now know I may be wrong in
case something works other than expected - in that case, ask here first
before getting nuts ;)).

Rainer

> 
> Thanks,
> Ralph
> 
> ----------------original message-----------------
> From: "Rainer Gerhards" rgerhards at hq.adiscon.com
> To: "rsyslog-users" rsyslog at lists.adiscon.com
> Date: Tue, 19 Jan 2010 10:44:04 +0100
> -------------------------------------------------
> 
> 
> > RELP did not provide fromhost-ip until recently. You need to use the
> most
> > recent development version of the git master branch (to be released
> soon)
> > TOGETHER with the most recent version of librelp to get that
> information.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> >> Sent: Monday, January 18, 2010 11:12 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] fromhost-ip
> >>
> >> No, I'm starting with -c4.
> >>
> >> I'll give it a try but ultimately I need to filter in IP.
> >>
> >> I'll try it when I get back from dinner......
> >>
> >> Thanks again for your help with this guys.
> >>
> >> david at lang.hm wrote:
> >> > Ok, this says that fromhost-ip is not being set in your case.
> >> >
> >> > I think I ran into a similar problem before, are you starting with
> -x
> >> to
> >> > disable name lookups?
> >> >
> >> > try changing from fromhost-ip to fromhost
> >> >
> >> > David Lang
> >> >
> >> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >> >
> >> >
> >> >> This ma be of help:
> >> >>
> >> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
> >> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.085416731:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.085812887:imrelp.c: tcpSend returns 17
> >> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> >> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
> >> command
> >> >> 'syslog'
> >> >> 0928.086053430:imrelp.c: in 'syslog' command handler
> >> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5',
> msg
> >> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]:
> >> lost
> >> >> connection after RCPT from 81-64-60-
> >> 151.rev.numericable.fr[81.64.60.151]
> >> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
> >> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.086486185:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.087044659:imrelp.c: tcpSend returns 17
> >> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> >> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
> >> command
> >> >> 'syslog'
> >> >> 0928.087131545:imrelp.c: in 'syslog' command handler
> >> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5',
> msg
> >> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
> >> disconnect
> >> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> >> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
> >> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.087581622:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.088020802:imrelp.c: tcpSend returns 17
> >> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> >> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> >> >> 0928.088099586:imrelp.c: ***
>  calling select, active file
> >> >> descriptors (max 23): 6 7 23
> >> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
> >> timeout,
> >> >> worker terminating...
> >> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
> >> command 1
> >> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
> >> terminating
> >> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
> >> 9bb5a08,
> >> >> terminated, num workers now 0
> >> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> >> >> 0x9bd1260 called
> >> >>
> >> >>
> >> >> Ralph Crongeyer wrote:
> >> >>
> >> >>> Here's the debug output when configured with single quotes.
> >> >>> I'm sending this off the list to Rainer.
> >> >>> David, let me know if you want this also.
> >> >>>
> >> >>> Thanks guys,
> >> >>> Ralph
> >> >>>
> >> >>> Rainer Gerhards wrote:
> >> >>>
> >> >>>
> >> >>>>> -----Original Message-----
> >> >>>>> From: rsyslog-bounces at lists.adiscon.com
> >> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >> david at lang.hm
> >> >>>>> Sent: Monday, January 18, 2010 10:02 PM
> >> >>>>> To: rsyslog-users
> >> >>>>> Subject: Re: [rsyslog] fromhost-ip
> >> >>>>>
> >> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> David,
> >> >>>>>>
> >> >>>>>> Single quotes are right in the scripting engine (double
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> quotes are reserved
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> for future use - they shall provide the capability to
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> extend macros, e.g.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> to be the string
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> "BC").
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> that is the normal behavior of single vs double quotes, but in
> >> such
> >> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
> >> >>>>> it's only
> >> >>>>> when you have variables involved that there would be a
> >> difference.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>> Jup, that's right - but double quotes are not yet implemented
> ;)
> >> >>>>
> >> >>>> Rainer
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>> David Lang
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> I don't have an idea what may be wrong, but running rsyslog
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> in debug mode
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> will most probably pinpoint it.
> >> >>>>>>
> >> >>>>>> Rainer
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> -----Original Message-----
> >> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
> >> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>> david at lang.hm
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
> >> >>>>>>> To: rsyslog-users
> >> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
> >> >>>>>>>
> >> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> When I switched to double quotes I get the error in
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>> /var/log/syslog and
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> no logs are collected?
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>> what was the error you got this time?
> >> >>>>>>>
> >> >>>>>>> David Lang
> >> >>>>>>>
> >> >>>>>>> _______________________________________________
> >> >>>>>>> rsyslog mailing list
> >> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>>>> http://www.rsyslog.com
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>> _______________________________________________
> >> >>>>>> rsyslog mailing list
> >> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>>> http://www.rsyslog.com
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> _______________________________________________
> >> >>>>> rsyslog mailing list
> >> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>> http://www.rsyslog.com
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>> _______________________________________________
> >> >>>> rsyslog mailing list
> >> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>> http://www.rsyslog.com
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> >
> >>
> >>
> >> --
> >> Reminds me of my expedition into the wilds of Afghanistan. We lost
> our
> >> corkscrew and were compelled to live on food and water for several
> >> days. -
> >> WC Fields
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From epiphani at gmail.com  Tue Jan 19 19:55:55 2010
From: epiphani at gmail.com (Aaron Wiebe)
Date: Tue, 19 Jan 2010 13:55:55 -0500
Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
Message-ID: <e7ca40f71001191055k308bfa4dw2682e4fc702fefa8@mail.gmail.com>

Greetings,

I'm trying to sort out applying rulesets to IMUDP, and there is not
module-specific documentation for imudp as there is with imtcp.

What is the equivilent for udp input of:

$InputTCPServerInputName
$InputTCPServerBindRuleSet

?

I want to be able to apply rules to specific ports in the same way I
can with tcp...  Changing TCP to UDP doesn't seem to work.

-Aaron


From david at lang.hm  Wed Jan 20 00:26:15 2010
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jan 2010 15:26:15 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
	<aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
	<alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
Message-ID: <alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>

I've now had v5.3.6 running on my production environemnets since friday 
with no problems

one side effect of the cleanups is that previously when I had multiple 
filters write to one file I was getting lots of corrupt lines, but the 
change to have omfile write each transaction rather than just as the 
buffer filled up seems to have eliminated this (it went from 10's of 
thousands of corrupted lines/day to none over the weekend and monday, 
tonight's report will be the acid test to see if it's fully cleaned up)

I realize there is still a window for corruption (if two output threads 
running at the same time both decide they need to write at the same time), 
but it seems that in practice it's effectively gone.

David Lang

On Fri, 15 Jan 2010, david at lang.hm wrote:

> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> BTW, I'm actually surprised that you don't encounter those problems 
>> yourself.
>
> I'm running 5.3.5 still, I haven't had time to build a new version (hopefully 
> tomorrow)
>
> David Lang
>


From rgerhards at hq.adiscon.com  Wed Jan 20 12:09:18 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 12:09:18 +0100
Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
References: <e7ca40f71001191055k308bfa4dw2682e4fc702fefa8@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103710@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Aaron Wiebe
> Sent: Tuesday, January 19, 2010 7:56 PM
> To: rsyslog-users
> Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
> 
> Greetings,
> 
> I'm trying to sort out applying rulesets to IMUDP, and there is not
> module-specific documentation for imudp as there is with imtcp.
> 
> What is the equivilent for udp input of:
> 
> $InputTCPServerInputName
> $InputTCPServerBindRuleSet
> 
> ?
> 
> I want to be able to apply rules to specific ports in the same way I
> can with tcp...  Changing TCP to UDP doesn't seem to work.

In v4, imudp has considerable less functionality than imtcp has. I think I
changed that only in v5. For example, you can NOT bind a ruleset to a
listener in imudp.

Rainer


From rgerhards at hq.adiscon.com  Wed Jan 20 16:19:46 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 16:19:46 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>

David,

thanks for the feedback - and a quick note.

With the new engine, you can do ruleset inclusion (via omruleset[1]). That is
probably *the* method to handle files that are written to by multiple
actions. Of course, no need to change if all works in default config. But you
can gain some extra performance by using buffered mode (for busy files) and
to use that, you need to have only one action write to each file. This is
where ruleset inclusion enters the game.

Rainer

[1] http://www.rsyslog.com/doc-omruleset.html

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 12:26 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> I've now had v5.3.6 running on my production environemnets since friday
> with no problems
> 
> one side effect of the cleanups is that previously when I had multiple
> filters write to one file I was getting lots of corrupt lines, but the
> change to have omfile write each transaction rather than just as the
> buffer filled up seems to have eliminated this (it went from 10's of
> thousands of corrupted lines/day to none over the weekend and monday,
> tonight's report will be the acid test to see if it's fully cleaned up)
> 
> I realize there is still a window for corruption (if two output threads
> running at the same time both decide they need to write at the same
> time),
> but it seems that in practice it's effectively gone.
> 
> David Lang
> 
> On Fri, 15 Jan 2010, david at lang.hm wrote:
> 
> > On Fri, 15 Jan 2010, Michael Biebl wrote:
> >
> >> BTW, I'm actually surprised that you don't encounter those problems
> >> yourself.
> >
> > I'm running 5.3.5 still, I haven't had time to build a new version
> (hopefully
> > tomorrow)
> >
> > David Lang
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 20 16:22:12 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 16:22:12 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>

I forgot to mention:

> I realize there is still a window for corruption (if two output threads
> running at the same time both decide they need to write at the same
> time),
> but it seems that in practice it's effectively gone.

The current code writes a single line with a single API call. I guess that
call is rather atomic from an OS point of view, so the window of corruption
probably don't even exists with current rsyslog and linux code.

Rainer


From david at lang.hm  Wed Jan 20 17:57:02 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 08:57:02 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001200851350.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Rainer Gerhards wrote:

> David,
>
> thanks for the feedback - and a quick note.
>
> With the new engine, you can do ruleset inclusion (via omruleset[1]). That is
> probably *the* method to handle files that are written to by multiple
> actions. Of course, no need to change if all works in default config. But you
> can gain some extra performance by using buffered mode (for busy files) and
> to use that, you need to have only one action write to each file. This is
> where ruleset inclusion enters the game.

thanks for this, I was thinking about how this could be improved, but this 
looks like it deals with the issue.

on my central box I currently have all the logs written to one file, roll 
that every 5 min, and then at night split this into 45 different files 
based on 100 simplified program names (where I strip out versions so that 
blah-2.3[123] and blah-2.4[123] end up in the same file). I was thinking 
of experimenting to see what happened if I did this in rsyslog instead. 
This is a very good pointer to what I would need to do.

David Lang

> Rainer
>
> [1] http://www.rsyslog.com/doc-omruleset.html
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Wednesday, January 20, 2010 12:26 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> I've now had v5.3.6 running on my production environemnets since friday
>> with no problems
>>
>> one side effect of the cleanups is that previously when I had multiple
>> filters write to one file I was getting lots of corrupt lines, but the
>> change to have omfile write each transaction rather than just as the
>> buffer filled up seems to have eliminated this (it went from 10's of
>> thousands of corrupted lines/day to none over the weekend and monday,
>> tonight's report will be the acid test to see if it's fully cleaned up)
>>
>> I realize there is still a window for corruption (if two output threads
>> running at the same time both decide they need to write at the same
>> time),
>> but it seems that in practice it's effectively gone.
>>
>> David Lang
>>
>> On Fri, 15 Jan 2010, david at lang.hm wrote:
>>
>>> On Fri, 15 Jan 2010, Michael Biebl wrote:
>>>
>>>> BTW, I'm actually surprised that you don't encounter those problems
>>>> yourself.
>>>
>>> I'm running 5.3.5 still, I haven't had time to build a new version
>> (hopefully
>>> tomorrow)
>>>
>>> David Lang
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Wed Jan 20 18:00:51 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 18:00:51 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm><alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001200851350.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371D@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 5:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On Wed, 20 Jan 2010, Rainer Gerhards wrote:
> 
> > David,
> >
> > thanks for the feedback - and a quick note.
> >
> > With the new engine, you can do ruleset inclusion (via omruleset[1]).
> That is
> > probably *the* method to handle files that are written to by multiple
> > actions. Of course, no need to change if all works in default config.
> But you
> > can gain some extra performance by using buffered mode (for busy
> files) and
> > to use that, you need to have only one action write to each file.
> This is
> > where ruleset inclusion enters the game.
> 
> thanks for this, I was thinking about how this could be improved, but
> this
> looks like it deals with the issue.
> 
> on my central box I currently have all the logs written to one file,
> roll
> that every 5 min, and then at night split this into 45 different files
> based on 100 simplified program names (where I strip out versions so
> that
> blah-2.3[123] and blah-2.4[123] end up in the same file). I was
> thinking
> of experimenting to see what happened if I did this in rsyslog instead.
> This is a very good pointer to what I would need to do.

I would be quite interested in feedback on omruleset. I doubt anyone has put
it into production yet, at least in a demanding environment (aka "bugs to be
expected" ;)). Note that this functionality is very hard to configure with
the current config language... (it was omruleset that made me believe that
finally something must be done to improve that part of the system).

Rainer

> 
> David Lang
> 
> > Rainer
> >
> > [1] http://www.rsyslog.com/doc-omruleset.html
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >> Sent: Wednesday, January 20, 2010 12:26 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> I've now had v5.3.6 running on my production environemnets since
> friday
> >> with no problems
> >>
> >> one side effect of the cleanups is that previously when I had
> multiple
> >> filters write to one file I was getting lots of corrupt lines, but
> the
> >> change to have omfile write each transaction rather than just as the
> >> buffer filled up seems to have eliminated this (it went from 10's of
> >> thousands of corrupted lines/day to none over the weekend and
> monday,
> >> tonight's report will be the acid test to see if it's fully cleaned
> up)
> >>
> >> I realize there is still a window for corruption (if two output
> threads
> >> running at the same time both decide they need to write at the same
> >> time),
> >> but it seems that in practice it's effectively gone.
> >>
> >> David Lang
> >>
> >> On Fri, 15 Jan 2010, david at lang.hm wrote:
> >>
> >>> On Fri, 15 Jan 2010, Michael Biebl wrote:
> >>>
> >>>> BTW, I'm actually surprised that you don't encounter those
> problems
> >>>> yourself.
> >>>
> >>> I'm running 5.3.5 still, I haven't had time to build a new version
> >> (hopefully
> >>> tomorrow)
> >>>
> >>> David Lang
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Wed Jan 20 18:00:56 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 09:00:56 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001200857260.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Rainer Gerhards wrote:

> I forgot to mention:
>
>> I realize there is still a window for corruption (if two output threads
>> running at the same time both decide they need to write at the same
>> time),
>> but it seems that in practice it's effectively gone.
>
> The current code writes a single line with a single API call. I guess that
> call is rather atomic from an OS point of view, so the window of corruption
> probably don't even exists with current rsyslog and linux code.

even when things are batched? with 5.3.5 I was very definantly 
experianceing problems with lines getting combined in the writes when I 
had multiple outputs to the same file (using different formats to fix up 
bad input)

David Lang


From rgerhards at hq.adiscon.com  Wed Jan 20 18:02:22 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 18:02:22 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm><alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001200857260.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371E@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 6:01 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On Wed, 20 Jan 2010, Rainer Gerhards wrote:
> 
> > I forgot to mention:
> >
> >> I realize there is still a window for corruption (if two output
> threads
> >> running at the same time both decide they need to write at the same
> >> time),
> >> but it seems that in practice it's effectively gone.
> >
> > The current code writes a single line with a single API call. I guess
> that
> > call is rather atomic from an OS point of view, so the window of
> corruption
> > probably don't even exists with current rsyslog and linux code.
> 
> even when things are batched? with 5.3.5 I was very definantly
> experianceing problems with lines getting combined in the writes when I
> had multiple outputs to the same file (using different formats to fix
> up
> bad input)

good point. No, you are right. With batches, buffered mode is used by
default, with a flush at the end of batch.

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From sur5r at sur5r.net  Wed Jan 20 19:20:31 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Wed, 20 Jan 2010 19:20:31 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
Message-ID: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 6 Jan 2010 16:14:59 +0100
Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:

> which encoding should be chosen for the database when using postgres?

As far as I understand the syslog protocol (at least the legacy one), it has
no concept of character encodings at all.  So if you simply want to make sure
that everything ends up in the database "as is", then choose SQL_ASCII.

> My rsyslog version is 4.4.3.
> 
> Which client_encoding does rsyslog use in ompgsql?

Right now, it does net set an encoding by itself, so the database default
applies. If I'm not mistaken, you can even set that per user from inside of
postgres. So I would rather vote against another configuration parameter here.

> I currently have set UTF-8 on the database. It worked for a while until
> some special message arrived at the server where postgres denies the INSERT:
> 
> 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for 
> encoding "UTF8": 0xd220
> 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
> the byte sequence does not match the encoding expected by the server, which
> is controlled by "client_encoding".

Were you able to isolate the message? Or find out which program was sending
it?

> Now rsyslog is not able to log anything... it is currently spooling to disk 
> because it "hangs" at this message not being accepted by postgres.

This is bad, because if the machine is an open syslog server that simply
collects everything it gets, we have a potential DoS vector here.

I can think of three options:

* Drop the message and report that we did so. That would be rather easy,
  but might not be what people want.

* Re-insert the message after converting it from ASCII to UTF-8 or whatever
  the DB encoding is. But this might/will produce garbage if the input is not
  ASCII. It also creates more load on the system if these messages are
  frequent. Guessing the input encoding is hard or even impossible, depending
  on the set you guess from.

* Make the database SQL_ASCII. This will silently accept anything but will
  create nonsense from UTF/UCS encoded messages. Also might create trouble
  for programs like phplogcon that analyze the logs.

For me, this sums up to one question:

Can we make ompgsql UTF/UCS-clean and at the same time not choke on non-UTF8
strings? Everyone is trying to be UTF-8 clean these days, so it would be bad
if ompgsql could not keep up.

Comments please.

Regards,
Jakab Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktXSW8ACgkQ1YAhDic+adbqXACeIJcx6GW6PhSXFO1YF72PafJG
7t8AoLNwnJYMZ4bssqMZt/nkTIPWs0LI
=vuWN
-----END PGP SIGNATURE-----

From david at lang.hm  Wed Jan 20 19:44:42 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 10:44:42 -0800 (PST)
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Jakob Haufe wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> This is bad, because if the machine is an open syslog server that simply
> collects everything it gets, we have a potential DoS vector here.
>
> I can think of three options:
>
> * Drop the message and report that we did so. That would be rather easy,
>  but might not be what people want.
>
> * Re-insert the message after converting it from ASCII to UTF-8 or whatever
>  the DB encoding is. But this might/will produce garbage if the input is not
>  ASCII. It also creates more load on the system if these messages are
>  frequent. Guessing the input encoding is hard or even impossible, depending
>  on the set you guess from.
>
> * Make the database SQL_ASCII. This will silently accept anything but will
>  create nonsense from UTF/UCS encoded messages. Also might create trouble
>  for programs like phplogcon that analyze the logs.
>
> For me, this sums up to one question:
>
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on non-UTF8
> strings? Everyone is trying to be UTF-8 clean these days, so it would be bad
> if ompgsql could not keep up.

my thought is that just like we have a filter to change control characters 
to escape sequences, it would be good to have a filter to escape non-ascii 
characters. this will mangle other character sets, but they are unlikly to 
go through cleanly anyway.

David Lang


From marc.schiffbauer at mightycare.de  Thu Jan 21 01:49:47 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Thu, 21 Jan 2010 01:49:47 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <201001210149.48041.marc.schiffbauer@mightycare.de>

Am Mittwoch, 20. Januar 2010 19:20:31 schrieb Jakob Haufe:
> On Wed, 6 Jan 2010 16:14:59 +0100
> 
> Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:
> > which encoding should be chosen for the database when using postgres?
> 
> As far as I understand the syslog protocol (at least the legacy one), it
> has no concept of character encodings at all.  So if you simply want to
> make sure that everything ends up in the database "as is", then choose
> SQL_ASCII.

This is what I did in the end. And it works good now.

> 
> > My rsyslog version is 4.4.3.
> > 
> > Which client_encoding does rsyslog use in ompgsql?
> 
> Right now, it does net set an encoding by itself, so the database default
> applies. If I'm not mistaken, you can even set that per user from inside of
> postgres. So I would rather vote against another configuration parameter
> here.

ACK

> 
> > I currently have set UTF-8 on the database. It worked for a while until
> > some special message arrived at the server where postgres denies the
> > INSERT:
> > 
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen
> > if the byte sequence does not match the encoding expected by the server,
> > which is controlled by "client_encoding".
> 
> Were you able to isolate the message? Or find out which program was sending
> it?

I was able to identify it: Some servers sent data about strings found in 
system BIOS (read by dmidecode so something like that)
It was just some strange charcters in a model or device name string set by a 
hardware vendor (compaq IIRC)


> 
> > Now rsyslog is not able to log anything... it is currently spooling to
> > disk because it "hangs" at this message not being accepted by postgres.
> 
> This is bad, because if the machine is an open syslog server that simply
> collects everything it gets, we have a potential DoS vector here.
> 

True.


> I can think of three options:
> 
> * Drop the message and report that we did so. That would be rather easy,
>   but might not be what people want.
> 

But this might be the best option I guess. Maybe the original message could 
then be written to a special logfile on disk.

> * Re-insert the message after converting it from ASCII to UTF-8 or whatever
>   the DB encoding is. But this might/will produce garbage if the input is
> not ASCII. It also creates more load on the system if these messages are
> frequent. Guessing the input encoding is hard or even impossible,
> depending on the set you guess from.

Yes but this would be an option. I would vote for creating a warning message 
in these cases as well.

> 
> * Make the database SQL_ASCII. This will silently accept anything but will
>   create nonsense from UTF/UCS encoded messages. Also might create trouble
>   for programs like phplogcon that analyze the logs.
> 

This is what I did. And phplogcon had no problems at all displaying everything 
as expected. Even those strange messages that were not accepted by postgres 
look as in the original message that came via syslog.

This might only work if apache and the browser all "speak" UTF-8.



> For me, this sums up to one question:
> 
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> non-UTF8 strings? Everyone is trying to be UTF-8 clean these days, so it
> would be bad if ompgsql could not keep up.

I think this is a special case because rsyslog is not the originator of those 
messages. It "just" transports them. And because the syslog-Protocol does not 
define something like encoding in any way the best thing to do is just leave 
those strings "as-is" and make the database behind it do so as well with 
SQL_ASCII.

I thing everythign else will be error prone in some way. The Documentation of 
rsyslog should bring a big fat NOTE that the database must be SQL_ASCII as 
other wise thesrings might not be accepted.


-Marc

> 
> Comments please.
> 
> Regards,
> Jakab Haufe (sur5r)


From xkubina at fi.muni.cz  Thu Jan 21 11:21:27 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Thu, 21 Jan 2010 11:21:27 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>	<4B4DCF31.6090105@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
Message-ID: <4B582AA7.3040906@fi.muni.cz>

Rainer Gerhards wrote:
> Thanks for the code. Unfortunately, adding the config switch to it is not
> quite easy in that case (good I asked for the actual code). I'd say that you
> best do it similar to the other config directives, like the authentication
> mode. They actual directives are in the upper level code (imtcp/omfwd).
> There, they are shuffled over to the instance data, which goes along with
> each of the configured listeners/sender. Then, when a new network stream is
> created, the params are passed down to the generic stream interface and there
> passed down to the selected stream driver, which finally stores and acts on
> them. It's clumpsy and quite some work, but that is what is needed for the
> old config system. You probably need to add around 50 to 100 lines of code
> altogether to the various files. It's not complex, but easy to forget
> something. Best start by a directive (like $..AuthMode), see how it is
> handled (and passed down) in imtcp and work your way down the stack ;)
>
> Rainer 
>
>   
Hi Rainer,

I have added some code that I have thought was necessary, but
I am stuck now. In nsd_gtls.c is added function:

static rsRetVal
SetAddClientCN(nsd_t *pNsd, int mode)
{
    DEFiRet;
    nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;

    ISOBJ_TYPE_assert((pThis), nsd_gtls);
    if(mode != 0 && mode != 1) {
        errmsg.LogError(0, RS_RET_INVALID_DRVR_MODE, "error: driver mode 
%d not supported by "
                "gtls netstream driver", mode);
        ABORT_FINALIZE(RS_RET_INVALID_DRVR_MODE);
    }
       
    pThis->iAddClientCN = mode;
        dbgprintf("GTLS:%d\n", pThis->iAddClientCN);
finalize_it:
    RETiRet;
}

The "dbgprintf" shows correct value in pThis, but if I check 
pThis->iAddClientCN
later in function:

static rsRetVal
Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
{
    DEFiRet;
    ssize_t iBytesCopy; /* how many bytes are to be copied to the client 
buffer? */
    nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
    ISOBJ_TYPE_assert(pThis, nsd_gtls);

        cstr_t *pstrCN = NULL;
        const gnutls_datum *cert_list;
    unsigned int cert_list_size = 0;
        gnutls_x509_crt cert;       
        int len = 0;       
        char *buf_temp;

    if(pThis->bAbortConn)
        ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);

    if(pThis->iMode == 0) {
        CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
        FINALIZE;
    }

    /* --- in TLS mode now --- */

    /* Buffer logic applies only if we are in TLS mode. Here we
     * assume that we will switch from plain to TLS, but never back. This
     * assumption may be unsafe, but it is the model for the time being 
and I
     * do not see any valid reason why we should switch back to plain 
TCP after
     * we were in TLS mode. However, in that case we may lose something that
     * is already in the receive buffer ... risk accepted. -- rgerhards, 
2008-06-23
     */

    if(pThis->pszRcvBuf == NULL) {
        /* we have no buffer, so we need to malloc one */
        CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
        pThis->lenRcvBuf = -1;
    }

    /* now check if we have something in our buffer. If so, we satisfy
     * the request from buffer contents.
     */
    if(pThis->lenRcvBuf == -1) { /* no data present, must read */
        CHKiRet(gtlsRecordRecv(pThis));
    }

    if(pThis->lenRcvBuf == 0) { /* EOS */
        *pLenBuf = 0;
        /* in this case, we also need to free the receive buffer, if we
         * allocated one. -- rgerhards, 2008-12-03
         */
        if(pThis->pszRcvBuf != NULL) {
            free(pThis->pszRcvBuf);
            pThis->pszRcvBuf = NULL;
        }
        ABORT_FINALIZE(RS_RET_CLOSED);
    }

    /* if we reach this point, data is present in the buffer and must be 
copied */
    iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
    if(iBytesCopy > *pLenBuf) {
        iBytesCopy = *pLenBuf;
    } else {
        pThis->lenRcvBuf = -1; /* buffer will be emptied below */
    }

        dbgprintf("!!!!!!!!!!!%d!!!!!!!!!!!!!!\n\n", pThis->iAddClientCN);
        if (pThis->iAddClientCN)
        {
          if (pThis->clientCNValid != 1)
          {
            cert_list = gnutls_certificate_get_peers(pThis->sess, 
&cert_list_size);

        if(cert_list_size > 0)
            {
          // we only print information about the first certificate
          gnutls_x509_crt_init(&cert);
          gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER);

              CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));

              len = snprintf(NULL, 0, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
              if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char))) )
                return -1;

              snprintf(pThis->clientCN, len + 1, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
              pThis->clientCN[len] = '\0';               
              pThis->clientCNLen = len + 1;

              pThis->clientCNValid = 1;
            }
          }
       
          iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ? 
iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;

          buf_temp = (char*)malloc(iBytesCopy);
       
          if (buf_temp)
          {
            memset(buf_temp, 0, iBytesCopy);
            strncpy(buf_temp, pThis->clientCN, iBytesCopy);
            buf_temp[strlen(buf_temp)] ='\0';
            strncat(buf_temp, pThis->pszRcvBuf, iBytesCopy - 
strlen(buf_temp));
            buf_temp[strlen(buf_temp)] ='\0';
          }        
       
          memset(pBuf, 0, *pLenBuf);
      memcpy(pBuf, buf_temp + pThis->ptrRcvBuf, iBytesCopy);

          if (buf_temp)
            free(buf_temp);
        }
        else
        {
          memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, iBytesCopy);
    }

        pThis->ptrRcvBuf += iBytesCopy;
    *pLenBuf = iBytesCopy;

finalize_it:
    dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf 
%d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
    RETiRet;
}

The value is zero. Can you help me what I have to check in the sources code?

Thanks.

Regards,

Tomas


From sur5r at sur5r.net  Thu Jan 21 21:33:00 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Thu, 21 Jan 2010 21:33:00 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
Message-ID: <20100121213300.2abb07bf@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 20 Jan 2010 10:44:42 -0800 (PST)
david at lang.hm wrote:

> my thought is that just like we have a filter to change control characters 
> to escape sequences, it would be good to have a filter to escape non-ascii 
> characters. this will mangle other character sets, but they are unlikly to 
> go through cleanly anyway.

This is not an escaping issue, but an issue of byte sequences that are not
valid UTF8. That's why PostgreSQL rejects them. So we either need to make
ompgsql set SQL_ASCII as a client encoding (which will result in extended
characters being transcoded to UTF-8, which results in garbage) or make the
database SQL_ASCII.

Regards,
Jakob Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktYuf0ACgkQ1YAhDic+adY60QCbBqyEzDJtaEiWmg1cqKlMEJ2N
PnwAn2wAfPIpGlCOx2LdPJivrElU83Bu
=eTVw
-----END PGP SIGNATURE-----

From sur5r at sur5r.net  Thu Jan 21 22:26:26 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Thu, 21 Jan 2010 22:26:26 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<201001210149.48041.marc.schiffbauer@mightycare.de>
Message-ID: <20100121222626.083c7a49@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 21 Jan 2010 01:49:47 +0100
Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:

> Am Mittwoch, 20. Januar 2010 19:20:31 schrieb Jakob Haufe:
> > * Drop the message and report that we did so. That would be rather easy,
> >   but might not be what people want.
> > 
> 
> But this might be the best option I guess. Maybe the original message could 
> then be written to a special logfile on disk.

And then you have to check every now and then whether something ended up
there? That's not nice, and rather complex to implement as well (file name
should be configurable, maybe size limited, rotated, whatever)

> > For me, this sums up to one question:
> > 
> > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days, so it
> > would be bad if ompgsql could not keep up.
> 
> I think this is a special case because rsyslog is not the originator of
> those messages. It "just" transports them. And because the syslog-Protocol
> does not define something like encoding in any way the best thing to do is
> just leave those strings "as-is" and make the database behind it do so as
> well with SQL_ASCII.

I like the idea of seeing rsyslog as some kind of transport only. This is the
best argument for switching to SQL_ASCII altogether so far.

Rainer, do you have any thoughts on this?

> I thing everythign else will be error prone in some way. The Documentation
> of rsyslog should bring a big fat NOTE that the database must be SQL_ASCII
> as other wise thesrings might not be accepted.

Yes, and the createDB.sql for ompgsql should be changed as well.

Regards,
Jakob Haufe (sur5r)


- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
=1POP
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Fri Jan 22 10:51:41 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 10:51:41 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>

V5 has the capability to discard messages that cause an action failure.
However, this is mostly untested yet, AND the action must support it by
providing proper status information - it must differentiate between
system-induced errors (which can be retried) and message-induced errors
(which need the discard). ompgsql currently does not provide that status
information.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Wednesday, January 20, 2010 7:21 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Wed, 6 Jan 2010 16:14:59 +0100
> Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:
> 
> > which encoding should be chosen for the database when using postgres?
> 
> As far as I understand the syslog protocol (at least the legacy one),
> it has
> no concept of character encodings at all.  So if you simply want to
> make sure
> that everything ends up in the database "as is", then choose SQL_ASCII.
> 
> > My rsyslog version is 4.4.3.
> >
> > Which client_encoding does rsyslog use in ompgsql?
> 
> Right now, it does net set an encoding by itself, so the database
> default
> applies. If I'm not mistaken, you can even set that per user from
> inside of
> postgres. So I would rather vote against another configuration
> parameter here.
> 
> > I currently have set UTF-8 on the database. It worked for a while
> until
> > some special message arrived at the server where postgres denies the
> INSERT:
> >
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence
> for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also
> happen if
> > the byte sequence does not match the encoding expected by the server,
> which
> > is controlled by "client_encoding".
> 
> Were you able to isolate the message? Or find out which program was
> sending
> it?
> 
> > Now rsyslog is not able to log anything... it is currently spooling
> to disk
> > because it "hangs" at this message not being accepted by postgres.
> 
> This is bad, because if the machine is an open syslog server that
> simply
> collects everything it gets, we have a potential DoS vector here.
> 
> I can think of three options:
> 
> * Drop the message and report that we did so. That would be rather
> easy,
>   but might not be what people want.
> 
> * Re-insert the message after converting it from ASCII to UTF-8 or
> whatever
>   the DB encoding is. But this might/will produce garbage if the input
> is not
>   ASCII. It also creates more load on the system if these messages are
>   frequent. Guessing the input encoding is hard or even impossible,
> depending
>   on the set you guess from.
> 
> * Make the database SQL_ASCII. This will silently accept anything but
> will
>   create nonsense from UTF/UCS encoded messages. Also might create
> trouble
>   for programs like phplogcon that analyze the logs.
> 
> For me, this sums up to one question:
> 
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> non-UTF8
> strings? Everyone is trying to be UTF-8 clean these days, so it would
> be bad
> if ompgsql could not keep up.
> 
> Comments please.
> 
> Regards,
> Jakab Haufe (sur5r)
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktXSW8ACgkQ1YAhDic+adbqXACeIJcx6GW6PhSXFO1YF72PafJG
> 7t8AoLNwnJYMZ4bssqMZt/nkTIPWs0LI
> =vuWN
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 10:54:04 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 10:54:04 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>


> my thought is that just like we have a filter to change control
> characters
> to escape sequences, it would be good to have a filter to escape non-
> ascii
> characters. this will mangle other character sets, but they are unlikly
> to
> go through cleanly anyway.

Just to be on the right path, you suggest escaping charactes with hex values
> 7f?

Rainer


From david at lang.hm  Fri Jan 22 11:07:02 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 22 Jan 2010 02:07:02 -0800 (PST)
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>

On Fri, 22 Jan 2010, Rainer Gerhards wrote:

>> my thought is that just like we have a filter to change control
>> characters
>> to escape sequences, it would be good to have a filter to escape non-
>> ascii
>> characters. this will mangle other character sets, but they are unlikly
>> to
>> go through cleanly anyway.
>
> Just to be on the right path, you suggest escaping charactes with hex values
>> 7f?

correct. they can cause as much grief (or more) than control characters.

since control characters get escaped by default, rsyslog will already 
mangle UTF8 text sent to it if the final byte is in that range.

David Lang


From rgerhards at hq.adiscon.com  Fri Jan 22 11:09:52 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:09:52 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710372A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 11:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> >> my thought is that just like we have a filter to change control
> >> characters
> >> to escape sequences, it would be good to have a filter to escape
> non-
> >> ascii
> >> characters. this will mangle other character sets, but they are
> unlikly
> >> to
> >> go through cleanly anyway.
> >
> > Just to be on the right path, you suggest escaping charactes with hex
> values
> >> 7f?
> 
> correct. they can cause as much grief (or more) than control
> characters.
> 
> since control characters get escaped by default, rsyslog will already
> mangle UTF8 text sent to it if the final byte is in that range.

jup, just wanted to be sure. that can probably be best implemented as a
property replacer option (or at the parser level, but then it applies to
everything). Note that many European languages use these characters (and
without grief), much as Asian languages use sequences which would be
destroyed by the current escaping (which thus can be turned off). But I
definitely see the value. Given it looks easy to implement, I'll see if I can
integrate an option.

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 11:15:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:15:59 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems with
	character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>

Been on the road and will be over the weekend and part of next week (thus
sluggish responses ;)).

> > > For me, this sums up to one question:
> > >
> > > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days,
> so it
> > > would be bad if ompgsql could not keep up.
> >
> > I think this is a special case because rsyslog is not the originator
> of
> > those messages. It "just" transports them. And because the syslog-
> Protocol
> > does not define something like encoding in any way the best thing to
> do is
> > just leave those strings "as-is" and make the database behind it do
> so as
> > well with SQL_ASCII.
> 
> I like the idea of seeing rsyslog as some kind of transport only. This
> is the
> best argument for switching to SQL_ASCII altogether so far.
> 
> Rainer, do you have any thoughts on this?

Let me elaborte a bit: the new IETF syslog standards *do* specify character
encoding and strongly recommend Unicode (UTF-8) to be used. Of course, this
does not solve the issue with original senders that use another, unspecified,
coding. But it helps.

Unfortunately, rsyslog's "old" code is far from being Unicode-aware. As a
side-activity, I am upgrading "old" code to "new" code, which then uses
rsyslog's string classes. While they do not yet support Unicode, it is much
easier to make them support it once all string handling is done consistently.

However, even then I need to have a build time switch to turn this on/off,
because rsyslog in Unicode mode will take not only considerably more space
(especially with larger in-memory queues), it will also considerably affect
its performance (in terms of bytes, the memory transfer rate is effectively
cut in half, as most data in syslog is character-based - also think about the
effects on cache performance).

So moving the whole system to Unicode, while desirable, is far from being a
trivial task. Having seen extremely low demand for that, I have so far opted
to do this at a very low priorty (even though that means I violate RFC5424).
	
> 
> > I thing everythign else will be error prone in some way. The
> Documentation
> > of rsyslog should bring a big fat NOTE that the database must be
> SQL_ASCII
> > as other wise thesrings might not be accepted.
> 
> Yes, and the createDB.sql for ompgsql should be changed as well.
> 

The doc needs to be written so that I can add this warning ;) Is someone with
actual Postgres knowledge up for this task. Plain text is OK, I can then
copy&paste that into a module doc template.

As for createDB.sql: let me know what I need to change, and I'll apply that
change.

Rainer

> Regards,
> Jakob Haufe (sur5r)
> 
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
> m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
> =1POP
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 11:50:20 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:50:20 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>	<4B4DCF31.6090105@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
	<4B582AA7.3040906@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103735@GRFEXC.intern.adiscon.com>

mhhh... doesn't look too bad. Maybe it's a problem with the calling sequence.
When do you call your new function? If should be called after the
nsdConstruct but before the nsdConsructFinalize (actual function names may be
slightly different).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Thursday, January 21, 2010 11:21 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to add new configuration option
> 
> Rainer Gerhards wrote:
> > Thanks for the code. Unfortunately, adding the config switch to it is
> not
> > quite easy in that case (good I asked for the actual code). I'd say
> that you
> > best do it similar to the other config directives, like the
> authentication
> > mode. They actual directives are in the upper level code
> (imtcp/omfwd).
> > There, they are shuffled over to the instance data, which goes along
> with
> > each of the configured listeners/sender. Then, when a new network
> stream is
> > created, the params are passed down to the generic stream interface
> and there
> > passed down to the selected stream driver, which finally stores and
> acts on
> > them. It's clumpsy and quite some work, but that is what is needed
> for the
> > old config system. You probably need to add around 50 to 100 lines of
> code
> > altogether to the various files. It's not complex, but easy to forget
> > something. Best start by a directive (like $..AuthMode), see how it
> is
> > handled (and passed down) in imtcp and work your way down the stack
> ;)
> >
> > Rainer
> >
> >
> Hi Rainer,
> 
> I have added some code that I have thought was necessary, but
> I am stuck now. In nsd_gtls.c is added function:
> 
> static rsRetVal
> SetAddClientCN(nsd_t *pNsd, int mode)
> {
>     DEFiRet;
>     nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 
>     ISOBJ_TYPE_assert((pThis), nsd_gtls);
>     if(mode != 0 && mode != 1) {
>         errmsg.LogError(0, RS_RET_INVALID_DRVR_MODE, "error: driver
> mode
> %d not supported by "
>                 "gtls netstream driver", mode);
>         ABORT_FINALIZE(RS_RET_INVALID_DRVR_MODE);
>     }
> 
>     pThis->iAddClientCN = mode;
>         dbgprintf("GTLS:%d\n", pThis->iAddClientCN);
> finalize_it:
>     RETiRet;
> }
> 
> The "dbgprintf" shows correct value in pThis, but if I check
> pThis->iAddClientCN
> later in function:
> 
> static rsRetVal
> Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
> {
>     DEFiRet;
>     ssize_t iBytesCopy; /* how many bytes are to be copied to the
> client
> buffer? */
>     nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
>     ISOBJ_TYPE_assert(pThis, nsd_gtls);
> 
>         cstr_t *pstrCN = NULL;
>         const gnutls_datum *cert_list;
>     unsigned int cert_list_size = 0;
>         gnutls_x509_crt cert;
>         int len = 0;
>         char *buf_temp;
> 
>     if(pThis->bAbortConn)
>         ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);
> 
>     if(pThis->iMode == 0) {
>         CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
>         FINALIZE;
>     }
> 
>     /* --- in TLS mode now --- */
> 
>     /* Buffer logic applies only if we are in TLS mode. Here we
>      * assume that we will switch from plain to TLS, but never back.
> This
>      * assumption may be unsafe, but it is the model for the time being
> and I
>      * do not see any valid reason why we should switch back to plain
> TCP after
>      * we were in TLS mode. However, in that case we may lose something
> that
>      * is already in the receive buffer ... risk accepted. --
> rgerhards,
> 2008-06-23
>      */
> 
>     if(pThis->pszRcvBuf == NULL) {
>         /* we have no buffer, so we need to malloc one */
>         CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
>         pThis->lenRcvBuf = -1;
>     }
> 
>     /* now check if we have something in our buffer. If so, we satisfy
>      * the request from buffer contents.
>      */
>     if(pThis->lenRcvBuf == -1) { /* no data present, must read */
>         CHKiRet(gtlsRecordRecv(pThis));
>     }
> 
>     if(pThis->lenRcvBuf == 0) { /* EOS */
>         *pLenBuf = 0;
>         /* in this case, we also need to free the receive buffer, if we
>          * allocated one. -- rgerhards, 2008-12-03
>          */
>         if(pThis->pszRcvBuf != NULL) {
>             free(pThis->pszRcvBuf);
>             pThis->pszRcvBuf = NULL;
>         }
>         ABORT_FINALIZE(RS_RET_CLOSED);
>     }
> 
>     /* if we reach this point, data is present in the buffer and must
> be
> copied */
>     iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
>     if(iBytesCopy > *pLenBuf) {
>         iBytesCopy = *pLenBuf;
>     } else {
>         pThis->lenRcvBuf = -1; /* buffer will be emptied below */
>     }
> 
>         dbgprintf("!!!!!!!!!!!%d!!!!!!!!!!!!!!\n\n", pThis-
> >iAddClientCN);
>         if (pThis->iAddClientCN)
>         {
>           if (pThis->clientCNValid != 1)
>           {
>             cert_list = gnutls_certificate_get_peers(pThis->sess,
> &cert_list_size);
> 
>         if(cert_list_size > 0)
>             {
>           // we only print information about the first certificate
>           gnutls_x509_crt_init(&cert);
>           gnutls_x509_crt_import(cert, &cert_list[0],
> GNUTLS_X509_FMT_DER);
> 
>               CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));
> 
>               len = snprintf(NULL, 0, "CN:%s ",
> (char*)cstrGetSzStr(pstrCN));
>               if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char)))
> )
>                 return -1;
> 
>               snprintf(pThis->clientCN, len + 1, "CN:%s ",
> (char*)cstrGetSzStr(pstrCN));
>               pThis->clientCN[len] = '\0';
>               pThis->clientCNLen = len + 1;
> 
>               pThis->clientCNValid = 1;
>             }
>           }
> 
>           iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ?
> iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;
> 
>           buf_temp = (char*)malloc(iBytesCopy);
> 
>           if (buf_temp)
>           {
>             memset(buf_temp, 0, iBytesCopy);
>             strncpy(buf_temp, pThis->clientCN, iBytesCopy);
>             buf_temp[strlen(buf_temp)] ='\0';
>             strncat(buf_temp, pThis->pszRcvBuf, iBytesCopy -
> strlen(buf_temp));
>             buf_temp[strlen(buf_temp)] ='\0';
>           }
> 
>           memset(pBuf, 0, *pLenBuf);
>       memcpy(pBuf, buf_temp + pThis->ptrRcvBuf, iBytesCopy);
> 
>           if (buf_temp)
>             free(buf_temp);
>         }
>         else
>         {
>           memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf,
> iBytesCopy);
>     }
> 
>         pThis->ptrRcvBuf += iBytesCopy;
>     *pLenBuf = iBytesCopy;
> 
> finalize_it:
>     dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf
> %d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
>     RETiRet;
> }
> 
> The value is zero. Can you help me what I have to check in the sources
> code?
> 
> Thanks.
> 
> Regards,
> 
> Tomas
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From marc.schiffbauer at mightycare.de  Fri Jan 22 15:20:01 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Fri, 22 Jan 2010 15:20:01 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
	<9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
Message-ID: <201001221520.02432.marc.schiffbauer@mightycare.de>

Am Freitag, 22. Januar 2010 11:15:59 schrieb Rainer Gerhards:
> 
> The doc needs to be written so that I can add this warning ;) Is someone
> with actual Postgres knowledge up for this task. Plain text is OK, I can
> then copy&paste that into a module doc template.
> 
> As for createDB.sql: let me know what I need to change, and I'll apply that
> change.
> 

I can write it. I will send it to you/this thread next week!

Have a nice weekend
-Marc


From rgerhards at hq.adiscon.com  Fri Jan 22 17:09:21 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 17:09:21 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problemswith
	character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<201001221520.02432.marc.schiffbauer@mightycare.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710373A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer
> Sent: Friday, January 22, 2010 3:20 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problemswith character encoding
> 
> Am Freitag, 22. Januar 2010 11:15:59 schrieb Rainer Gerhards:
> >
> > The doc needs to be written so that I can add this warning ;) Is
> someone
> > with actual Postgres knowledge up for this task. Plain text is OK, I
> can
> > then copy&paste that into a module doc template.
> >
> > As for createDB.sql: let me know what I need to change, and I'll
> apply that
> > change.
> >
> 
> I can write it. I will send it to you/this thread next week!

excellent! looking forward to it :)

Rainer


From david at lang.hm  Fri Jan 22 19:19:25 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 22 Jan 2010 10:19:25 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
 with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
	<9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>

On Fri, 22 Jan 2010, Rainer Gerhards wrote:

> However, even then I need to have a build time switch to turn this on/off,
> because rsyslog in Unicode mode will take not only considerably more space
> (especially with larger in-memory queues), it will also considerably affect
> its performance (in terms of bytes, the memory transfer rate is effectively
> cut in half, as most data in syslog is character-based - also think about the
> effects on cache performance).

if the code uses UTF-8 throughout this doesn't make sense. assuming the 
input is plain ascii, UTF-8 strings and ASCII strings should be the same 
size (there is some additional cpu cycles involved to figure out the 
length in characters for any output routines that grab substrings, but 
that should be all)

the only way things would take double the space (and therefor halve the 
memory transfer rate) is if it converts everything to UTF-16 strings 
internally. This is a bad idea to start with as UTF-16 does not handle all 
characters (which is why there is UTF-32 as well), but also because UTF-16 
is significantly more expensive to store/copy/etc than UTF-8 for the 
common case where most of the characters are ASCII.

It may be that you have picked the wrong string library to use. prior to 
UTF-8 being defined 'unicode' and UTF-16 were basicly synonomous and a 
_lot_ of string libraries have been written with this assumption 
(converting everything to UTF-16 on input and to whatever on output). If 
you can find one that can handle the strings as UTF-8 internally it should 
be able to just about eliminate the overhead.

David Lang


From sur5r at sur5r.net  Sun Jan 24 20:14:41 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Sun, 24 Jan 2010 20:14:41 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>
Message-ID: <20100124201441.7990c850@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 22 Jan 2010 10:51:41 +0100
"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:

> V5 has the capability to discard messages that cause an action failure.
> However, this is mostly untested yet, AND the action must support it by
> providing proper status information - it must differentiate between
> system-induced errors (which can be retried) and message-induced errors
> (which need the discard). ompgsql currently does not provide that status
> information.

If you can point me at some example code or docs on how to do this, I would
like to try and add this functionality to ompgsql. Does ommysql already
implement that?

Regards,
Jakob Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktcnCEACgkQ1YAhDic+ada9WACeMkawcNTL/lt5E70mWeVjd38G
ARoAn1OAkEqm7NXRMwwVzUDC3B/2TeCB
=eDPw
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Mon Jan 25 08:34:53 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 08:34:53 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>
	<20100124201441.7990c850@samsa>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710373F@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Sunday, January 24, 2010 8:15 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Fri, 22 Jan 2010 10:51:41 +0100
> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:
> 
> > V5 has the capability to discard messages that cause an action
> failure.
> > However, this is mostly untested yet, AND the action must support it
> by
> > providing proper status information - it must differentiate between
> > system-induced errors (which can be retried) and message-induced
> errors
> > (which need the discard). ompgsql currently does not provide that
> status
> > information.
> 
> If you can point me at some example code or docs on how to do this, I
> would
> like to try and add this functionality to ompgsql. Does ommysql already
> implement that?


It's pretty new functionality and there is not yet a good example plugin that
uses it (it makes most sense for database plugins, where I have limited
knowledge). It would be useful to read this first (unfortunately not an easy
read:

http://download.rsyslog.com/design.pdf

Actually implementing it is rather easy. The core point is that for
system-induced errors (those that can be retried) the plugin must return
RS_RET_SUSPENDED and for message-induced errors it must return an "real"
error state (like RS_RET_ERR, but it would be better, and I'd be glad to
include, more precise error codes). The core engine than knows what to do.
Well, the core may have undiscovered bugs right now, as this functionality
was never before used in practice. It is very criticial to think about which
error class a failure belongs to. Messages with message-induced errors are
simply thrown away, so one needs to think twice before assigning this class -
but on the contrary if such a message is flagged as system-induced, it will
block the system, just as you can currently see...

HTH
Rainer

> 
> Regards,
> Jakob Haufe (sur5r)
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktcnCEACgkQ1YAhDic+ada9WACeMkawcNTL/lt5E70mWeVjd38G
> ARoAn1OAkEqm7NXRMwwVzUDC3B/2TeCB
> =eDPw
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Jan 25 09:12:08 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 09:12:08 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 7:19 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> > However, even then I need to have a build time switch to turn this
> on/off,
> > because rsyslog in Unicode mode will take not only considerably more
> space
> > (especially with larger in-memory queues), it will also considerably
> affect
> > its performance (in terms of bytes, the memory transfer rate is
> effectively
> > cut in half, as most data in syslog is character-based - also think
> about the
> > effects on cache performance).

David,

we need to make a distinction between UTF, a transformation (and transfer)
format and UCS, the actual native encoding format here. I think you mix these
two things up. Unicode has two (primary) flavors, which are usually encoded
in UCS-16 and UCS-32 (or ws it named UCS-2 and UCS-4 - guess so), being 2 and
4 bytes respectively. UCS-16 is what is implemented for example in Windows.
It covers many of this worlds scripts, but has proven to not cover all, which
caused additional code tables and UCS-32 presentation (at least as far as I
know, I am not an Unicode expert ;)).

UTF-8 is an encoding of Unicode code tables. You can think of it as
traditional multi-byte character set which means each character takes up a
varying number of bytes. Usually, UTF representations are converted into UCS
and then UCS is used to do the processing. While UCS requires more bytes, UTF
requires parsing of the message *each time* it is processed (e.g. to check
for a string match, count character sizes, obtain a substring). So using UTF
may use up fewer bytes, but can very considerably increase processing time
need and program complexity. For US-ASCII, of course, this is no problem. But
for other encodings, the performance hit can be very sever, much more than
the hit by double memory consumption (UCS-2 is still being considered as
"sufficient" for almost all cases, even in the future).

So I don't think it would serve the non-US-ASCII world well to process the
transformation formats. I guess that's a good option if you have a US-ASCII
based system that only very occasionally needs to process a foreign language
string (and even then, you need to parse the message *each* time you access
it, specifically when obtaining substrings...).

My conclusion is that rsyslog needs to do a UTF to UCS conversion on entry to
the system and then uses UCS internally (and converts back when messages are
output). Many software systems do so, and, as I said, IMHO do so for good
reasons.

Rainer

> 
> if the code uses UTF-8 throughout this doesn't make sense. assuming the
> input is plain ascii, UTF-8 strings and ASCII strings should be the
> same
> size (there is some additional cpu cycles involved to figure out the
> length in characters for any output routines that grab substrings, but
> that should be all)
> 
> the only way things would take double the space (and therefor halve the
> memory transfer rate) is if it converts everything to UTF-16 strings
> internally. This is a bad idea to start with as UTF-16 does not handle
> all
> characters (which is why there is UTF-32 as well), but also because
> UTF-16
> is significantly more expensive to store/copy/etc than UTF-8 for the
> common case where most of the characters are ASCII.
> 
> It may be that you have picked the wrong string library to use. prior
> to
> UTF-8 being defined 'unicode' and UTF-16 were basicly synonomous and a
> _lot_ of string libraries have been written with this assumption
> (converting everything to UTF-16 on input and to whatever on output).
> If
> you can find one that can handle the strings as UTF-8 internally it
> should
> be able to just about eliminate the overhead.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Mon Jan 25 09:42:32 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 25 Jan 2010 00:42:32 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
 Problems	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001250020090.3093@asgard.lang.hm>

On Mon, 25 Jan 2010, Rainer Gerhards wrote:

> David,
>
> we need to make a distinction between UTF, a transformation (and transfer)
> format and UCS, the actual native encoding format here. I think you mix these
> two things up. Unicode has two (primary) flavors, which are usually encoded
> in UCS-16 and UCS-32 (or ws it named UCS-2 and UCS-4 - guess so), being 2 and
> 4 bytes respectively. UCS-16 is what is implemented for example in Windows.
> It covers many of this worlds scripts, but has proven to not cover all, which
> caused additional code tables and UCS-32 presentation (at least as far as I
> know, I am not an Unicode expert ;)).
>
> UTF-8 is an encoding of Unicode code tables. You can think of it as
> traditional multi-byte character set which means each character takes up a
> varying number of bytes. Usually, UTF representations are converted into UCS
> and then UCS is used to do the processing. While UCS requires more bytes, UTF
> requires parsing of the message *each time* it is processed (e.g. to check
> for a string match, count character sizes, obtain a substring). So using UTF
> may use up fewer bytes, but can very considerably increase processing time
> need and program complexity. For US-ASCII, of course, this is no problem. But
> for other encodings, the performance hit can be very sever, much more than
> the hit by double memory consumption (UCS-2 is still being considered as
> "sufficient" for almost all cases, even in the future).

thanks for the clarification on terms. I had the basic understanding, but 
not the exact terminology.

> So I don't think it would serve the non-US-ASCII world well to process the
> transformation formats. I guess that's a good option if you have a US-ASCII
> based system that only very occasionally needs to process a foreign language
> string (and even then, you need to parse the message *each* time you access
> it, specifically when obtaining substrings...).
>
> My conclusion is that rsyslog needs to do a UTF to UCS conversion on entry to
> the system and then uses UCS internally (and converts back when messages are
> output). Many software systems do so, and, as I said, IMHO do so for good
> reasons.

the question is how many different places/times are we parsing the data as 
strings, vs how many places are we just moving the data around as 
essentially opaque blobs.

when we receive and parse the message we have to deal with the data as 
strings of characters, but this is generally done in one pass through the 
input data, so it would be about the same to process the data as-is as to 
convert it to UCS-2 (let alone then processing it as UCS-2). This pass can 
calculate the number of characters in the string (i.e. 'length') and store 
it

then these parsed chunks of data get copied around (in complex 
configurations with many queues, they get copied around a LOT).

At some point (or points) comparisons are made, but in most cases these 
comparisons can be done byte-by-byte, you don't actually have to parse the 
data (for regex matches you do, and for contains you would have to check 
the byte prior to the start of the match to make sure that that first 
matching byte isn't the tail end of a prior character, but I think that's 
it)

and then eventually we create the output string. At that point we are 
assembling the string from the various substrings that we have stored 
(which still can be treated as a series of bytes). It's only when the 
property replacer is invoked with either character positions or options 
that the data needs to be treated as a UTF-8 string instead of a series of 
bytes again. Yes there are a lot of things that it can do, but how much 
are they used in real life (other than setting a max length, which could 
be special cased to not be checked if the number of bytes is less than 
the length you are checking against)?

Remember that this is not general-purpose input and output that we are 
dealing with, it's logs. And like it or not, most logs really are in 
ASCII, simply because for so many years there was no option.

Also consider that the input and output stages can be split into multiple 
worker threads, while the queue manipulation (and copying) is done inside 
locks.

It may be best to leave the data as UTF-8 unless the property replacer has 
been given options, and then let the property replacer convert the data, 
work on it, and convert it back (if there is more than one option being 
invoked)

David Lang


From zhengfeng at cn.fujitsu.com  Mon Jan 25 11:36:47 2010
From: zhengfeng at cn.fujitsu.com (zhengfeng)
Date: Mon, 25 Jan 2010 18:36:47 +0800
Subject: [rsyslog] help: what induced syslogd test results are so fluctuated?
Message-ID: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>

Hi~,all

For test the performance of syslogd in RHEL5.4GA, I had written one small program by calling the interfaces syslog()...

But the results is not very steady.
Please look the results below:

The 1st time, some days ago, I test 10 times, every time sending logs to syslogd 30 secs, after then reboot.

Results:
      1,110  1,101  1,103  1,092  1,088  1,101  1,098  1,096  1,087  1,087  2.12% 



But Today , I use the same codes and method to test , the results are:

      1,295  1,292  1,297  1,291  1,288  1,287  1,284  1,279  1,275  1,270  2.13% 


(1290-1100)/1100 > 10%

Why the resluts upwards are so different? What induced? And how can i avoid that?

Thanks a lot.:-D  

From david at lang.hm  Mon Jan 25 14:10:44 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 25 Jan 2010 05:10:44 -0800 (PST)
Subject: [rsyslog] help: what induced syslogd test results are so
 fluctuated?
In-Reply-To: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>
References: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>
Message-ID: <alpine.DEB.2.00.1001250500200.3093@asgard.lang.hm>

On Mon, 25 Jan 2010, zhengfeng wrote:

> Hi~,all
>
> For test the performance of syslogd in RHEL5.4GA, I had written one small program by calling the interfaces syslog()...
>
> But the results is not very steady.
> Please look the results below:
>
> The 1st time, some days ago, I test 10 times, every time sending logs to syslogd 30 secs, after then reboot.
>
> Results:
>      1,110  1,101  1,103  1,092  1,088  1,101  1,098  1,096  1,087  1,087  2.12%
>
>
>
> But Today , I use the same codes and method to test , the results are:
>
>      1,295  1,292  1,297  1,291  1,288  1,287  1,284  1,279  1,275  1,270  2.13%
>
>
> (1290-1100)/1100 > 10%
>
> Why the resluts upwards are so different? What induced? And how can i avoid that?

there are a lot of things that could be causing this. However you didn't 
give us enough information to figure it out.

what else is running on the system?

what are the specs of the system? (is it a single core single processor, 
or do you have multiple processors)

what filesystem are you using?

when you say that you write logs for 30 seconds and reboot, are you 
allowing rsyslog to flush out pending writes, or are you loosing all logs 
that haven't been written yet (this will also involve what version of 
rsyslog are you testing and how do you have it configured)

how much cpu is rsyslog using during the time that it is running the test? 
(total time, and if you have multiple cpu cores on the system, the peak 
cpu of individual threads)

how large is the queue that rsyslog is allowed to use?

if the numbers you are reporting are the total logs written, they seem 
very low. on a current rsyslog with reasonable hardware I would expect the 
numbers to be tens of thousands of log messages per second, it may be that 
the bottleneck is the process writing to syslog() rather than rsyslog 
itself.

30 seconds is a very short time for a test, depending on your filesystem 
it may not have written anything out to the disk by the time you finish 
the test.


David Lang


From rgerhards at hq.adiscon.com  Mon Jan 25 14:38:30 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 14:38:30 +0100
Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog 5.3.6
	(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>

Hi Michael,

finally, good news: I finally managed to reproduce the problem under 32-bit
Debian sid. It looks like only v5 is affected, and not the quite similar
v4-beta. I will now try to pinpoint the problem (hoping that the repro is
stable).

Will post more news when I have it.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, January 17, 2010 12:52 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>> -----Original Message-----
> >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >>> Sent: Friday, January 15, 2010 11:57 PM
> >>> To: rsyslog-users
> >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>>
> >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>> > Michael,
> >>> >
> >>> > Fix now in git, links at the bug tracker:
> >>> >
> >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >>> >
> >>> > Please let me know if it works for you (the patch is a bit
> trickier
> >>> than it
> >>> > looks, so confirmations would be good).
> >>>
> >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> >>> But now I'm getting a crash when rsyslog encounters the xconsole
> pipe
> >>> config.
> >>
> >> I am a bit puzzled, but will try to reproduce that on my Debian box.
> I assume
> >> stock Debian config?
> >
> > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> > got the crash. I use the default rsyslog.conf from the official
> debian
> > package.
> 
> As an additonal hint: If I start xconsole (a process reading from
> /dev/xconsole) before I start rsyslogd, then the crash does not occur.
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Tue Jan 26 05:18:04 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Tue, 26 Jan 2010 04:18:04 +0000
Subject: [rsyslog] [patch]: fix from-host dns name reporting,
 add microseconds to MySQL
In-Reply-To: <mailman.112.1264479218.2883.rsyslog@lists.adiscon.com>
References: <mailman.112.1264479218.2883.rsyslog@lists.adiscon.com>
Message-ID: <4B5E6CFC.7070303@p6m7g8.com>

re-send from subscribed address

> ------------------------------------------------------------------------
> 
> Subject:
> [patch]: fix from-host dns name reporting, add microseconds to MySQL
> From:
> "Philip M. Gollucci" <pgollucci at ridecharge.com>
> Date:
> Mon, 25 Jan 2010 20:04:10 -0800
> To:
> rsyslog-users <rsyslog at lists.adiscon.com>
> 
> To:
> rsyslog-users <rsyslog at lists.adiscon.com>
> CC:
> "cristianorolim at hotmail.com" <cristianorolim at hotmail.com>
> 
> 
> Hi,
> 
> I have the following local patches running on a patched 5.3.6 on 50+ 
> FreeBSD machines at $work.
> 
> 1) I wanted the FQDN for $from-host
>     yes, I have this var set $PreserveFQDN to on
> 
> 2) I *need* microseconds int time:::*
> 3) Optionally add an #ifdef for the _PATH_MODDIR
>     to get the right default for fbsd
> 
> Maybe someone can explain to me why getting the host name is so complex, 
> it shouldn't be.
> 
> You can fetch them here --
> ASF mirror
> 1)
> http://people.freebsd.org/~pgollucci/patch-runtime__datetime.c
> http://people.freebsd.org/~pgollucci/patch-runtime__msg.c
> 2,3)
> http://people.freebsd.org/~pgollucci/patch-tools__syslogd.c
> 
> FreeBSD mirror
> 1)
> http://people.apache.org/~pgollucci/patch-runtime__datetime.c
> 2,3)
> http://people.apache.org/~pgollucci/patch-runtime__msg.c
> http://people.apache.org/~pgollucci/patch-tools__syslogd.c
> 
> 
> 
> 


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From rgerhards at hq.adiscon.com  Tue Jan 26 16:44:57 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 16:44:57 +0100
Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
	5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>

An update: this bug is *not* related to debian, but requires certain compiler
settings. I now also get it on Fedora. Also, it has nothing to do with the
named pipe. In fact, there seems to be a problem with the way direct queues
are handled. I don't have full details yet, but finally I begin to understand
the issue. It is a v5-only bug, introduced by the new queue engine. Direct
queues (at least action queues) can cause a segfault, at least if something
goes wrong in the action. Will post more details and/or a fix when I have
better info.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, January 25, 2010 2:39 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
> 5.3.6(v5-beta) released
> 
> Hi Michael,
> 
> finally, good news: I finally managed to reproduce the problem under
> 32-bit
> Debian sid. It looks like only v5 is affected, and not the quite
> similar
> v4-beta. I will now try to pinpoint the problem (hoping that the repro
> is
> stable).
> 
> Will post more news when I have it.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > Sent: Sunday, January 17, 2010 12:52 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >
> > 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > >>> -----Original Message-----
> > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > >>> Sent: Friday, January 15, 2010 11:57 PM
> > >>> To: rsyslog-users
> > >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > >>>
> > >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > >>> > Michael,
> > >>> >
> > >>> > Fix now in git, links at the bug tracker:
> > >>> >
> > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> > >>> >
> > >>> > Please let me know if it works for you (the patch is a bit
> > trickier
> > >>> than it
> > >>> > looks, so confirmations would be good).
> > >>>
> > >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of
> 5.3.6.
> > >>> But now I'm getting a crash when rsyslog encounters the xconsole
> > pipe
> > >>> config.
> > >>
> > >> I am a bit puzzled, but will try to reproduce that on my Debian
> box.
> > I assume
> > >> stock Debian config?
> > >
> > > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and
> then
> > > got the crash. I use the default rsyslog.conf from the official
> > debian
> > > package.
> >
> > As an additonal hint: If I start xconsole (a process reading from
> > /dev/xconsole) before I start rsyslogd, then the crash does not
> occur.
> >
> >
> >
> > --
> > Why is it that all of the instruments seeking intelligent life in the
> > universe are pointed away from Earth?
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 26 17:48:19 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 17:48:19 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid - was:RE:
	rsyslog5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>

OK, once the problematic spot is found, a fix is not far away...

The (very small) patch is self-explanatory, please see:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bab3ee566c883ac88df369ec
32df0c9100f97343

I've run it through a couple of tests now, and both theory and practice seem
to agree that this was the bug. 

Michael,
I'd appreciate if you could check if this solves the issue for you as well.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, January 26, 2010 4:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog v5 crash on Debian sid - was:RE:
> rsyslog5.3.6(v5-beta) released
> 
> An update: this bug is *not* related to debian, but requires certain
> compiler
> settings. I now also get it on Fedora. Also, it has nothing to do with
> the
> named pipe. In fact, there seems to be a problem with the way direct
> queues
> are handled. I don't have full details yet, but finally I begin to
> understand
> the issue. It is a v5-only bug, introduced by the new queue engine.
> Direct
> queues (at least action queues) can cause a segfault, at least if
> something
> goes wrong in the action. Will post more details and/or a fix when I
> have
> better info.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Monday, January 25, 2010 2:39 PM
> > To: rsyslog-users
> > Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
> > 5.3.6(v5-beta) released
> >
> > Hi Michael,
> >
> > finally, good news: I finally managed to reproduce the problem under
> > 32-bit
> > Debian sid. It looks like only v5 is affected, and not the quite
> > similar
> > v4-beta. I will now try to pinpoint the problem (hoping that the
> repro
> > is
> > stable).
> >
> > Will post more news when I have it.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > > Sent: Sunday, January 17, 2010 12:52 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > >
> > > 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > > > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > >>> -----Original Message-----
> > > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > > >>> Sent: Friday, January 15, 2010 11:57 PM
> > > >>> To: rsyslog-users
> > > >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > > >>>
> > > >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > >>> > Michael,
> > > >>> >
> > > >>> > Fix now in git, links at the bug tracker:
> > > >>> >
> > > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> > > >>> >
> > > >>> > Please let me know if it works for you (the patch is a bit
> > > trickier
> > > >>> than it
> > > >>> > looks, so confirmations would be good).
> > > >>>
> > > >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of
> > 5.3.6.
> > > >>> But now I'm getting a crash when rsyslog encounters the
> xconsole
> > > pipe
> > > >>> config.
> > > >>
> > > >> I am a bit puzzled, but will try to reproduce that on my Debian
> > box.
> > > I assume
> > > >> stock Debian config?
> > > >
> > > > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > > > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and
> > then
> > > > got the crash. I use the default rsyslog.conf from the official
> > > debian
> > > > package.
> > >
> > > As an additonal hint: If I start xconsole (a process reading from
> > > /dev/xconsole) before I start rsyslogd, then the crash does not
> > occur.
> > >
> > >
> > >
> > > --
> > > Why is it that all of the instruments seeking intelligent life in
> the
> > > universe are pointed away from Earth?
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Tue Jan 26 18:57:10 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 26 Jan 2010 18:57:10 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid - was:RE:
	rsyslog5.3.6(v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001260957r6d7cae7gc2ad4f21e3955fc2@mail.gmail.com>

010/1/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> OK, once the problematic spot is found, a fix is not far away...
>
> The (very small) patch is self-explanatory, please see:
>
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bab3ee566c883ac88df369ec
> 32df0c9100f97343
>
> I've run it through a couple of tests now, and both theory and practice seem
> to agree that this was the bug.
>
> Michael,
> I'd appreciate if you could check if this solves the issue for you as well.

Looks like you nailed the bug.

I can no longer reproduce the crash with the above patch applied.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Tue Jan 26 18:59:07 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 18:59:07 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid -
	was:RE:rsyslog5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
	<aa76cdac1001260957r6d7cae7gc2ad4f21e3955fc2@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103769@GRFEXC.intern.adiscon.com>

> > Michael,
> > I'd appreciate if you could check if this solves the issue for you as
> well.
> 
> Looks like you nailed the bug.
> 
> I can no longer reproduce the crash with the above patch applied.

Excellent. I am going through some minor things which may be useful to fix,
but that probably means we'll have a re-release soon :)

Rainer


From rgerhards at hq.adiscon.com  Wed Jan 27 07:28:02 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 07:28:02 +0100
Subject: [rsyslog] Tools to detect stack Adressing Problems?
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710376D@GRFEXC.intern.adiscon.com>

Hi all,

since I have begun to use the valgrind memory debugger routinely in
development (some two years ago), the quality of the source has much
increased. Unfortunately, however, valgrind is not able to detect problems
related to misaddressing variables on the stack. The 5.3.6 bug I was hunting
for almost a week is a good example of this. Valgrind also provides only
limited support for global data, as far as I know (and see from testing
results).

This becomes an even more important restriction as I moved a lot of former
heap memory use to the stack for performance reasons. I remember at least one
more major bug hunting effort that was hard to find because it affected only
stack space.

So I am currently looking for tools that could complement valgrind by
providing good stack checking capabilities. As one tool, mudflap was
suggested to me. It sounds interesting, but gives me a very hard time [very
hard to read debug output (no symbolic names for dlloade'ed modules, (false?)
reports for areas where I can not see anything wrong as well as frequent
(threading-related?) crashes when running under instrumentation). Maybe I am
just misinterpreting the output...

In short: I would highly appreciate suggestions for tools that can help with
debugging stack memory access (global data would be a plus) - and/or
instructions on how to interpret mudflap, if that is considered to be *the*
tool for that use case.

Thanks,
Rainer


From janfrode at tanso.net  Wed Jan 27 13:42:38 2010
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Wed, 27 Jan 2010 13:42:38 +0100
Subject: [rsyslog] filtering postfix/smtpd
Message-ID: <20100127124238.GA25239@janfrode.ibm.com>

I'm drowning in logs from postfix/smtpd, and need to filter these
messages out to a separate file. The maillog looks something like:

Jan 27 13:34:02 asav5.example.net postfix/lmtp[31977]::  53843908E2: to=<me at example.com>, relay=127.0.0.1[127.0.0.1]:10020, delay=0.54, delays=0.03/0.33/0.01/0.49, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 249FB906AD)
Jan 27 13:34:02 asav3.example.net postfix/smtpd[12077]::  connect from 26.81-111-54.customer.example.net[21.111.54.26]
Jan 27 13:34:02 asav5.example.net postfix/qmgr[32165]::  53843908E2: removed
Jan 27 13:34:02 asav3.mro.example.net postfix/smtpd[12077]::  disconnect from 26.81-111-54.customer.example.net[21.111.54.26]

So I want to separate out the lines from "postfix/smtpd" to
its own file, and not touch the postfix/lmtp or postfix/qmgr
or whatever-lines.

>From the documentation it seems to me that I should be able
to use:

	:programname,   isequal,        "postfix/smtpd" -?HourlyMaillogNonSplunked;MaillogTemplate
	:programname,   isequal,        "postfix/smtpd"        ~

But these doesn't match anything. If I use simply "postfix",
it matched all "postfix/*" messages:

	:programname,   isequal,        "postfix" -?HourlyMaillogNonSplunked;MaillogTemplate
	:programname,   isequal,        "postfix"        ~

So, any idea for how I can match just "postfix/smtpd" ?


  -jf



From rgerhards at hq.adiscon.com  Wed Jan 27 14:40:37 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 14:40:37 +0100
Subject: [rsyslog] filtering postfix/smtpd
References: <20100127124238.GA25239@janfrode.ibm.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103776@GRFEXC.intern.adiscon.com>

Hi,

could you run it in debug mode and post the relevant part of a log message
being processed? I guess that %programname% gets some weird value...

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Wednesday, January 27, 2010 1:43 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] filtering postfix/smtpd
> 
> I'm drowning in logs from postfix/smtpd, and need to filter these
> messages out to a separate file. The maillog looks something like:
> 
> Jan 27 13:34:02 asav5.example.net postfix/lmtp[31977]::  53843908E2:
> to=<me at example.com>, relay=127.0.0.1[127.0.0.1]:10020, delay=0.54,
> delays=0.03/0.33/0.01/0.49, dsn=2.0.0, status=sent (250 2.0.0 Ok:
> queued as 249FB906AD)
> Jan 27 13:34:02 asav3.example.net postfix/smtpd[12077]::  connect from
> 26.81-111-54.customer.example.net[21.111.54.26]
> Jan 27 13:34:02 asav5.example.net postfix/qmgr[32165]::  53843908E2:
> removed
> Jan 27 13:34:02 asav3.mro.example.net postfix/smtpd[12077]::
> disconnect from 26.81-111-54.customer.example.net[21.111.54.26]
> 
> So I want to separate out the lines from "postfix/smtpd" to
> its own file, and not touch the postfix/lmtp or postfix/qmgr
> or whatever-lines.
> 
> >From the documentation it seems to me that I should be able
> to use:
> 
> 	:programname,   isequal,        "postfix/smtpd" -
> ?HourlyMaillogNonSplunked;MaillogTemplate
> 	:programname,   isequal,        "postfix/smtpd"        ~
> 
> But these doesn't match anything. If I use simply "postfix",
> it matched all "postfix/*" messages:
> 
> 	:programname,   isequal,        "postfix" -
> ?HourlyMaillogNonSplunked;MaillogTemplate
> 	:programname,   isequal,        "postfix"        ~
> 
> So, any idea for how I can match just "postfix/smtpd" ?
> 
> 
>   -jf
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 27 15:27:51 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 15:27:51 +0100
Subject: [rsyslog] 8Bit character escaping - was: RE: PostgreSQL: Problems
	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103778@GRFEXC.intern.adiscon.com>

David,

I have now added the functionality to escape 8-bit characters. Patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=85045270f69e4dcb25c409c9
661e96e3172d7f30

I hope it is useful.

I plan to release a new v5 devel soon, probably tomorrow or friday.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 11:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> >> my thought is that just like we have a filter to change control
> >> characters
> >> to escape sequences, it would be good to have a filter to escape
> non-
> >> ascii
> >> characters. this will mangle other character sets, but they are
> unlikly
> >> to
> >> go through cleanly anyway.
> >
> > Just to be on the right path, you suggest escaping charactes with hex
> values
> >> 7f?
> 
> correct. they can cause as much grief (or more) than control
> characters.
> 
> since control characters get escaped by default, rsyslog will already
> mangle UTF8 text sent to it if the final byte is in that range.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Wed Jan 27 20:16:36 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Wed, 27 Jan 2010 19:16:36 +0000
Subject: [rsyslog] config file help
Message-ID: <4B609114.9090103@p6m7g8.com>

rsyslog.conf:

...
if $facility == '1' && $priority == '7' then ~
*.* :ommysql:localhost,logs,logs,logs;db_std ## not actual l/p
EOF

select facility,priority, count(1) as c
from syslogs
where facility = 1
and priority = 7
group by facility,priority;
+----------+----------+------+
| facility | priority | c    |
+----------+----------+------+
|        1 |        7 | 1637 |
+----------+----------+------+
1 row in set (0.00 sec)

am I missing something ? I just want to throw it away.



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From joe at joetify.com  Wed Jan 27 20:16:57 2010
From: joe at joetify.com (Joe Williams)
Date: Wed, 27 Jan 2010 11:16:57 -0800
Subject: [rsyslog] tripling of log lines
Message-ID: <4B609129.6040301@joetify.com>


I have an odd issue where with a specific config I see triple of each 
line in the log but using another config that should effectively be 
doing the same thing it does not.

Doing something like the following produces three identical lines in the 
log.

$template DbFormat,"%timegenerated% %HOSTNAME% %msg:::drop-last-lf%\n"
$template DbNotice,"<%= @log_dir 
%>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
local2.notice      -?DbNotice;DbFormat

Example:

Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0


The following produces the expected one line in the log without duplication.

$template DbFormat,"%timegenerated% %fromhost% %msg:::drop-last-lf%\n"
$template DbNotice,"<%= @log_dir 
%>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"

if \
   ( $syslogfacility-text == 'local2' ) \
     and \
   ( $syslogseverity-text == 'notice' ) \
then -?DbNotice;DbFormat


For brevity in both examples I just showed an example for one severity 
level, we have individual log templates and filters for all of them.

Any ideas what could be going on here? To me these should be equivalent.

Thanks.

-Joe

-- 
Name: Joseph A. Williams
Email: joe at joetify.com
Blog: http://www.joeandmotorboat.com/



From rgerhards at hq.adiscon.com  Wed Jan 27 21:34:48 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 21:34:48 +0100
Subject: [rsyslog] tripling of log lines
References: <4B609129.6040301@joetify.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103783@GRFEXC.intern.adiscon.com>

Maybe you don't discard the message after writing it?

Please see:

http://cookbook.rsyslog.com/node7.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Joe Williams
> Sent: Wednesday, January 27, 2010 8:17 PM
> To: rsyslog-users
> Subject: [rsyslog] tripling of log lines
> 
> 
> I have an odd issue where with a specific config I see triple of each
> line in the log but using another config that should effectively be
> doing the same thing it does not.
> 
> Doing something like the following produces three identical lines in
> the
> log.
> 
> $template DbFormat,"%timegenerated% %HOSTNAME% %msg:::drop-last-lf%\n"
> $template DbNotice,"<%= @log_dir
> %>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
> local2.notice      -?DbNotice;DbFormat
> 
> Example:
> 
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> 
> 
> The following produces the expected one line in the log without
> duplication.
> 
> $template DbFormat,"%timegenerated% %fromhost% %msg:::drop-last-lf%\n"
> $template DbNotice,"<%= @log_dir
> %>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
> 
> if \
>    ( $syslogfacility-text == 'local2' ) \
>      and \
>    ( $syslogseverity-text == 'notice' ) \
> then -?DbNotice;DbFormat
> 
> 
> For brevity in both examples I just showed an example for one severity
> level, we have individual log templates and filters for all of them.
> 
> Any ideas what could be going on here? To me these should be
> equivalent.
> 
> Thanks.
> 
> -Joe
> 
> --
> Name: Joseph A. Williams
> Email: joe at joetify.com
> Blog: http://www.joeandmotorboat.com/
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 27 22:26:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 22:26:24 +0100
Subject: [rsyslog] config file help
References: <4B609114.9090103@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Wednesday, January 27, 2010 8:17 PM
> To: rsyslog-users
> Subject: [rsyslog] config file help
> 
> rsyslog.conf:
> 
> ...
> if $facility == '1' && $priority == '7' then ~

I don't have the code at hand right now, but I guess the codes must be
numeric:

if $facility == 1 && $priority == 7 then ~

The scripting engine may not spit out a meaningful error message - it is in
its infancy with no time til today to complete it...

Rainer

> *.* :ommysql:localhost,logs,logs,logs;db_std ## not actual l/p
> EOF
> 
> select facility,priority, count(1) as c
> from syslogs
> where facility = 1
> and priority = 7
> group by facility,priority;
> +----------+----------+------+
> | facility | priority | c    |
> +----------+----------+------+
> |        1 |        7 | 1637 |
> +----------+----------+------+
> 1 row in set (0.00 sec)
> 
> am I missing something ? I just want to throw it away.
> 
> 
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Consultant,                       P6M7G8 Inc.
> Sr. System Admin,                 Ridecharge Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Wed Jan 27 22:59:00 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Wed, 27 Jan 2010 21:59:00 +0000
Subject: [rsyslog] config file help
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
Message-ID: <4B60B724.8060506@p6m7g8.com>

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
>> Sent: Wednesday, January 27, 2010 8:17 PM
>> To: rsyslog-users
>> Subject: [rsyslog] config file help
>>
>> rsyslog.conf:
>>
>> ...
>> if $facility == '1' && $priority == '7' then ~
> 
> I don't have the code at hand right now, but I guess the codes must be
> numeric:
> 
> if $facility == 1 && $priority == 7 then ~

Ha, you think I didn't try that too.  No dice either way.

Forget meaningful, it spits out nothing [with debugging and/or ktracing]
Just merely goes along and 'works' too well.


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From pgollucci at p6m7g8.com  Thu Jan 28 02:35:45 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Thu, 28 Jan 2010 01:35:45 +0000
Subject: [rsyslog] config file help
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
Message-ID: <4B60E9F1.6000800@p6m7g8.com>

Rainer Gerhards wrote:
> if $facility == 1 && $priority == 7 then ~
looking up the text values in includes/syslog.h does  work

user.debug ~

but

1.7 ~

does not.

-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From jbondc at openmv.com  Thu Jan 28 03:32:13 2010
From: jbondc at openmv.com (Jonathan Bond-Caron)
Date: Wed, 27 Jan 2010 21:32:13 -0500
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
	Problems	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
Message-ID: <000f01ca9fc2$1a574840$4f05d8c0$@com>

On Mon Jan 25 03:12 AM, Rainer Gerhards wrote:
> So I don't think it would serve the non-US-ASCII world well to process 
> the transformation formats. I guess that's a good option if you have a 
> US-ASCII based system that only very occasionally needs to process a 
> foreign language string (and even then, you need to parse the message
> *each* time you access it, specifically when obtaining substrings...).
> 
> My conclusion is that rsyslog needs to do a UTF to UCS conversion on 
> entry to the system and then uses UCS internally (and converts back 
> when messages are output). Many software systems do so, and, as I 
> said, IMHO do so for good reasons.
> 

What about adding a property option ~ 'normalize-utf8' where invalid utf8
bytes would be escaped?

$template dbFormat,"insert into text_logs (utf8_message) values
('%msg:::normalize-utf8%')",stdsql

I can probably dig through postgresql to find the code to detect invalid
utf8 bytes.

I'm not sure if I understood but are you suggesting that all input to
rsyslog is converted to UCS internally?
That seems like a huge performance penalty to pay when most people (?) log
US-ascii or UTF-8 data.




From david at lang.hm  Thu Jan 28 06:32:07 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 27 Jan 2010 21:32:07 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE:
 PostgreSQL:	Problems	with character encoding
In-Reply-To: <000f01ca9fc2$1a574840$4f05d8c0$@com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
	<000f01ca9fc2$1a574840$4f05d8c0$@com>
Message-ID: <alpine.DEB.2.00.1001272129130.28980@asgard.lang.hm>

On Wed, 27 Jan 2010, Jonathan Bond-Caron wrote:

> On Mon Jan 25 03:12 AM, Rainer Gerhards wrote:
>> So I don't think it would serve the non-US-ASCII world well to process
>> the transformation formats. I guess that's a good option if you have a
>> US-ASCII based system that only very occasionally needs to process a
>> foreign language string (and even then, you need to parse the message
>> *each* time you access it, specifically when obtaining substrings...).
>>
>> My conclusion is that rsyslog needs to do a UTF to UCS conversion on
>> entry to the system and then uses UCS internally (and converts back
>> when messages are output). Many software systems do so, and, as I
>> said, IMHO do so for good reasons.
>>
>
> What about adding a property option ~ 'normalize-utf8' where invalid utf8
> bytes would be escaped?
>
> $template dbFormat,"insert into text_logs (utf8_message) values
> ('%msg:::normalize-utf8%')",stdsql
>
> I can probably dig through postgresql to find the code to detect invalid
> utf8 bytes.

Rainer just added a property option to escape characters > 127. you could 
probably take that patch and basicly clone it to make a version that only 
escapes things if they aren't valid UTF8 instead.

> I'm not sure if I understood but are you suggesting that all input to
> rsyslog is converted to UCS internally?
> That seems like a huge performance penalty to pay when most people (?) log
> US-ascii or UTF-8 data.

right now rsyslog doesn't do any unicode stuff, it treats everything as a 
string of bytes (with some code to escape specific characters). He is 
saying that the path he has been planning to take would convert everything 
to UCS internally. you saw my argument against that.

David Lang


From rgerhards at hq.adiscon.com  Thu Jan 28 08:52:00 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 08:52:00 +0100
Subject: [rsyslog] config file help
References: <4B609114.9090103@p6m7g8.com><9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60E9F1.6000800@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103788@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Thursday, January 28, 2010 2:36 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] config file help
> 
> Rainer Gerhards wrote:
> > if $facility == 1 && $priority == 7 then ~
> looking up the text values in includes/syslog.h does  work
> 
> user.debug ~
> 
> but
> 
> 1.7 ~

These kind of filters are a different beast (and the traditional ones).
Rsyslog has three types of filters:

- the traditional ones
- property based
- script bases

Functionality increases on the way down, but also performance decreases.
Filters evolved, so each class has the syntax that best fits it. Note that
the if statement above and the traditional filter user.debug is *very*
different when looking from the executed code. User.debug is *much* faster
than starting up the script logic for the same thing. Should have mentioned
that yesterday...

Rainer

> 
> does not.
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Consultant,                       P6M7G8 Inc.
> Sr. System Admin,                 Ridecharge Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Jan 28 09:04:40 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 09:04:40 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE:
	PostgreSQL:	Problems	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com><000f01ca9fc2$1a574840$4f05d8c0$@com>
	<alpine.DEB.2.00.1001272129130.28980@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710378A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 28, 2010 6:32 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problems with character encoding
> 
> On Wed, 27 Jan 2010, Jonathan Bond-Caron wrote:
> > I'm not sure if I understood but are you suggesting that all input to
> > rsyslog is converted to UCS internally?
> > That seems like a huge performance penalty to pay when most people
> (?) log
> > US-ascii or UTF-8 data.
> 
> right now rsyslog doesn't do any unicode stuff, it treats everything as
> a
> string of bytes (with some code to escape specific characters). He is
> saying that the path he has been planning to take would convert
> everything
> to UCS internally. you saw my argument against that.

I didn't yet respond to the original message because David's argument is a
good one and I did not yet have time to think it over. Please note that there
are many subtle issues, especially when combining it with the demands of the
relevant RFCs (and if I implement it, I will definitely take a path that is
standards-compliant). David's argument and proposed solutions sounds good to
me, though I have some long-term concerns (eg. Can we really expect that
Japanese/Chinese systems always use US-ASCII for the core logging information
- I do not truly believe in that...).

However, I simply have no time to implement Unicode right now, so what I most
probably will do is copy over this valuable discussion and arguments into the
design doc, so that I have them ready at hand when I can turn into that
direction.

But in general, I now tend to agree to David's argument and think that it can
probably even speed up the process of a full Unicode implementation.

Rainer


From tbergfeld at hq.adiscon.com  Thu Jan 28 09:11:08 2010
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Thu, 28 Jan 2010 09:11:08 +0100
Subject: [rsyslog] rsyslog 5.3.7 (v5-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710378B@GRFEXC.intern.adiscon.com>

Hi all,

We have released a new v5-beta, version 5.3.7. Most importantly, it contains
the fixes for the problem with named pipes that Michael Biebl discovered.
There are also some other fixes (see changelog for detail). No new
functionality is included.

Once again, this is scheduled to become the new v5-stable, if no further
issues exist. As such, we would appreciate if you could try out the version
and report back your experience (even if everything works).
See Changelog for more details.

ChangeLog:

http://www.rsyslog.com/Article437.phtml

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-192.phtml


As always, feedback is appreciated.

Best regards,
Tom Bergfeld
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html .
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Jan 28 18:38:55 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 18:38:55 +0100
Subject: [rsyslog] config file help
In-Reply-To: <4B60B724.8060506@p6m7g8.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60B724.8060506@p6m7g8.com>
Message-ID: <1264700335.11821.2.camel@localhost>

On Wed, 2010-01-27 at 22:59 +0100, Philip M. Gollucci wrote:
> Rainer Gerhards wrote:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> >> Sent: Wednesday, January 27, 2010 8:17 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] config file help
> >>
> >> rsyslog.conf:
> >>
> >> ...
> >> if $facility == '1' && $priority == '7' then ~
> >
> > I don't have the code at hand right now, but I guess the codes must
> be
> > numeric:
> >
> > if $facility == 1 && $priority == 7 then ~
> 
> Ha, you think I didn't try that too.  No dice either way.
> 
> Forget meaningful, it spits out nothing [with debugging and/or
> ktracing]
> Just merely goes along and 'works' too well.

I just tried it out. I got the following error message:

===
2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
expression [try http://www.rsyslog.com/e/2051 ]
2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
$priority == 7 then ~"
2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
without actions will be discarded
2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
could not interpret master config file
'/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]
===

Not too precise (as expected), but far from not existent ;)

The syntax error is &&, you need to use "and". Also, the property names
were incorrect. So the correct line would have been:

if $syslogfacility == 1 and $syslogseverity == 7 then ~

While I have verified that this line works, you are far better of
(performance-wise) with the traditional priority filter that you now
use.

Rainer



From pgollucci at p6m7g8.com  Thu Jan 28 18:41:41 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Thu, 28 Jan 2010 17:41:41 +0000
Subject: [rsyslog] config file help
In-Reply-To: <1264700335.11821.2.camel@localhost>
References: <4B609114.9090103@p6m7g8.com>	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>	<4B60B724.8060506@p6m7g8.com>
	<1264700335.11821.2.camel@localhost>
Message-ID: <4B61CC55.1030106@p6m7g8.com>

> I just tried it out. I got the following error message:
> 
> ===
> 2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
> expression [try http://www.rsyslog.com/e/2051 ]
> 2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
> in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
> $priority == 7 then ~"
> 2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
> without actions will be discarded
> 2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
> could not interpret master config file
> '/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]

well thats useful at least.  I wonder why I don't see it.

> The syntax error is &&, you need to use "and". Also, the property names
> were incorrect. So the correct line would have been:
> 
> if $syslogfacility == 1 and $syslogseverity == 7 then ~
d'oh

> 
> While I have verified that this line works, you are far better of
> (performance-wise) with the traditional priority filter that you now
> use.
Yes!

Thx!


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From rgerhards at hq.adiscon.com  Thu Jan 28 18:44:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 18:44:24 +0100
Subject: [rsyslog] config file help
In-Reply-To: <4B61CC55.1030106@p6m7g8.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60B724.8060506@p6m7g8.com> <1264700335.11821.2.camel@localhost>
	<4B61CC55.1030106@p6m7g8.com>
Message-ID: <1264700664.11821.4.camel@localhost>

On Thu, 2010-01-28 at 17:41 +0000, Philip M. Gollucci wrote:
> > I just tried it out. I got the following error message:
> > 
> > ===
> > 2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
> > expression [try http://www.rsyslog.com/e/2051 ]
> > 2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
> > in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
> > $priority == 7 then ~"
> > 2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
> > without actions will be discarded
> > 2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
> > could not interpret master config file
> > '/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]
> 
> well thats useful at least.  I wonder why I don't see it.
> 

well, I guess that's the ole question on alternatives to using the
logging system itself to log error messages... The mailing list has a
couple of posts on this, one I thread I think in December or early this
month. I guess you did not capture syslog messages themselves.

Rainer

> > The syntax error is &&, you need to use "and". Also, the property names
> > were incorrect. So the correct line would have been:
> > 
> > if $syslogfacility == 1 and $syslogseverity == 7 then ~
> d'oh
> 
> > 
> > While I have verified that this line works, you are far better of
> > (performance-wise) with the traditional priority filter that you now
> > use.
> Yes!
> 
> Thx!
> 
> 




From david at lang.hm  Fri Jan 29 04:21:00 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 28 Jan 2010 19:21:00 -0800 (PST)
Subject: [rsyslog] no v5.3.7 announcement?
Message-ID: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>

I see it in git, I even see an announcement on freshmeat, but I didn't see 
an announcement that it was released here ;-)

for those who have missed it, 5.3.7 includes a couple fixes that were 
discussed here over the last couple of weeks.

David Lang


From rgerhards at hq.adiscon.com  Fri Jan 29 14:56:21 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jan 2010 14:56:21 +0100
Subject: [rsyslog] no v5.3.7 announcement?
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>

Mhhh... Tom sent it out yesterday, and I also see it in the archive:

http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html

Maybe we have some mail delivery problems...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 29, 2010 4:21 AM
> To: rsyslog-users
> Subject: [rsyslog] no v5.3.7 announcement?
> 
> I see it in git, I even see an announcement on freshmeat, but I didn't
> see
> an announcement that it was released here ;-)
> 
> for those who have missed it, 5.3.7 includes a couple fixes that were
> discussed here over the last couple of weeks.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ktm at rice.edu  Fri Jan 29 14:59:17 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Fri, 29 Jan 2010 07:59:17 -0600
Subject: [rsyslog] no v5.3.7 announcement?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
Message-ID: <20100129135917.GT1221@it.is.rice.edu>

I saw it here.

Ken
On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
> 
> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
> 
> Maybe we have some mail delivery problems...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Friday, January 29, 2010 4:21 AM
> > To: rsyslog-users
> > Subject: [rsyslog] no v5.3.7 announcement?
> > 
> > I see it in git, I even see an announcement on freshmeat, but I didn't
> > see
> > an announcement that it was released here ;-)
> > 
> > for those who have missed it, 5.3.7 includes a couple fixes that were
> > discussed here over the last couple of weeks.
> > 
> > David Lang
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Fri Jan 29 16:15:53 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 29 Jan 2010 07:15:53 -0800 (PST)
Subject: [rsyslog] no v5.3.7 announcement?
In-Reply-To: <20100129135917.GT1221@it.is.rice.edu>
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
	<20100129135917.GT1221@it.is.rice.edu>
Message-ID: <alpine.DEB.2.00.1001290715390.28980@asgard.lang.hm>

in that case, sorry for the noise.

David Lang

On Fri, 29 Jan 2010, Kenneth Marshall wrote:

> I saw it here.
>
> Ken
> On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
>> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
>>
>> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
>>
>> Maybe we have some mail delivery problems...
>>
>> Rainer
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Friday, January 29, 2010 4:21 AM
>>> To: rsyslog-users
>>> Subject: [rsyslog] no v5.3.7 announcement?
>>>
>>> I see it in git, I even see an announcement on freshmeat, but I didn't
>>> see
>>> an announcement that it was released here ;-)
>>>
>>> for those who have missed it, 5.3.7 includes a couple fixes that were
>>> discussed here over the last couple of weeks.
>>>
>>> David Lang
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Fri Jan 29 16:50:38 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jan 2010 16:50:38 +0100
Subject: [rsyslog] no v5.3.7 announcement?
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com><20100129135917.GT1221@it.is.rice.edu>
	<alpine.DEB.2.00.1001290715390.28980@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71037A3@GRFEXC.intern.adiscon.com>

no probalem at all - better twice than never. We had some problems with mail
delivery in december, and so I am always alerted if something in that
direction comes up...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 29, 2010 4:16 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] no v5.3.7 announcement?
> 
> in that case, sorry for the noise.
> 
> David Lang
> 
> On Fri, 29 Jan 2010, Kenneth Marshall wrote:
> 
> > I saw it here.
> >
> > Ken
> > On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
> >> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
> >>
> >> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
> >>
> >> Maybe we have some mail delivery problems...
> >>
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >>> Sent: Friday, January 29, 2010 4:21 AM
> >>> To: rsyslog-users
> >>> Subject: [rsyslog] no v5.3.7 announcement?
> >>>
> >>> I see it in git, I even see an announcement on freshmeat, but I
> didn't
> >>> see
> >>> an announcement that it was released here ;-)
> >>>
> >>> for those who have missed it, 5.3.7 includes a couple fixes that
> were
> >>> discussed here over the last couple of weeks.
> >>>
> >>> David Lang
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From gmanea-ma at lists.mszet.de  Sat Jan  2 11:43:51 2010
From: gmanea-ma at lists.mszet.de (Michael =?UTF-8?B?U3RyYXXDnw==?=)
Date: Sat, 2 Jan 2010 11:43:51 +0100
Subject: [rsyslog] rsyslog config-test croaks
Message-ID: <20100102114351.00f8f080@merkur.home.mszet.de>

Hello

I'm Michael Strau? and i am tying to use rsyslog.

I'm using debian Lenny width rsyslog 4.4.2 from backports.org.
I changed the debian standard configuration and adapted it for my needs.

It works in my opinion fully correctly. So I am impossible to
comprehend this error:

# rsyslogd -c4 -N1
rsyslogd: version 4.4.2, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: the last error occured in /etc/rsyslog.conf, line 25
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

These are the criticized Lines:

24: $ModLoad imrelp
25: $InputRELPServerRun 2514
26:

I can't detect a failure.

Best regards
Michael

-- 




From mbiebl at gmail.com  Sat Jan  2 15:41:05 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sat, 2 Jan 2010 15:41:05 +0100
Subject: [rsyslog] rsyslog config-test croaks
In-Reply-To: <20100102114351.00f8f080@merkur.home.mszet.de>
References: <20100102114351.00f8f080@merkur.home.mszet.de>
Message-ID: <aa76cdac1001020641r2be6200cnbfa29607a100e963@mail.gmail.com>

2010/1/2 Michael Strau? <gmanea-ma at lists.mszet.de>:
> Hello
>
> I'm Michael Strau? and i am tying to use rsyslog.
>
> I'm using debian Lenny width rsyslog 4.4.2 from backports.org.
> I changed the debian standard configuration and adapted it for my needs.
>
> It works in my opinion fully correctly. So I am impossible to
> comprehend this error:
>
> # rsyslogd -c4 -N1
> rsyslogd: version 4.4.2, config validation run (level 1), master config /etc/rsyslog.conf
> rsyslogd: the last error occured in /etc/rsyslog.conf, line 25
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> These are the criticized Lines:
>
> 24: $ModLoad imrelp
> 25: $InputRELPServerRun 2514
> 26:
>
> I can't detect a failure.

Have you installed the rsyslog-relp package which contains the imrelp module?

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From gmanea-ma at lists.mszet.de  Sat Jan  2 21:20:26 2010
From: gmanea-ma at lists.mszet.de (Michael =?UTF-8?B?U3RyYXXDnw==?=)
Date: Sat, 2 Jan 2010 21:20:26 +0100
Subject: [rsyslog] rsyslog config-test croaks
References: <20100102114351.00f8f080@merkur.home.mszet.de>
	<aa76cdac1001020641r2be6200cnbfa29607a100e963@mail.gmail.com>
Message-ID: <20100102212026.65b0fd56@merkur.home.mszet.de>

On Sat, 2 Jan 2010 15:41:05 +0100, Michael Biebl <mbiebl at gmail.com> wrote:

> 
> Have you installed the rsyslog-relp package which contains the imrelp module?
> 

Yes, and also it receives the messages from the client correctly.
This is merely a cosmetic problem.

Regards,
Michael

-- 




From ktm at rice.edu  Tue Jan  5 20:53:49 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Tue, 5 Jan 2010 13:53:49 -0600
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
Message-ID: <20100105195349.GO18110@it.is.rice.edu>

I am running rsyslog version 4.2.0 on a Redhat 5 machine
and noticed slow logins to the box. The strace on the login
sshd shows the following:

9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
9937       0.000019 connect(4, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be restarted)
5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 9844
5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [], SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted system call)
5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished ...>
9937       8.001608 <... sendto resumed> ) = 90
9937       0.000028 close(4)            = 0
9937       0.000039 read(6, "\0\0\5\36", 4) = 4
9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
9937       0.000104 close(6)            = 0
9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9938
9937       0.000032 alarm(0)            = 102
9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
...

The problem seems to be caused by writing to /dev/log which should
be being managed by the rsyslog program. I see a similar problem
reported earlier on the forum:

rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)

This was for version 3.18.4 but the symptom sounded very similar.
I restarted the rsyslog process and the login times returned to normal.
Let me know if there is something further I can do to help you debug
this matter.

Regards,
Ken


From david at lang.hm  Tue Jan  5 22:12:43 2010
From: david at lang.hm (david at lang.hm)
Date: Tue, 5 Jan 2010 13:12:43 -0800 (PST)
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
In-Reply-To: <20100105195349.GO18110@it.is.rice.edu>
References: <20100105195349.GO18110@it.is.rice.edu>
Message-ID: <alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>

this sounds like rsyslog is failing to send the logs out to the RELP 
server, and so is building up a large queue. restarting rsyslog would 
clear the queued up log messages and make it fast again.

David Lang


  On Tue, 5 Jan 2010, Kenneth Marshall wrote:

> Date: Tue, 5 Jan 2010 13:53:49 -0600
> From: Kenneth Marshall <ktm at rice.edu>
> Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> To: rsyslog at lists.adiscon.com
> Cc: sandmant at rice.edu
> Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
> 
> I am running rsyslog version 4.2.0 on a Redhat 5 machine
> and noticed slow logins to the box. The strace on the login
> sshd shows the following:
>
> 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> 9937       0.000019 connect(4, {sa_family=AF_FILE, path="/dev/log"...}, 110) = 0
> 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "..., 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be restarted)
> 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG, NULL) = 9844
> 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [], SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted system call)
> 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished ...>
> 9937       8.001608 <... sendto resumed> ) = 90
> 9937       0.000028 close(4)            = 0
> 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> 9937       0.000104 close(6)            = 0
> 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 9938
> 9937       0.000032 alarm(0)            = 102
> 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> ...
>
> The problem seems to be caused by writing to /dev/log which should
> be being managed by the rsyslog program. I see a similar problem
> reported earlier on the forum:
>
> rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
>
> This was for version 3.18.4 but the symptom sounded very similar.
> I restarted the rsyslog process and the login times returned to normal.
> Let me know if there is something further I can do to help you debug
> this matter.
>
> Regards,
> Ken
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From kenneho.ndu at gmail.com  Wed Jan  6 15:57:25 2010
From: kenneho.ndu at gmail.com (Kenneth Holter)
Date: Wed, 6 Jan 2010 15:57:25 +0100
Subject: [rsyslog] rsyslog+stunnel works only when running "rsyslogd"
	fromthe shell
In-Reply-To: <3A240503F9F2194780469F072D9A70541162FF2B@m342.silverspringnet.com>
References: <c25f25140912230012s5570db5r9c1959ab0f2d5b5e@mail.gmail.com>
	<3A240503F9F2194780469F072D9A70541162FF2B@m342.silverspringnet.com>
Message-ID: <c25f25141001060657p475097b8o44be587c2a1a1fb3@mail.gmail.com>

Yeah, it's really old, but for now I'll have to stick with it. :(


I added the "-d" option to rsyslog daemon, and came across this:

1098717504: Called fprintlog, logging to builtin-fwd 127.0.0.1:61514/tcp
1098717504: create tcp connection failed, reason Permission denied
1098717504: no working socket could be obtained
1098717504: error forwarding via tcp, suspending

Seems like the reason why it doesn't work is that it fails to create the TCP
session from itself (i.e. rsyslog) to the stunnel port. I've sent this
information to Red Hat support, but if anyone here have an ideas as to
what's causing this please do let me know.


- Kenneth

On Wed, Dec 23, 2009 at 9:59 PM, Siddhartha Jain
<sjain at silverspringnet.com>wrote:

> Kenneth,
>
> Not sure why RedHat/CentOS continue to bundle rsyslog 2.0.6. This
> version is ancient. Since 2.x, rsyslog has gone through 2.x, 4.x and now
> the current, 5.x.
>
> I would highly recommend rolling your own RPM from recent 5.x or 4.x
> code.
>
> - Siddhartha
>
>
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Kenneth Holter
> > Sent: Wednesday, December 23, 2009 12:13 AM
> > To: rsyslog at lists.adiscon.com
>  > Subject: [rsyslog] rsyslog+stunnel works only when running "rsyslogd"
> > fromthe shell
> >
> > Hi.
> >
> >
> > I'm running rsyslog v2.0.6 provided with my RHEL 5 installation. For
> > some
> > time now I've had rsyslog issues with some of my RHEL 5 servers, and
> > I've
> > not been able to figure out the problems, and would like to hear from
> > others
> > that may have experienced the same problem. I've been in contact with
> > Red
> > Hat support, but they've not been able to reproduce this problem, so
> > we'be
> > not succeeded in resolving the issue.
> >
> > First, let me describe my setup: My RHEL 5 servers have set up a TLS
> > tunnel
> > (using stunnel) between themselves and the log host. This works
> > perfectly.
> > I've configured rsyslog to forward messages to this tunnel by adding a
> > "
> > *.*  @@127.0.0.1:61514 " line to the bottom of /etc/rsyslog.conf file.
> > The
> > stunnel is listening on port 61514.
> >
> > On almost all my servers, this works as planned. But for some reason,
> a
> > few
> > servers are having problems forwarding messages to their stunnel
> > connection.
> > By running "tcpdump -i lo" I can see that these servers are not
> > transmitting
> > anything on the loopback interface, and are thus not forwarding
> > anything to
> > the stunnel port. One of my theories was that the line above simply
> > wasn't
> > picked up by rsyslog daemon. So I stopped the daemon, ran "rsyslogd
> -d"
> > to
> > view the debug output, and everthing works fine.
> >
> > For some reason, when I run rsyslog like this (i.e by issuing
> > "rsyslogd" in
> > the command prompt) instead of issuing "/etc/init.d/rsyslog start",
> > everything work fine. I'm really puzzled as to why this is so. Does
> > anyone
> > know why this is so? I have the exact same setup one all my servers,
> > but one
> > a small number of them have this problem.
> >
> >
> > Best regards,
> > Kenneth Holter
>  > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From marc.schiffbauer at mightycare.de  Wed Jan  6 16:14:59 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 16:14:59 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
Message-ID: <201001061615.00121.marc.schiffbauer@mightycare.de>

Hi all,

which encoding should be chosen for the database when using postgres?

My rsyslog version is 4.4.3.

Which client_encoding does rsyslog use in ompgsql?


I currently have set UTF-8 on the database. It worked for a while until some 
special message arrived at the server where postgres denies the INSERT:

2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for 
encoding "UTF8": 0xd220
2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if the 
byte sequence does not match the encoding expected by the server, which is 
controlled by "client_encoding".

Now rsyslog is not able to log anything... it is currently spooling to disk 
because it "hangs" at this message not being accepted by postgres.

Any hints?
TIA
-Marc



From marc.schiffbauer at mightycare.de  Wed Jan  6 16:48:02 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 16:48:02 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061615.00121.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
Message-ID: <201001061648.02984.marc.schiffbauer@mightycare.de>

Hi all again,

replying to myself because I think I found the solution:

With an db encoding of SQL_ASCII the postgres server will not do any character 
conversion which seems to be the right thing for syslog messages where the 
encoding cannot be determined reliably.

Maybe this is an important piece for the rsyslog documentation as well.

Now everthing is working again.

To convert my existing database I switch to user postgres and used "pg_dump -C 
syslog > syslog.sql" to dump the database. Then added a "DROP DATABASE syslog" 
before the "CREATE DATABASE", changed any encodings from "UTF-8" to 
"SQL_ASCII" (client_encoding and in the CREATE DATABASE statement) and then 
loaded the data again with "psql < syslog.sql".

-Marc



Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> Hi all,
> 
> which encoding should be chosen for the database when using postgres?
> 
> My rsyslog version is 4.4.3.
> 
> Which client_encoding does rsyslog use in ompgsql?
> 
> 
> I currently have set UTF-8 on the database. It worked for a while until
>  some special message arrived at the server where postgres denies the
>  INSERT:
> 
> 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> encoding "UTF8": 0xd220
> 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
>  the byte sequence does not match the encoding expected by the server,
>  which is controlled by "client_encoding".
> 
> Now rsyslog is not able to log anything... it is currently spooling to disk
> because it "hangs" at this message not being accepted by postgres.
> 
> Any hints?
> TIA
> -Marc
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Wed Jan  6 16:53:52 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Wed, 6 Jan 2010 09:53:52 -0600
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061648.02984.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
Message-ID: <20100106155352.GU18110@it.is.rice.edu>

Would it be possible to send the poorly behaving loggers to
a different port to allow it to be cleaned up properly? Using
SQL_ASCII does allow truly anything into the database, which
means that all the output pieces need to process it appropriately
too.

Regards,
Ken

On Wed, Jan 06, 2010 at 04:48:02PM +0100, Marc Schiffbauer wrote:
> Hi all again,
> 
> replying to myself because I think I found the solution:
> 
> With an db encoding of SQL_ASCII the postgres server will not do any character 
> conversion which seems to be the right thing for syslog messages where the 
> encoding cannot be determined reliably.
> 
> Maybe this is an important piece for the rsyslog documentation as well.
> 
> Now everthing is working again.
> 
> To convert my existing database I switch to user postgres and used "pg_dump -C 
> syslog > syslog.sql" to dump the database. Then added a "DROP DATABASE syslog" 
> before the "CREATE DATABASE", changed any encodings from "UTF-8" to 
> "SQL_ASCII" (client_encoding and in the CREATE DATABASE statement) and then 
> loaded the data again with "psql < syslog.sql".
> 
> -Marc
> 
> 
> 
> Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> > Hi all,
> > 
> > which encoding should be chosen for the database when using postgres?
> > 
> > My rsyslog version is 4.4.3.
> > 
> > Which client_encoding does rsyslog use in ompgsql?
> > 
> > 
> > I currently have set UTF-8 on the database. It worked for a while until
> >  some special message arrived at the server where postgres denies the
> >  INSERT:
> > 
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
> >  the byte sequence does not match the encoding expected by the server,
> >  which is controlled by "client_encoding".
> > 
> > Now rsyslog is not able to log anything... it is currently spooling to disk
> > because it "hangs" at this message not being accepted by postgres.
> > 
> > Any hints?
> > TIA
> > -Marc
> > 
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From marc.schiffbauer at mightycare.de  Wed Jan  6 17:32:43 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Wed, 6 Jan 2010 17:32:43 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <20100106155352.GU18110@it.is.rice.edu>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
	<20100106155352.GU18110@it.is.rice.edu>
Message-ID: <201001061732.44115.marc.schiffbauer@mightycare.de>

Am Mittwoch, 6. Januar 2010 16:53:52 schrieb Kenneth Marshall:
> Would it be possible to send the poorly behaving loggers to
> a different port to allow it to be cleaned up properly? 

No, not in that case I am afraid.

An option in rsyslog that would allow it to skip/trash/log-to-a-file those bad 
messages would be a nice thing.

> Using
> SQL_ASCII does allow truly anything into the database, which
> means that all the output pieces need to process it appropriately
> too.

Yes but this is working nicely here with phplogcon.

-Marc

> 
> Regards,
> Ken
> 
> On Wed, Jan 06, 2010 at 04:48:02PM +0100, Marc Schiffbauer wrote:
> > Hi all again,
> >
> > replying to myself because I think I found the solution:
> >
> > With an db encoding of SQL_ASCII the postgres server will not do any
> > character conversion which seems to be the right thing for syslog
> > messages where the encoding cannot be determined reliably.
> >
> > Maybe this is an important piece for the rsyslog documentation as well.
> >
> > Now everthing is working again.
> >
> > To convert my existing database I switch to user postgres and used
> > "pg_dump -C syslog > syslog.sql" to dump the database. Then added a "DROP
> > DATABASE syslog" before the "CREATE DATABASE", changed any encodings from
> > "UTF-8" to "SQL_ASCII" (client_encoding and in the CREATE DATABASE
> > statement) and then loaded the data again with "psql < syslog.sql".
> >
> > -Marc
> >
> > Am Mittwoch, 6. Januar 2010 16:14:59 schrieb Marc Schiffbauer:
> > > Hi all,
> > >
> > > which encoding should be chosen for the database when using postgres?
> > >
> > > My rsyslog version is 4.4.3.
> > >
> > > Which client_encoding does rsyslog use in ompgsql?
> > >
> > >
> > > I currently have set UTF-8 on the database. It worked for a while until
> > >  some special message arrived at the server where postgres denies the
> > >  INSERT:
> > >
> > > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > > encoding "UTF8": 0xd220
> > > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen
> > > if the byte sequence does not match the encoding expected by the
> > > server, which is controlled by "client_encoding".
> > >
> > > Now rsyslog is not able to log anything... it is currently spooling to
> > > disk because it "hangs" at this message not being accepted by postgres.
> > >
> > > Any hints?
> > > TIA
> > > -Marc
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Wed Jan  6 17:40:16 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Wed, 6 Jan 2010 10:40:16 -0600
Subject: [rsyslog] PostgreSQL: Problems with character encoding [Solved]
In-Reply-To: <201001061732.44115.marc.schiffbauer@mightycare.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<201001061648.02984.marc.schiffbauer@mightycare.de>
	<20100106155352.GU18110@it.is.rice.edu>
	<201001061732.44115.marc.schiffbauer@mightycare.de>
Message-ID: <20100106164016.GV18110@it.is.rice.edu>

On Wed, Jan 06, 2010 at 05:32:43PM +0100, Marc Schiffbauer wrote:
> Am Mittwoch, 6. Januar 2010 16:53:52 schrieb Kenneth Marshall:
> > Would it be possible to send the poorly behaving loggers to
> > a different port to allow it to be cleaned up properly? 
> 
> No, not in that case I am afraid.
> 
> An option in rsyslog that would allow it to skip/trash/log-to-a-file those bad 
> messages would be a nice thing.
> 
> > Using
> > SQL_ASCII does allow truly anything into the database, which
> > means that all the output pieces need to process it appropriately
> > too.
> 
> Yes but this is working nicely here with phplogcon.
> 
> -Marc
> 
I was more concerned about possible compromizes caused by the
ability to insert pretty arbitrary binary data into the system.
If we have this problem in the future, I will investigate other
options further. It might be possible to have the driver also
store them in a bad record table using such an option.

Cheers,
Ken


From a.smith at ukgrid.net  Thu Jan  7 15:53:36 2010
From: a.smith at ukgrid.net (Andy Smith)
Date: Thu, 07 Jan 2010 14:53:36 +0000
Subject: [rsyslog] help with config syntax
Message-ID: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>

Hi,

   Im having trouble getting the config setup how I need it. On a mail  
server I have a lot of data being written to the main messages file,  
thats because I have mail daemons writting data with a "notice"  
severity that is configured to be written to messages (so this is  
expected). How can I prevent just mail.notice going to the messages  
file while keeping all other *.notice stuff going there? I tried  
adding !mail.notice to the config for the messages file but this didnt  
seem to work...
Here is my config:

*.err;kern.warning;auth.notice;mail.crit;local7.none             
/dev/console;Tra
ditionalFormatWithPRI
mail.info;mail.notice                                    
-/var/log/maillog;Tradit
ionalFormatWithPRI
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local7.none       
  -/var/log/messages;TraditionalFormatWithPRI
security.*                                      /var/log/security
auth.info;authpriv.info                         /var/log/auth.log
lpr.info                                        /var/log/lpd-errs
ftp.info                                        /var/log/xferlog
cron.*                                          /var/log/cron
*.=debug;local7.none                                    /var/log/debug.log
*.emerg                                         *

thanks Andy.


From danson at rackspace.com  Thu Jan  7 19:43:54 2010
From: danson at rackspace.com (Daniel Anson)
Date: Thu, 7 Jan 2010 12:43:54 -0600
Subject: [rsyslog] RHEL5 rsyslog 4 rpms
Message-ID: <7616_1262890053_o07IlMci013462_8DFDF421C24C4B4883F75F4E81EF785627D32BEBD6@DFW1MXM01.RACKSPACE.CORP>

If anyone is interested, an RPM engineer I know has packaged RHEL5 rsyslog4 rpms.  These are available for public download and testing @ http://dl.iuscommunity.org/pub/ius  Any comments can be emailed directly to him at ius-coredev at lists.launchpad.net

rpms are regularly packaged by him so let him know what you think.  I believe you just have to add the yum repo.

--Daniel M. Anson
--Linux Systems Engineer



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is prohibited.
If you receive this transmission in error, please notify us immediately by e-mail
at abuse at rackspace.com, and delete the original message.
Your cooperation is appreciated.



From david at lang.hm  Fri Jan  8 17:41:33 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 8 Jan 2010 08:41:33 -0800 (PST)
Subject: [rsyslog] help with config syntax
In-Reply-To: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>
References: <20100107145336.583424mu03flx7j4@horde.ukgrid.net>
Message-ID: <alpine.DEB.2.00.1001080840040.3341@asgard.lang.hm>

On Thu, 7 Jan 2010, Andy Smith wrote:

> Hi,
>
>   Im having trouble getting the config setup how I need it. On a mail
> server I have a lot of data being written to the main messages file,
> thats because I have mail daemons writting data with a "notice"
> severity that is configured to be written to messages (so this is
> expected). How can I prevent just mail.notice going to the messages
> file while keeping all other *.notice stuff going there? I tried
> adding !mail.notice to the config for the messages file but this didnt
> seem to work...
> Here is my config:
>
> *.err;kern.warning;auth.notice;mail.crit;local7.none  /dev/console;TraditionalFormatWithPRI
> mail.info;mail.notice -/var/log/maillog;TraditionalFormatWithPRI

at this point you can tell it to drop the message by adding the line

& ~

this tells it to use the same matchine rules as the line above, and drop 
the message (don't process it in any further rules)

David Lang

> *.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err;local7.none
>  -/var/log/messages;TraditionalFormatWithPRI
> security.*                                      /var/log/security
> auth.info;authpriv.info                         /var/log/auth.log
> lpr.info                                        /var/log/lpd-errs
> ftp.info                                        /var/log/xferlog
> cron.*                                          /var/log/cron
> *.=debug;local7.none                                    /var/log/debug.log
> *.emerg                                         *
>
> thanks Andy.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Mon Jan 11 12:15:55 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jan 2010 12:15:55 +0100
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5 machine
References: <20100105195349.GO18110@it.is.rice.edu>
	<alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>

I think there is a patch (or a recommendation) regarding RELP in my mail
backlog. If I got it right, RELP does not necessarily detect a broken
connection, and thus no recovery action is initiated. I'll try to get to this
ASAP, but I am now the second day in office and there is still a pile of
things I need to look into ...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, January 05, 2010 10:13 PM
> To: rsyslog-users
> Cc: sandmant at rice.edu
> Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> 
> this sounds like rsyslog is failing to send the logs out to the RELP
> server, and so is building up a large queue. restarting rsyslog would
> clear the queued up log messages and make it fast again.
> 
> David Lang
> 
> 
>   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> 
> > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > From: Kenneth Marshall <ktm at rice.edu>
> > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > To: rsyslog at lists.adiscon.com
> > Cc: sandmant at rice.edu
> > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> >
> > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > and noticed slow logins to the box. The strace on the login
> > sshd shows the following:
> >
> > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> path="/dev/log"...}, 110) = 0
> > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be
> restarted)
> > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> 0}], WNOHANG, NULL) = 9844
> > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [],
> SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> system call)
> > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> ...>
> > 9937       8.001608 <... sendto resumed> ) = 90
> > 9937       0.000028 close(4)            = 0
> > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> > 9937       0.000104 close(6)            = 0
> > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> 0}], 0, NULL) = 9938
> > 9937       0.000032 alarm(0)            = 102
> > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [],
> SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > ...
> >
> > The problem seems to be caused by writing to /dev/log which should
> > be being managed by the rsyslog program. I see a similar problem
> > reported earlier on the forum:
> >
> > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> >
> > This was for version 3.18.4 but the symptom sounded very similar.
> > I restarted the rsyslog process and the login times returned to
> normal.
> > Let me know if there is something further I can do to help you debug
> > this matter.
> >
> > Regards,
> > Ken
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ktm at rice.edu  Mon Jan 11 14:52:19 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Mon, 11 Jan 2010 07:52:19 -0600
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5	machine
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
References: <20100105195349.GO18110@it.is.rice.edu>
	<alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
Message-ID: <20100111135218.GM1895@it.is.rice.edu>

It does seem to act like the RELP problem, but my use is only
with a regular TCP connection using @@logmachine. It had the
same symptom and restarting rsyslog cleared it up.

Regards,
Ken

On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote:
> I think there is a patch (or a recommendation) regarding RELP in my mail
> backlog. If I got it right, RELP does not necessarily detect a broken
> connection, and thus no recovery action is initiated. I'll try to get to this
> ASAP, but I am now the second day in office and there is still a pile of
> things I need to look into ...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Tuesday, January 05, 2010 10:13 PM
> > To: rsyslog-users
> > Cc: sandmant at rice.edu
> > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > machine
> > 
> > this sounds like rsyslog is failing to send the logs out to the RELP
> > server, and so is building up a large queue. restarting rsyslog would
> > clear the queued up log messages and make it fast again.
> > 
> > David Lang
> > 
> > 
> >   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> > 
> > > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > > From: Kenneth Marshall <ktm at rice.edu>
> > > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > > To: rsyslog at lists.adiscon.com
> > > Cc: sandmant at rice.edu
> > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > machine
> > >
> > > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > > and noticed slow logins to the box. The strace on the login
> > > sshd shows the following:
> > >
> > > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> > path="/dev/log"...}, 110) = 0
> > > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]: "...,
> > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To be
> > restarted)
> > > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > 0}], WNOHANG, NULL) = 9844
> > > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740, [],
> > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> > system call)
> > > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> > ...>
> > > 9937       8.001608 <... sendto resumed> ) = 90
> > > 9937       0.000028 close(4)            = 0
> > > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) = 1310
> > > 9937       0.000104 close(6)            = 0
> > > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > 0}], 0, NULL) = 9938
> > > 9937       0.000032 alarm(0)            = 102
> > > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0, [],
> > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > > ...
> > >
> > > The problem seems to be caused by writing to /dev/log which should
> > > be being managed by the rsyslog program. I see a similar problem
> > > reported earlier on the forum:
> > >
> > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> > >
> > > This was for version 3.18.4 but the symptom sounded very similar.
> > > I restarted the rsyslog process and the login times returned to
> > normal.
> > > Let me know if there is something further I can do to help you debug
> > > this matter.
> > >
> > > Regards,
> > > Ken
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From rgerhards at hq.adiscon.com  Mon Jan 11 16:39:01 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 Jan 2010 16:39:01 +0100
Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5	machine
References: <20100105195349.GO18110@it.is.rice.edu><alpine.DEB.2.00.1001051311490.24130@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710369A@GRFEXC.intern.adiscon.com>
	<20100111135218.GM1895@it.is.rice.edu>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036A0@GRFEXC.intern.adiscon.com>

A "problem" I am aware of is that a died peer (or connection dropped an
interim firewall) is not detected as broken, because no messages are
exchanged any longer. An often-used solution is KEEPALIVE, but this can also
take some time to timeout (and may have bad effects on slow connection or
those with outages of interim systems). I know that I wanted to implement the
capability to activate KEEPALIVE, but I am not sure if I found time to
actually do it. Will let you know once I can check that.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Kenneth Marshall
> Sent: Monday, January 11, 2010 2:52 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> machine
> 
> It does seem to act like the RELP problem, but my use is only
> with a regular TCP connection using @@logmachine. It had the
> same symptom and restarting rsyslog cleared it up.
> 
> Regards,
> Ken
> 
> On Mon, Jan 11, 2010 at 12:15:55PM +0100, Rainer Gerhards wrote:
> > I think there is a patch (or a recommendation) regarding RELP in my
> mail
> > backlog. If I got it right, RELP does not necessarily detect a broken
> > connection, and thus no recovery action is initiated. I'll try to get
> to this
> > ASAP, but I am now the second day in office and there is still a pile
> of
> > things I need to look into ...
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > > Sent: Tuesday, January 05, 2010 10:13 PM
> > > To: rsyslog-users
> > > Cc: sandmant at rice.edu
> > > Subject: Re: [rsyslog] rsyslog hang with imklog (/dev/log) on a
> RHEL5
> > > machine
> > >
> > > this sounds like rsyslog is failing to send the logs out to the
> RELP
> > > server, and so is building up a large queue. restarting rsyslog
> would
> > > clear the queued up log messages and make it fast again.
> > >
> > > David Lang
> > >
> > >
> > >   On Tue, 5 Jan 2010, Kenneth Marshall wrote:
> > >
> > > > Date: Tue, 5 Jan 2010 13:53:49 -0600
> > > > From: Kenneth Marshall <ktm at rice.edu>
> > > > Reply-To: rsyslog-users <rsyslog at lists.adiscon.com>
> > > > To: rsyslog at lists.adiscon.com
> > > > Cc: sandmant at rice.edu
> > > > Subject: [rsyslog] rsyslog hang with imklog (/dev/log) on a RHEL5
> > > machine
> > > >
> > > > I am running rsyslog version 4.2.0 on a Redhat 5 machine
> > > > and noticed slow logins to the box. The strace on the login
> > > > sshd shows the following:
> > > >
> > > > 9937       0.000045 socket(PF_FILE, SOCK_DGRAM, 0) = 4
> > > > 9937       0.000025 fcntl(4, F_SETFD, FD_CLOEXEC) = 0
> > > > 9937       0.000019 connect(4, {sa_family=AF_FILE,
> > > path="/dev/log"...}, 110) = 0
> > > > 9937       0.000040 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]:
> "...,
> > > 90, MSG_NOSIGNAL, NULL, 0) = ? ERESTARTSYS (To be restarted)
> > > > 9937       0.000042 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > > 9937       0.000018 sendto(4, "<86>Jan  5 13:36:12 sshd[9937]:
> "...,
> > > 90, MSG_NOSIGNAL, NULL, 0 <unfinished ...>
> > > > 5095       7.001495 <... select resumed> ) = ? ERESTARTNOHAND (To
> be
> > > restarted)
> > > > 5095       0.000040 --- SIGCHLD (Child exited) @ 0 (0) ---
> > > > 5095       0.000025 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) ==
> > > 0}], WNOHANG, NULL) = 9844
> > > > 5095       0.000055 wait4(-1, 0x7fffbf6d198c, WNOHANG, NULL) = 0
> > > > 5095       0.000021 rt_sigaction(SIGCHLD, NULL, {0x2ad5c3ab2740,
> [],
> > > SA_RESTORER, 0x2ad5c65922d0}, 8) = 0
> > > > 5095       0.000028 rt_sigreturn(0x11)  = -1 EINTR (Interrupted
> > > system call)
> > > > 5095       0.000027 select(7, [3 5], NULL, NULL, NULL <unfinished
> > > ...>
> > > > 9937       8.001608 <... sendto resumed> ) = 90
> > > > 9937       0.000028 close(4)            = 0
> > > > 9937       0.000039 read(6, "\0\0\5\36", 4) = 4
> > > > 9937       0.000037 read(6, "\31\0\0\0\24'\363w{\376B\364Ye
> > > !\365\232\216\220\352\343\"\262\334\0\0\0\20\0\0\0"..., 1310) =
> 1310
> > > > 9937       0.000104 close(6)            = 0
> > > > 9937       0.000029 mmap(NULL, 1310720, PROT_READ|PROT_WRITE,
> > > MAP_SHARED|MAP_ANONYMOUS, -1, 0) = 0x2ad8627d9000
> > > > 9937       0.000074 munmap(0x2ad85caed000, 65536) = 0
> > > > 9937       0.000037 wait4(9938, [{WIFEXITED(s) && WEXITSTATUS(s)
> ==
> > > 0}], 0, NULL) = 9938
> > > > 9937       0.000032 alarm(0)            = 102
> > > > 9937       0.000023 rt_sigaction(SIGALRM, NULL, {0x2ad85c8637a0,
> [],
> > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, 8) = 0
> > > > 9937       0.000029 rt_sigaction(SIGALRM, {SIG_DFL, [],
> > > SA_RESTORER|SA_INTERRUPT, 0x2ad85f3432d0}, NULL, 8) = 0
> > > > ...
> > > >
> > > > The problem seems to be caused by writing to /dev/log which
> should
> > > > be being managed by the rsyslog program. I see a similar problem
> > > > reported earlier on the forum:
> > > >
> > > > rsyslog hangs with imklog + omrelp (Same bug a imuxlog FC ?)
> > > >
> > > > This was for version 3.18.4 but the symptom sounded very similar.
> > > > I restarted the rsyslog process and the login times returned to
> > > normal.
> > > > Let me know if there is something further I can do to help you
> debug
> > > > this matter.
> > > >
> > > > Regards,
> > > > Ken
> > > > _______________________________________________
> > > > rsyslog mailing list
> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com
> > > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From dirk.schulz at kinzesberg.de  Mon Jan 11 18:23:18 2010
From: dirk.schulz at kinzesberg.de (Dirk H. Schulz)
Date: Mon, 11 Jan 2010 18:23:18 +0100
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
Message-ID: <4B4B5E86.9050409@kinzesberg.de>

Hi folks,

I am running two central logservers using rsyslog that several dozen 
servers report to (mostly also rsyslog).

The  central logservers are writing everything into a  database and 
additionally into local logfiles.

I would like to change configuration in a way that only local messages 
are written to local logfiles, and all messages (local and received from 
remote servers) into the database.

Is this possible with Rsyslog? I have searched the documentation, but 
did not find anything helpful.

Any hint or help is appreciated.

Dirk



From david at lang.hm  Mon Jan 11 19:42:20 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 11 Jan 2010 10:42:20 -0800 (PST)
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
In-Reply-To: <4B4B5E86.9050409@kinzesberg.de>
References: <4B4B5E86.9050409@kinzesberg.de>
Message-ID: <alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>

On Mon, 11 Jan 2010, Dirk H. Schulz wrote:

> Hi folks,
>
> I am running two central logservers using rsyslog that several dozen
> servers report to (mostly also rsyslog).
>
> The  central logservers are writing everything into a  database and
> additionally into local logfiles.
>
> I would like to change configuration in a way that only local messages
> are written to local logfiles, and all messages (local and received from
> remote servers) into the database.

yes, I do something similar to this on my systems.

All logs except local logs get written to local files, all local logs get 
sent over the network (at which point they then get picked up as remote 
logs), and all logs (local or remote) get sent to a remote system.

:fromhost, !isequal, "127.0.0.1"        /var/log/messages;TraditionalFormat
:fromhost, isequal, "127.0.0.1"         @192.168.1.8;TraditionalForwardFormat
*.*             @192.168.1.2





From paul.ruiz at gmail.com  Mon Jan 11 21:46:10 2010
From: paul.ruiz at gmail.com (Paul Ruiz)
Date: Mon, 11 Jan 2010 12:46:10 -0800
Subject: [rsyslog] Local Logging on Rsyslog Central Logserver
In-Reply-To: <alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>
References: <4B4B5E86.9050409@kinzesberg.de>
	<alpine.DEB.2.00.1001111037350.24130@asgard.lang.hm>
Message-ID: <f210e0b1001111246v56fe45acnf00c241f319148e4@mail.gmail.com>

I do this by running 2 rsyslog processes, one for local logs just like
all other installations and one that does only log collection.  The
log collection one has it's own init, config and pid file.  This way I
can rely on packaged config for local logging being identical in
production and a secondary package for log collection that only
includes the conf and init script depending on the standard rsyslog
package.

/usr/sbin/rsyslogd -c4 -f /etc/rsyslog-collector.conf -i
/var/run/rsyslogd-collector.pid


On Mon, Jan 11, 2010 at 10:42 AM,  <david at lang.hm> wrote:
> On Mon, 11 Jan 2010, Dirk H. Schulz wrote:
>
>> Hi folks,
>>
>> I am running two central logservers using rsyslog that several dozen
>> servers report to (mostly also rsyslog).
>>
>> The ?central logservers are writing everything into a ?database and
>> additionally into local logfiles.
>>
>> I would like to change configuration in a way that only local messages
>> are written to local logfiles, and all messages (local and received from
>> remote servers) into the database.
>
> yes, I do something similar to this on my systems.
>
> All logs except local logs get written to local files, all local logs get
> sent over the network (at which point they then get picked up as remote
> logs), and all logs (local or remote) get sent to a remote system.
>
> :fromhost, !isequal, "127.0.0.1" ? ? ? ?/var/log/messages;TraditionalFormat
> :fromhost, isequal, "127.0.0.1" ? ? ? ? @192.168.1.8;TraditionalForwardFormat
> *.* ? ? ? ? ? ? @192.168.1.2
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From sepperlot at googlemail.com  Tue Jan 12 16:16:30 2010
From: sepperlot at googlemail.com (Sepperlot)
Date: Tue, 12 Jan 2010 16:16:30 +0100
Subject: [rsyslog] Only log from network devices to database
Message-ID: <4B4C924E.200@googlemail.com>

Hello.

I'm trying to log messages from various network devices to rsyslog and
write them into a database.
Therefore I use a setup as described in
http://www.rsyslog.com/doc-rsyslog_mysql.html

My (simple) rsyslog.conf contains the following:

$ModLoad imudp
$UDPServerAddress x.x.x.x
$UDPServerRun 1514 # standard port is used by syslog-ng

$ModLoad ommysql
*.*             :ommysql:localhost,DBNAME,DBUSER,DBPASS


This writes all arriving log messages to the database and I can watch
them with phplogcon. Up to here everything is ok and works.

Now I only want to log messages from specific network devices identified
by ip address but I'm totaly lost when it comes to combine filter
conditions and actions. I've tried

:fromhost-ip, isequal "IP.IP.IP.IP" \
:ommysql:localhost,DBNAME,DBUSER,DBPASS

*.* :fromhost-ip, isequal "IP.IP.IP.IP" \
:ommysql:localhost,DBNAME,DBUSER,DBPASS

but obvious this is BS ;)
Goal is to log only network devices and maybe later log different
devices to different databases.

The backslash is added by me only in this mail. The commands are all in
one line.

Any help is appreciated.

Best regards
Sebastian


From rgerhards at hq.adiscon.com  Tue Jan 12 17:37:09 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 12 Jan 2010 17:37:09 +0100
Subject: [rsyslog] Only log from network devices to database
References: <4B4C924E.200@googlemail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036B7@GRFEXC.intern.adiscon.com>

The config does not look obviously wrong to me (but I am bad at catching
errors...). A good suggestion is to write a debug log, it will tell you in
detail what happened during the filter evaluation.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Sepperlot
> Sent: Tuesday, January 12, 2010 4:17 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] Only log from network devices to database
> 
> Hello.
> 
> I'm trying to log messages from various network devices to rsyslog and
> write them into a database.
> Therefore I use a setup as described in
> http://www.rsyslog.com/doc-rsyslog_mysql.html
> 
> My (simple) rsyslog.conf contains the following:
> 
> $ModLoad imudp
> $UDPServerAddress x.x.x.x
> $UDPServerRun 1514 # standard port is used by syslog-ng
> 
> $ModLoad ommysql
> *.*             :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> 
> This writes all arriving log messages to the database and I can watch
> them with phplogcon. Up to here everything is ok and works.
> 
> Now I only want to log messages from specific network devices
> identified
> by ip address but I'm totaly lost when it comes to combine filter
> conditions and actions. I've tried
> 
> :fromhost-ip, isequal "IP.IP.IP.IP" \
> :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> *.* :fromhost-ip, isequal "IP.IP.IP.IP" \
> :ommysql:localhost,DBNAME,DBUSER,DBPASS
> 
> but obvious this is BS ;)
> Goal is to log only network devices and maybe later log different
> devices to different databases.
> 
> The backslash is added by me only in this mail. The commands are all in
> one line.
> 
> Any help is appreciated.
> 
> Best regards
> Sebastian
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From xkubina at fi.muni.cz  Wed Jan 13 12:16:06 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 12:16:06 +0100
Subject: [rsyslog] How to add new configuration option
Message-ID: <4B4DAB76.7070201@fi.muni.cz>

Hi,

I would appreciate any help with adding support for a new configuration 
directive. I have done some
code and I need now something like:
$AddClientCN [on/off].
I have read the sources to find out how rsyslog processes conf file. 
There is some linked list with
known commands. I think that it is enough to add new item to this list 
but I don't know how.
Is this my idea right?

Thanks for any help.

Regards,

Tomas


From rgerhards at hq.adiscon.com  Wed Jan 13 12:17:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 12:17:59 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>

Hi Tomas,

it's probably the simplest if you post your code so that I can give you the
relevant hints.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Wednesday, January 13, 2010 12:16 PM
> To: rsyslog-users
> Subject: [rsyslog] How to add new configuration option
> 
> Hi,
> 
> I would appreciate any help with adding support for a new configuration
> directive. I have done some
> code and I need now something like:
> $AddClientCN [on/off].
> I have read the sources to find out how rsyslog processes conf file.
> There is some linked list with
> known commands. I think that it is enough to add new item to this list
> but I don't know how.
> Is this my idea right?
> 
> Thanks for any help.
> 
> Regards,
> 
> Tomas
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 13 12:43:11 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 12:43:11 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>

Hi all,

I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
contains a number of bug fixes, some of them important for some environments.
As usual for a beta, it does not contain anything else but fixes. The full
lest can be seen in the change log.

Please note that it is my intent do replace the current (instable ;))
v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
get a few thumbs up, I may be able to accelerate promoting it to stable.

An update for the current master branch will happen soon.

ChangeLog:
http://www.rsyslog.com/Article435.phtml

Download:
http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml

I hope this release is useful.

Rainer 



From xkubina at fi.muni.cz  Wed Jan 13 14:02:31 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 14:02:31 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
Message-ID: <4B4DC467.5000903@fi.muni.cz>

Rainer Gerhards wrote:
> Hi Tomas,
>
> it's probably the simplest if you post your code so that I can give you the
> relevant hints.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
>> Sent: Wednesday, January 13, 2010 12:16 PM
>> To: rsyslog-users
>> Subject: [rsyslog] How to add new configuration option
>>
>> Hi,
>>
>> I would appreciate any help with adding support for a new configuration
>> directive. I have done some
>> code and I need now something like:
>> $AddClientCN [on/off].
>> I have read the sources to find out how rsyslog processes conf file.
>> There is some linked list with
>> known commands. I think that it is enough to add new item to this list
>> but I don't know how.
>> Is this my idea right?
>>
>> Thanks for any help.
>>
>> Regards,
>>
>> Tomas
>>     
>   
Hi Rainer,

the modified files are attached. The alternative code is marked by #if 
statement.
I had to try to do this modification because the project, I am 
interested in, needs
to verify client's authentication. I realize that the patch is something 
like a hack,
because the rsyslog's architecture doesn't provide this feature (adding 
client CN to
syslog message) and it is not proper solution, but for our needs it is 
enough.
BTW I use this templete:
template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME% 
%syslogtag%%msg%\n";

I have done a similar code for adding client principal for imgssapi.

Thanks for help.

Regards,

Tomas



From r.bhatia at ipax.at  Wed Jan 13 14:17:04 2010
From: r.bhatia at ipax.at (Raoul Bhatia [IPAX])
Date: Wed, 13 Jan 2010 14:17:04 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <4B4DC467.5000903@fi.muni.cz>
References: <4B4DAB76.7070201@fi.muni.cz>	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
	<4B4DC467.5000903@fi.muni.cz>
Message-ID: <4B4DC7D0.30300@ipax.at>

-ENOATTACHMENT

the mailinglist strips off this stuff :)

cheers,

On 01/13/2010 02:02 PM, Tomas Kubina wrote:
> the modified files are attached. The alternative code is marked by #if
> statement.
> I had to try to do this modification because the project, I am
> interested in, needs
> to verify client's authentication. I realize that the patch is something
> like a hack,
> because the rsyslog's architecture doesn't provide this feature (adding
> client CN to
> syslog message) and it is not proper solution, but for our needs it is
> enough.
> BTW I use this templete:
> template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
> %syslogtag%%msg%\n";
> 
> I have done a similar code for adding client principal for imgssapi.


-- 
____________________________________________________________________
DI (FH) Raoul Bhatia M.Sc.          email.          r.bhatia at ipax.at
Technischer Leiter

IPAX - Aloy Bhatia Hava OEG         web.          http://www.ipax.at
Barawitzkagasse 10/2/2/11           email.            office at ipax.at
1190 Wien                           tel.               +43 1 3670030
FN 277995t HG Wien                  fax.            +43 1 3670030 15
____________________________________________________________________


From xkubina at fi.muni.cz  Wed Jan 13 14:48:33 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Wed, 13 Jan 2010 14:48:33 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
Message-ID: <4B4DCF31.6090105@fi.muni.cz>

Rainer Gerhards wrote:
> Hi Tomas,
>
> it's probably the simplest if you post your code so that I can give you the
> relevant hints.
>
> Rainer
>
>   
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
>> Sent: Wednesday, January 13, 2010 12:16 PM
>> To: rsyslog-users
>> Subject: [rsyslog] How to add new configuration option
>>
>> Hi,
>>
>> I would appreciate any help with adding support for a new configuration
>> directive. I have done some
>> code and I need now something like:
>> $AddClientCN [on/off].
>> I have read the sources to find out how rsyslog processes conf file.
>> There is some linked list with
>> known commands. I think that it is enough to add new item to this list
>> but I don't know how.
>> Is this my idea right?
>>
>> Thanks for any help.
>>
>> Regards,
>>
>> Tomas
>>     
>   
Hi Rainer,

the modified files are attached. The alternative code is marked by #if
statement. I had to try to do this modification because the project,
I am interested in, needs to verify client's authentication. I realize 
that the patch is something  like a hack, because the rsyslog's 
architecture doesn't provide this feature (adding client CN to
syslog message) and it is not proper solution, but for our needs it is
enough.
BTW I use this templete:
template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
%syslogtag%%msg%\n";

I have done a similar code for adding client principal for imgssapi.

Thanks for help.

Regards,

Tomas

FILES:

nsd_gtls.c

static rsRetVal
AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew)
{
	DEFiRet;
	int gnuRet;
	nsd_gtls_t *pNew = NULL;
	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;

         ISOBJ_TYPE_assert((pThis), nsd_gtls);
	CHKiRet(nsd_gtlsConstruct(&pNew)); // TODO: prevent construct/destruct!
	CHKiRet(nsd_ptcp.Destruct(&pNew->pTcp));
	CHKiRet(nsd_ptcp.AcceptConnReq(pThis->pTcp, &pNew->pTcp));
	
	if(pThis->iMode == 0) {
		/* we are in non-TLS mode, so we are done */
		*ppNew = (nsd_t*) pNew;
		FINALIZE;
	}

	/* if we reach this point, we are in TLS mode */
	CHKiRet(gtlsInitSession(pNew));
	gtlsSetTransportPtr(pNew, ((nsd_ptcp_t*) (pNew->pTcp))->sock);
	pNew->authMode = pThis->authMode;
	pNew->pPermPeers = pThis->pPermPeers;

	/* we now do the handshake. This is a bit complicated, because we are
	 * on non-blocking sockets. Usually, the handshake will not complete
	 * immediately, so that we need to retry it some time later.
	 */
	gnuRet = gnutls_handshake(pNew->sess);
	if(gnuRet == GNUTLS_E_AGAIN || gnuRet == GNUTLS_E_INTERRUPTED) {
		pNew->rtryCall = gtlsRtry_handshake;
		dbgprintf("GnuTLS handshake does not complete immediately - setting to 
retry (this is OK and normal)\n");
	} else if(gnuRet == 0) {
		/* we got a handshake, now check authorization */
		CHKiRet(gtlsChkPeerAuth(pNew));
	} else {
		ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR);
	}

	pNew->iMode = 1; /* this session is now in TLS mode! */
#if 1
         pNew->clientCNValid = 0;
#endif
	*ppNew = (nsd_t*) pNew;

finalize_it:
	if(iRet != RS_RET_OK) {
		if(pNew != NULL)
			nsd_gtlsDestruct(&pNew);
	}
	RETiRet;
}

static rsRetVal
Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
{
	DEFiRet;
	ssize_t iBytesCopy; /* how many bytes are to be copied to the client 
buffer? */
	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
	ISOBJ_TYPE_assert(pThis, nsd_gtls);
#if 1
         cstr_t *pstrCN = NULL;
         const gnutls_datum *cert_list;
	unsigned int cert_list_size = 0;
         gnutls_x509_crt cert;
         int len = 0;
         char *buf_temp;
#endif
	if(pThis->bAbortConn)
		ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);

	if(pThis->iMode == 0) {
		CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
		FINALIZE;
	}

	/* --- in TLS mode now --- */

	/* Buffer logic applies only if we are in TLS mode. Here we
	 * assume that we will switch from plain to TLS, but never back. This
	 * assumption may be unsafe, but it is the model for the time being and I
	 * do not see any valid reason why we should switch back to plain TCP after
	 * we were in TLS mode. However, in that case we may lose something that
	 * is already in the receive buffer ... risk accepted. -- rgerhards, 
2008-06-23
	 */

	if(pThis->pszRcvBuf == NULL) {
		/* we have no buffer, so we need to malloc one */
		CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
		pThis->lenRcvBuf = -1;
	}

	/* now check if we have something in our buffer. If so, we satisfy
	 * the request from buffer contents.
	 */
	if(pThis->lenRcvBuf == -1) { /* no data present, must read */
		CHKiRet(gtlsRecordRecv(pThis));
	}

	if(pThis->lenRcvBuf == 0) { /* EOS */
		*pLenBuf = 0;
		/* in this case, we also need to free the receive buffer, if we
		 * allocated one. -- rgerhards, 2008-12-03
		 */
		if(pThis->pszRcvBuf != NULL) {
			free(pThis->pszRcvBuf);
			pThis->pszRcvBuf = NULL;
		}
		ABORT_FINALIZE(RS_RET_CLOSED);
	}

	/* if we reach this point, data is present in the buffer and must be 
copied */
	iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
	if(iBytesCopy > *pLenBuf) {
		iBytesCopy = *pLenBuf;
	} else {
		pThis->lenRcvBuf = -1; /* buffer will be emptied below */
	}
#if 1
         if (pThis->clientCNValid != 1)
         {
           cert_list = gnutls_certificate_get_peers(pThis->sess, 
&cert_list_size);

	  if(cert_list_size > 0)
           {
	    // we only print information about the first certificate
	    gnutls_x509_crt_init(&cert);
	    gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER);

             CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));

             len = snprintf(NULL, 0, "CN:%s ", (char*)cstrGetSzStr(pstrCN));
             if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char))) )
               return -1;

             snprintf(pThis->clientCN, len + 1, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
             pThis->clientCN[len] = '\0';
             pThis->clientCNLen = len + 1;

             pThis->clientCNValid = 1;
           }
         }

         iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ? 
iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;

         buf_temp = (char*)malloc(iBytesCopy);

         if (buf_temp)
         {
           memset(buf_temp, 0, iBytesCopy);
           strncpy(buf_temp, pThis->clientCN, pThis->clientCNLen);
           strncat(buf_temp, pThis->pszRcvBuf, pThis->lenRcvBuf);
           buf_temp[iBytesCopy] ='\0';
         }

         memset(pBuf, 0, *pLenBuf);
	memcpy(pBuf, buf_temp, iBytesCopy);

         if (buf_temp)
           free(buf_temp);
#else
         memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, iBytesCopy);
#endif	
         pThis->ptrRcvBuf += iBytesCopy;
	*pLenBuf = iBytesCopy;

finalize_it:
	dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf 
%d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
	RETiRet;
}

tcps_sess.c

static rsRetVal
Close(tcps_sess_t *pThis)
{
	DEFiRet;

	ISOBJ_TYPE_assert(pThis, tcps_sess);
	netstrm.Destruct(&pThis->pStrm);
	if(pThis->fromHost != NULL) {
		prop.Destruct(&pThis->fromHost);
	}
	if(pThis->fromHostIP != NULL)
		prop.Destruct(&pThis->fromHostIP);
#if 1
        if(pThis->clientPrincipal != NULL)
		free(pThis->clientPrincipal);
#endif
	RETiRet;
}

tcps_sess.h

/* the tcps_sess object */
struct tcps_sess_s {
	BEGINobjInstance;	/* Data to implement generic object - MUST be the 
first data element! */
	tcpsrv_t *pSrv;	/* pointer back to my server (e.g. for callbacks) */
	tcpLstnPortList_t *pLstnInfo;	/* pointer back to listener info */
	netstrm_t *pStrm;
	int iMsg;		 /* index of next char to store in msg */
	int bAtStrtOfFram;	/* are we at the very beginning of a new frame? */
	enum {
		eAtStrtFram,
		eInOctetCnt,
		eInMsg
	} inputState;		/* our current state */
	int iOctetsRemain;	/* Number of Octets remaining in message */
	TCPFRAMINGMODE eFraming;
	uchar *pMsg;		/* message (fragment) received */
	prop_t *fromHost;	/* host name we received messages from */
	prop_t *fromHostIP;
	void *pUsr;		/* a user-pointer */
#if 1
         char *clientPrincipal;    /* client principal */
         int clientPrincipalLen;
#endif
	rsRetVal (*DoSubmitMessage)(tcps_sess_t*, uchar*, int); /* submit 
message callback */
};




From rgerhards at hq.adiscon.com  Wed Jan 13 15:27:31 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 Jan 2010 15:27:31 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>
	<4B4DCF31.6090105@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>

Thanks for the code. Unfortunately, adding the config switch to it is not
quite easy in that case (good I asked for the actual code). I'd say that you
best do it similar to the other config directives, like the authentication
mode. They actual directives are in the upper level code (imtcp/omfwd).
There, they are shuffled over to the instance data, which goes along with
each of the configured listeners/sender. Then, when a new network stream is
created, the params are passed down to the generic stream interface and there
passed down to the selected stream driver, which finally stores and acts on
them. It's clumpsy and quite some work, but that is what is needed for the
old config system. You probably need to add around 50 to 100 lines of code
altogether to the various files. It's not complex, but easy to forget
something. Best start by a directive (like $..AuthMode), see how it is
handled (and passed down) in imtcp and work your way down the stack ;)

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Wednesday, January 13, 2010 2:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to add new configuration option
> 
> Rainer Gerhards wrote:
> > Hi Tomas,
> >
> > it's probably the simplest if you post your code so that I 
> can give you the
> > relevant hints.
> >
> > Rainer
> >
> >   
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> >> Sent: Wednesday, January 13, 2010 12:16 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] How to add new configuration option
> >>
> >> Hi,
> >>
> >> I would appreciate any help with adding support for a new 
> configuration
> >> directive. I have done some
> >> code and I need now something like:
> >> $AddClientCN [on/off].
> >> I have read the sources to find out how rsyslog processes 
> conf file.
> >> There is some linked list with
> >> known commands. I think that it is enough to add new item 
> to this list
> >> but I don't know how.
> >> Is this my idea right?
> >>
> >> Thanks for any help.
> >>
> >> Regards,
> >>
> >> Tomas
> >>     
> >   
> Hi Rainer,
> 
> the modified files are attached. The alternative code is marked by #if
> statement. I had to try to do this modification because the project,
> I am interested in, needs to verify client's authentication. 
> I realize 
> that the patch is something  like a hack, because the rsyslog's 
> architecture doesn't provide this feature (adding client CN to
> syslog message) and it is not proper solution, but for our needs it is
> enough.
> BTW I use this templete:
> template ILS_template,"%timegenerated% %fromhost-ip% %HOSTNAME%
> %syslogtag%%msg%\n";
> 
> I have done a similar code for adding client principal for imgssapi.
> 
> Thanks for help.
> 
> Regards,
> 
> Tomas
> 
> FILES:
> 
> nsd_gtls.c
> 
> static rsRetVal
> AcceptConnReq(nsd_t *pNsd, nsd_t **ppNew)
> {
> 	DEFiRet;
> 	int gnuRet;
> 	nsd_gtls_t *pNew = NULL;
> 	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 
>          ISOBJ_TYPE_assert((pThis), nsd_gtls);
> 	CHKiRet(nsd_gtlsConstruct(&pNew)); // TODO: prevent 
> construct/destruct!
> 	CHKiRet(nsd_ptcp.Destruct(&pNew->pTcp));
> 	CHKiRet(nsd_ptcp.AcceptConnReq(pThis->pTcp, &pNew->pTcp));
> 	
> 	if(pThis->iMode == 0) {
> 		/* we are in non-TLS mode, so we are done */
> 		*ppNew = (nsd_t*) pNew;
> 		FINALIZE;
> 	}
> 
> 	/* if we reach this point, we are in TLS mode */
> 	CHKiRet(gtlsInitSession(pNew));
> 	gtlsSetTransportPtr(pNew, ((nsd_ptcp_t*) (pNew->pTcp))->sock);
> 	pNew->authMode = pThis->authMode;
> 	pNew->pPermPeers = pThis->pPermPeers;
> 
> 	/* we now do the handshake. This is a bit complicated, 
> because we are
> 	 * on non-blocking sockets. Usually, the handshake will 
> not complete
> 	 * immediately, so that we need to retry it some time later.
> 	 */
> 	gnuRet = gnutls_handshake(pNew->sess);
> 	if(gnuRet == GNUTLS_E_AGAIN || gnuRet == GNUTLS_E_INTERRUPTED) {
> 		pNew->rtryCall = gtlsRtry_handshake;
> 		dbgprintf("GnuTLS handshake does not complete 
> immediately - setting to 
> retry (this is OK and normal)\n");
> 	} else if(gnuRet == 0) {
> 		/* we got a handshake, now check authorization */
> 		CHKiRet(gtlsChkPeerAuth(pNew));
> 	} else {
> 		ABORT_FINALIZE(RS_RET_TLS_HANDSHAKE_ERR);
> 	}
> 
> 	pNew->iMode = 1; /* this session is now in TLS mode! */
> #if 1
>          pNew->clientCNValid = 0;
> #endif
> 	*ppNew = (nsd_t*) pNew;
> 
> finalize_it:
> 	if(iRet != RS_RET_OK) {
> 		if(pNew != NULL)
> 			nsd_gtlsDestruct(&pNew);
> 	}
> 	RETiRet;
> }
> 
> static rsRetVal
> Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
> {
> 	DEFiRet;
> 	ssize_t iBytesCopy; /* how many bytes are to be copied 
> to the client 
> buffer? */
> 	nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 	ISOBJ_TYPE_assert(pThis, nsd_gtls);
> #if 1
>          cstr_t *pstrCN = NULL;
>          const gnutls_datum *cert_list;
> 	unsigned int cert_list_size = 0;
>          gnutls_x509_crt cert;
>          int len = 0;
>          char *buf_temp;
> #endif
> 	if(pThis->bAbortConn)
> 		ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);
> 
> 	if(pThis->iMode == 0) {
> 		CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
> 		FINALIZE;
> 	}
> 
> 	/* --- in TLS mode now --- */
> 
> 	/* Buffer logic applies only if we are in TLS mode. Here we
> 	 * assume that we will switch from plain to TLS, but 
> never back. This
> 	 * assumption may be unsafe, but it is the model for 
> the time being and I
> 	 * do not see any valid reason why we should switch 
> back to plain TCP after
> 	 * we were in TLS mode. However, in that case we may 
> lose something that
> 	 * is already in the receive buffer ... risk accepted. 
> -- rgerhards, 
> 2008-06-23
> 	 */
> 
> 	if(pThis->pszRcvBuf == NULL) {
> 		/* we have no buffer, so we need to malloc one */
> 		CHKmalloc(pThis->pszRcvBuf = 
> MALLOC(NSD_GTLS_MAX_RCVBUF));
> 		pThis->lenRcvBuf = -1;
> 	}
> 
> 	/* now check if we have something in our buffer. If so, 
> we satisfy
> 	 * the request from buffer contents.
> 	 */
> 	if(pThis->lenRcvBuf == -1) { /* no data present, must read */
> 		CHKiRet(gtlsRecordRecv(pThis));
> 	}
> 
> 	if(pThis->lenRcvBuf == 0) { /* EOS */
> 		*pLenBuf = 0;
> 		/* in this case, we also need to free the 
> receive buffer, if we
> 		 * allocated one. -- rgerhards, 2008-12-03
> 		 */
> 		if(pThis->pszRcvBuf != NULL) {
> 			free(pThis->pszRcvBuf);
> 			pThis->pszRcvBuf = NULL;
> 		}
> 		ABORT_FINALIZE(RS_RET_CLOSED);
> 	}
> 
> 	/* if we reach this point, data is present in the 
> buffer and must be 
> copied */
> 	iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
> 	if(iBytesCopy > *pLenBuf) {
> 		iBytesCopy = *pLenBuf;
> 	} else {
> 		pThis->lenRcvBuf = -1; /* buffer will be 
> emptied below */
> 	}
> #if 1
>          if (pThis->clientCNValid != 1)
>          {
>            cert_list = gnutls_certificate_get_peers(pThis->sess, 
> &cert_list_size);
> 
> 	  if(cert_list_size > 0)
>            {
> 	    // we only print information about the first certificate
> 	    gnutls_x509_crt_init(&cert);
> 	    gnutls_x509_crt_import(cert, &cert_list[0], 
> GNUTLS_X509_FMT_DER);
> 
>              CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));
> 
>              len = snprintf(NULL, 0, "CN:%s ", 
> (char*)cstrGetSzStr(pstrCN));
>              if ( !(pThis->clientCN = malloc((len + 
> 1)*sizeof(char))) )
>                return -1;
> 
>              snprintf(pThis->clientCN, len + 1, "CN:%s ", 
> (char*)cstrGetSzStr(pstrCN));
>              pThis->clientCN[len] = '\0';
>              pThis->clientCNLen = len + 1;
> 
>              pThis->clientCNValid = 1;
>            }
>          }
> 
>          iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < 
> *pLenBuf ? 
> iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;
> 
>          buf_temp = (char*)malloc(iBytesCopy);
> 
>          if (buf_temp)
>          {
>            memset(buf_temp, 0, iBytesCopy);
>            strncpy(buf_temp, pThis->clientCN, pThis->clientCNLen);
>            strncat(buf_temp, pThis->pszRcvBuf, pThis->lenRcvBuf);
>            buf_temp[iBytesCopy] ='\0';
>          }
> 
>          memset(pBuf, 0, *pLenBuf);
> 	memcpy(pBuf, buf_temp, iBytesCopy);
> 
>          if (buf_temp)
>            free(buf_temp);
> #else
>          memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, 
> iBytesCopy);
> #endif	
>          pThis->ptrRcvBuf += iBytesCopy;
> 	*pLenBuf = iBytesCopy;
> 
> finalize_it:
> 	dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf 
> %d, ptrRcvBuf 
> %d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
> 	RETiRet;
> }
> 
> tcps_sess.c
> 
> static rsRetVal
> Close(tcps_sess_t *pThis)
> {
> 	DEFiRet;
> 
> 	ISOBJ_TYPE_assert(pThis, tcps_sess);
> 	netstrm.Destruct(&pThis->pStrm);
> 	if(pThis->fromHost != NULL) {
> 		prop.Destruct(&pThis->fromHost);
> 	}
> 	if(pThis->fromHostIP != NULL)
> 		prop.Destruct(&pThis->fromHostIP);
> #if 1
>         if(pThis->clientPrincipal != NULL)
> 		free(pThis->clientPrincipal);
> #endif
> 	RETiRet;
> }
> 
> tcps_sess.h
> 
> /* the tcps_sess object */
> struct tcps_sess_s {
> 	BEGINobjInstance;	/* Data to implement generic 
> object - MUST be the 
> first data element! */
> 	tcpsrv_t *pSrv;	/* pointer back to my server (e.g. for 
> callbacks) */
> 	tcpLstnPortList_t *pLstnInfo;	/* pointer back to 
> listener info */
> 	netstrm_t *pStrm;
> 	int iMsg;		 /* index of next char to store 
> in msg */
> 	int bAtStrtOfFram;	/* are we at the very beginning 
> of a new frame? */
> 	enum {
> 		eAtStrtFram,
> 		eInOctetCnt,
> 		eInMsg
> 	} inputState;		/* our current state */
> 	int iOctetsRemain;	/* Number of Octets remaining 
> in message */
> 	TCPFRAMINGMODE eFraming;
> 	uchar *pMsg;		/* message (fragment) received */
> 	prop_t *fromHost;	/* host name we received 
> messages from */
> 	prop_t *fromHostIP;
> 	void *pUsr;		/* a user-pointer */
> #if 1
>          char *clientPrincipal;    /* client principal */
>          int clientPrincipalLen;
> #endif
> 	rsRetVal (*DoSubmitMessage)(tcps_sess_t*, uchar*, int); 
> /* submit 
> message callback */
> };
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Thu Jan 14 09:37:08 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 00:37:08 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001140036380.15983@asgard.lang.hm>

I am not seeing a tag for 5.3.6 in git. Am I missing something?

David Lang

On Wed, 13 Jan 2010, Rainer Gerhards wrote:

> Hi all,
>
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
>
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> An update for the current master branch will happen soon.
>
> ChangeLog:
> http://www.rsyslog.com/Article435.phtml
>
> Download:
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
>
> I hope this release is useful.
>
> Rainer
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Thu Jan 14 10:35:16 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jan 2010 10:35:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001140036380.15983@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036C6@GRFEXC.intern.adiscon.com>

Oh, thanks - I added the tag, but forgot to push it (looks like the vacation
was too good ;)).

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 14, 2010 9:37 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> I am not seeing a tag for 5.3.6 in git. Am I missing something?
> 
> David Lang
> 
> On Wed, 13 Jan 2010, Rainer Gerhards wrote:
> 
> > Hi all,
> >
> > I have just released rsyslog 5.3.6, a new v5-beta. Note 
> that this version
> > contains a number of bug fixes, some of them important for 
> some environments.
> > As usual for a beta, it does not contain anything else but 
> fixes. The full
> > lest can be seen in the change log.
> >
> > Please note that it is my intent do replace the current 
> (instable ;))
> > v5-stable by this beta soon. So I would appreciate feedback 
> on 5.3.6 - if I
> > get a few thumbs up, I may be able to accelerate promoting 
> it to stable.
> >
> > An update for the current master branch will happen soon.
> >
> > ChangeLog:
> > http://www.rsyslog.com/Article435.phtml
> >
> > Download:
> > 
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> >
> > I hope this release is useful.
> >
> > Rainer
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ktm at rice.edu  Thu Jan 14 14:58:51 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Thu, 14 Jan 2010 07:58:51 -0600
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <20100114135851.GF1895@it.is.rice.edu>

Hi Rainer,

I have been running 5.3.6 with a PostgreSQL 8.4 backend and
it has not exhibited the problems that I saw in 5.3.5 that
caused me to roll back to 4.4.2. Thank you for the fixes to
the PostgreSQL transaction interface. I will be doing some
more testing of the new functionality but it looks good.

Regards,
Ken

On Wed, Jan 13, 2010 at 12:43:11PM +0100, Rainer Gerhards wrote:
> Hi all,
> 
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
> 
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.
> 
> An update for the current master branch will happen soon.
> 
> ChangeLog:
> http://www.rsyslog.com/Article435.phtml
> 
> Download:
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> 
> I hope this release is useful.
> 
> Rainer 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From rgerhards at hq.adiscon.com  Thu Jan 14 16:05:12 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 14 Jan 2010 16:05:12 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<20100114135851.GF1895@it.is.rice.edu>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036CC@GRFEXC.intern.adiscon.com>

Hi Ken,

Thanks for the feedback, much appreciated. Please let my know anything more
of interest that you may find out.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
> Kenneth Marshall
> Sent: Thursday, January 14, 2010 2:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Hi Rainer,
> 
> I have been running 5.3.6 with a PostgreSQL 8.4 backend and
> it has not exhibited the problems that I saw in 5.3.5 that
> caused me to roll back to 4.4.2. Thank you for the fixes to
> the PostgreSQL transaction interface. I will be doing some
> more testing of the new functionality but it looks good.
> 
> Regards,
> Ken
> 
> On Wed, Jan 13, 2010 at 12:43:11PM +0100, Rainer Gerhards wrote:
> > Hi all,
> > 
> > I have just released rsyslog 5.3.6, a new v5-beta. Note 
> that this version
> > contains a number of bug fixes, some of them important for 
> some environments.
> > As usual for a beta, it does not contain anything else but 
> fixes. The full
> > lest can be seen in the change log.
> > 
> > Please note that it is my intent do replace the current 
> (instable ;))
> > v5-stable by this beta soon. So I would appreciate feedback 
> on 5.3.6 - if I
> > get a few thumbs up, I may be able to accelerate promoting 
> it to stable.
> > 
> > An update for the current master branch will happen soon.
> > 
> > ChangeLog:
> > http://www.rsyslog.com/Article435.phtml
> > 
> > Download:
> > 
> http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-191.phtml
> > 
> > I hope this release is useful.
> > 
> > Rainer 
> > 
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> > 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ryan.b.lynch at gmail.com  Thu Jan 14 16:16:43 2010
From: ryan.b.lynch at gmail.com (Ryan Lynch)
Date: Thu, 14 Jan 2010 10:16:43 -0500
Subject: [rsyslog] MySQL output module: General questions.
Message-ID: <115906d11001140716o3c45a659ndddb7a5851fc7d35@mail.gmail.com>

Hi,

I was hoping that someone with experience using the MySQL output
module, or maybe someone familiar with the source, could help me
understand a few details about the module.

1) Can ommysql use SSL connections to the database server? If not, are
there any future plans to add SSL support?

2) Do failover destinations ('$ActionExecOnlyWhenPreviousIsSuspended',
http://wiki.rsyslog.com/index.php/FailoverSyslogServer) work correctly
with ommysql? If so, how and when do connection failures
register--does the failover happen when the MySQL client fails to
execute an INSERT statement, or when the TCP socket dies, or what?

3) Does ommysql support periodically re-connection to the database server?

4) Is the retry limit for ommysql's INSERT process configurable? The
HOWTO (http://www.rsyslog.com/doc-rsyslog_mysql.html), in the section
'On Reliability...', says "If rsyslogd is unable to store a message,
it performs one retry." I assume this means the retry limit is
hard-coded--is that right?

5) How efficient is ommysql in comparison to omtcp or omrelp? I
imagine there's more overhead for the MySQL protocol, but I don't know
whether are other considerations, too. I would love to hear what
levels of load people experience, with ommysql in production, and what
kind of log volumes they handle.

Thanks!

Ryan B. Lynch
ryan.b.lynch at gmail.com


From mbiebl at gmail.com  Fri Jan 15 07:48:46 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 07:48:46 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>

2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Hi all,
>
> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
> contains a number of bug fixes, some of them important for some environments.
> As usual for a beta, it does not contain anything else but fixes. The full
> lest can be seen in the change log.
>
> Please note that it is my intent do replace the current (instable ;))
> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
> get a few thumbs up, I may be able to accelerate promoting it to stable.

So I gave 5.3.6 a try and stumbled over some rather important regressions.

Compilation and installation went fine and rsyslog started up without
an error message.
I got one log message in the syslog
Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.

But silence afterwards.

When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice

Running rsyslogd -d -c4 I got
pluto:~# rsyslogd -d -c4
rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
rsyslogd: the last error occured in
/etc/rsyslog.d/network-manager.conf, line 2:"~"
rsyslogd: warning: selector line without actions will be discarded
rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
rsyslogd: the last error occured in
/etc/rsyslog.d/network-manager.conf, line 4:"~"
rsyslogd: warning: selector line without actions will be discarded
rsyslogd: the last error occured in /etc/rsyslog.conf, line
46:"$IncludeConfig /etc/rsyslog.d/*.conf"
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

The file in question (which worked fine with 4.4.2) contains
=====
:programname, contains, "NetworkManager" /var/log/NetworkManager.log
~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
~
=====


several issues here:
first of all, the above statements no longer work.
second, rsyslog can't be killed anymore with a single SIGTERM
third, it shouldn't just silently fail. I neither got an error message
on stdout/stderr, nor in the log file.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 07:52:17 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 07:52:17 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
Message-ID: <aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Hi all,
>>
>> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
>> contains a number of bug fixes, some of them important for some environments.
>> As usual for a beta, it does not contain anything else but fixes. The full
>> lest can be seen in the change log.
>>
>> Please note that it is my intent do replace the current (instable ;))
>> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
>> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> So I gave 5.3.6 a try and stumbled over some rather important regressions.
>
> Compilation and installation went fine and rsyslog started up without
> an error message.
> I got one log message in the syslog
> Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.
>
> But silence afterwards.
>
> When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice
>
> Running rsyslogd -d -c4 I got
> pluto:~# rsyslogd -d -c4
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 2:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 4:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 46:"$IncludeConfig /etc/rsyslog.d/*.conf"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> The file in question (which worked fine with 4.4.2) contains
> =====
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> =====
>
>
> several issues here:
> first of all, the above statements no longer work.
> second, rsyslog can't be killed anymore with a single SIGTERM
> third, it shouldn't just silently fail. I neither got an error message
> on stdout/stderr, nor in the log file.

fwiw, changing -c4 to -c5 and removing network-manager.conf didn't help.

rsyslog still logs nothing and rsyslog -d is suspicously silent (i.e.
no output).

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 07:53:07 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 22:53:07 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142252150.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/13 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Hi all,
>>
>> I have just released rsyslog 5.3.6, a new v5-beta. Note that this version
>> contains a number of bug fixes, some of them important for some environments.
>> As usual for a beta, it does not contain anything else but fixes. The full
>> lest can be seen in the change log.
>>
>> Please note that it is my intent do replace the current (instable ;))
>> v5-stable by this beta soon. So I would appreciate feedback on 5.3.6 - if I
>> get a few thumbs up, I may be able to accelerate promoting it to stable.
>
> So I gave 5.3.6 a try and stumbled over some rather important regressions.
>
> Compilation and installation went fine and rsyslog started up without
> an error message.
> I got one log message in the syslog
> Jan 15 07:39:29 pluto kernel: imklog 5.3.6, log source = /proc/kmsg started.
>
> But silence afterwards.
>
> When I tried to stop rsyslog using SIGTERM it failed, I had to use SIGTERM twice
>
> Running rsyslogd -d -c4 I got

it may not matter, but I think you need to do -c5 with 5.x

David Lang

> pluto:~# rsyslogd -d -c4
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 2:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
> rsyslogd: the last error occured in
> /etc/rsyslog.d/network-manager.conf, line 4:"~"
> rsyslogd: warning: selector line without actions will be discarded
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 46:"$IncludeConfig /etc/rsyslog.d/*.conf"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
> The file in question (which worked fine with 4.4.2) contains
> =====
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> =====
>
>
> several issues here:
> first of all, the above statements no longer work.
> second, rsyslog can't be killed anymore with a single SIGTERM
> third, it shouldn't just silently fail. I neither got an error message
> on stdout/stderr, nor in the log file.
>
> Cheers,
> Michael
>
>
>


From pgollucci at p6m7g8.com  Fri Jan 15 07:59:45 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Fri, 15 Jan 2010 06:59:45 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
Message-ID: <4B501261.1070901@p6m7g8.com>

Michael Biebl wrote:
>> =====
>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> ~
>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> ~
>> =====

1) rsyslogd always pukes on itself if the config file doesn't parse.
    not new.

2) Its documented that selectors syntax is not a stable API / config 
file syntax, though, the maintainer should have noted it in UPDATING.
I would have hit this tomorrow myself.

http://www.rsyslog.com/doc-rsyslog_conf_filter.html
Expression-Based Filters
Expression based filters allow filtering on arbitrary complex 
expressions, which can include boolean, arithmetic and string 
operations. Expression filters will evolve into a full configuration 
scripting language. Unfortunately, their syntax will slightly change 
during that process. So if you use them now, you need to be prepared to 
change your configuration files some time later. However, we try to 
implement the scripting facility as soon as possible (also in respect to 
stage work needed). So the window of exposure is probably not too long.

-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From mbiebl at gmail.com  Fri Jan 15 08:15:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:15:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <4B501261.1070901@p6m7g8.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
Message-ID: <aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>

2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
> Michael Biebl wrote:
>>> =====
>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>> ~
>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>> ~
>>> =====
>
> 1) rsyslogd always pukes on itself if the config file doesn't parse.
> ? ?not new.
>
> 2) Its documented that selectors syntax is not a stable API / config
> file syntax, though, the maintainer should have noted it in UPDATING.
> I would have hit this tomorrow myself.
>

You forgot the part, where I said that I remove those lines and
rsyslog still doesn't log anything.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 08:19:09 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:19:09 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
Message-ID: <aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>> Michael Biebl wrote:
>>>> =====
>>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>>> ~
>>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>>> ~
>>>> =====
>>
>> 1) rsyslogd always pukes on itself if the config file doesn't parse.
>> ? ?not new.
>>
>> 2) Its documented that selectors syntax is not a stable API / config
>> file syntax, though, the maintainer should have noted it in UPDATING.
>> I would have hit this tomorrow myself.
>>
>
> You forgot the part, where I said that I remove those lines and
> rsyslog still doesn't log anything.

And the fact that if it fails to parse, it should complain loudly and
not fail silently.

Anyway, let's see what Rainer has to say.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:23:51 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:23:51 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>>> Michael Biebl wrote:
>>>>> =====
>>>>> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>>>>> ~
>>>>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>>>>> ~
>>>>> =====
>>>
>>> 1) rsyslogd always pukes on itself if the config file doesn't parse.
>>> ? ?not new.
>>>
>>> 2) Its documented that selectors syntax is not a stable API / config
>>> file syntax, though, the maintainer should have noted it in UPDATING.
>>> I would have hit this tomorrow myself.
>>>
>>
>> You forgot the part, where I said that I remove those lines and
>> rsyslog still doesn't log anything.
>
> And the fact that if it fails to parse, it should complain loudly and
> not fail silently.

unfortunantly in my experiance it doesn't complain loudly :-(

in V5 there is a new option to tell it to exit if it can't read the 
config.

I suspect that the actual config error is significantly earlier in the 
config. One thing that I frequently tripped over when switching back and 
forth was th HUPisRestart option. In V5 that's not a valid option anymore 
and needs to be removed.

can you post your full config?

David Lang

> Anyway, let's see what Rainer has to say.
>
> Michael
>
>
>

From mbiebl at gmail.com  Fri Jan 15 08:27:16 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:27:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> in V5 there is a new option to tell it to exit if it can't read the config.
>
> I suspect that the actual config error is significantly earlier in the
> config. One thing that I frequently tripped over when switching back and
> forth was th HUPisRestart option. In V5 that's not a valid option anymore
> and needs to be removed.
>
> can you post your full config?

Here is the rsyslog.conf (default Debian install)
http://paste.debian.net/56723/
and the included network-manager.conf file
http://paste.debian.net/56724/

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:34:03 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:34:03 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>> in V5 there is a new option to tell it to exit if it can't read the config.
>>
>> I suspect that the actual config error is significantly earlier in the
>> config. One thing that I frequently tripped over when switching back and
>> forth was th HUPisRestart option. In V5 that's not a valid option anymore
>> and needs to be removed.
>>
>> can you post your full config?
>
> Here is the rsyslog.conf (default Debian install)
> http://paste.debian.net/56723/
> and the included network-manager.conf file
> http://paste.debian.net/56724/

I think I just realized the problem

you have

:programname, contains, "NetworkManager" /var/log/NetworkManager.log
~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
~

when you should have

:programname, contains, "NetworkManager" /var/log/NetworkManager.log
& ~
:programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
& ~

give that a shot.

David Lang


From pgollucci at p6m7g8.com  Fri Jan 15 08:37:41 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Fri, 15 Jan 2010 07:37:41 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
Message-ID: <4B501B45.2080006@p6m7g8.com>

david at lang.hm wrote:
  > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
> 
> when you should have
> 
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> & ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> & ~
Sounds reasonable.  I typically do --

# MySQL
:programname, contains, "mysql"  ?by_prog
& :omrelp:cl.tld:2514
& ~

# REST
*.* :omrelp:cl.tld:2514



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From mbiebl at gmail.com  Fri Jan 15 08:42:10 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:42:10 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>

2010/1/15  <david at lang.hm>:

> I think I just realized the problem
>
> you have
>
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> ~
>
> when you should have
>
> :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> & ~
> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> & ~
>
> give that a shot.

The error message goes away but rsyslog still logs nothing.

Interesting fact is, that the above syntax worked fine with 4.4.2



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 08:50:19 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 14 Jan 2010 23:50:19 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>
>> :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> & ~
>>
>> give that a shot.
>
> The error message goes away but rsyslog still logs nothing.
>
> Interesting fact is, that the above syntax worked fine with 4.4.2

You can wait for Rainer to weigh in, but if you want to test more I would 
start by commenting out everything you can and see if it works, then 
putting more stuff back until it fails.

I have noticed that V5 tends to be a bit more sensitive to invalid lines 
than v4 was, v4 seemed to just ignore what it couldn't understand and 
continue, v5 just goes nuts (very similar to what you re reporting)

you may also try adding '$AboortOnUncleanConfig yes' to the config. I 
found that gave me an error in some cases where it just wouldn't do what I 
expected without it.

David Lang


From mbiebl at gmail.com  Fri Jan 15 08:55:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 08:55:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>

2010/1/15  <david at lang.hm>:
>
> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
> found that gave me an error in some cases where it just wouldn't do what I
> expected without it.

Oh, the irony :-)

rsyslogd: Option value must be on or off, but is 'yes'
rsyslogd: the last error occured in /etc/rsyslog.conf, line
11:"$AbortOnUncleanConfig yes"
rsyslogd: CONFIG ERROR: could not interpret master config file
'/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]


Unfortunately, no further clues. Will try the undocument-everything
approach now.
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:01:49 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:01:49 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> 2010/1/15  <david at lang.hm>:
>>
>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>> found that gave me an error in some cases where it just wouldn't do what I
>> expected without it.
>
> Oh, the irony :-)
>
> rsyslogd: Option value must be on or off, but is 'yes'
> rsyslogd: the last error occured in /etc/rsyslog.conf, line
> 11:"$AbortOnUncleanConfig yes"
> rsyslogd: CONFIG ERROR: could not interpret master config file
> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>
>
> Unfortunately, no further clues. Will try the undocument-everything
> approach now.

anything different if you use 'on' instead of 'yes'?

David Lang


From mbiebl at gmail.com  Fri Jan 15 09:07:08 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:07:08 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> 2010/1/15 ?<david at lang.hm>:
>>>
>>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>>> found that gave me an error in some cases where it just wouldn't do what I
>>> expected without it.
>>
>> Oh, the irony :-)
>>
>> rsyslogd: Option value must be on or off, but is 'yes'
>> rsyslogd: the last error occured in /etc/rsyslog.conf, line
>> 11:"$AbortOnUncleanConfig yes"
>> rsyslogd: CONFIG ERROR: could not interpret master config file
>> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>
>>
>> Unfortunately, no further clues. Will try the undocument-everything
>> approach now.
>
> anything different if you use 'on' instead of 'yes'?

Tried that of course. There is no relevant error message.

Further testing revealed:
-d no longer gives me the debug messages on stdout.
I had to run kill -USR1 $(cat /var/run/rsyslogd.pid) to get a verbose output.

With a tiny rsyslog.conf like
$ModLoad imuxsock
*.* /var/log/debug-rsyslog

I finally get log messages again.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Fri Jan 15 09:08:23 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:08:23 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
Message-ID: <aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>

2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/15 ?<david at lang.hm>:
>> On Fri, 15 Jan 2010, Michael Biebl wrote:
>>
>>> 2010/1/15 ?<david at lang.hm>:
>>>>
>>>> you may also try adding '$AboortOnUncleanConfig yes' to the config. I
>>>> found that gave me an error in some cases where it just wouldn't do what I
>>>> expected without it.
>>>
>>> Oh, the irony :-)
>>>
>>> rsyslogd: Option value must be on or off, but is 'yes'
>>> rsyslogd: the last error occured in /etc/rsyslog.conf, line
>>> 11:"$AbortOnUncleanConfig yes"
>>> rsyslogd: CONFIG ERROR: could not interpret master config file
>>> '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>>
>>>
>>> Unfortunately, no further clues. Will try the undocument-everything
>>> approach now.
>>
>> anything different if you use 'on' instead of 'yes'?
>
> Tried that of course. There is no relevant error message.
>
> Further testing revealed:
> -d no longer gives me the debug messages on stdout.
> I had to run kill -USR1 $(cat /var/run/rsyslogd.pid) to get a verbose output.
>
> With a tiny rsyslog.conf like
> $ModLoad imuxsock
> *.* /var/log/debug-rsyslog
>
> I finally get log messages again.

BTW, I'm actually surprised that you don't encounter those problems yourself.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:11:08 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:11:08 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
	<aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> BTW, I'm actually surprised that you don't encounter those problems yourself.

I'm running 5.3.5 still, I haven't had time to build a new version 
(hopefully tomorrow)

David Lang


From mrdemeanour at jackpot.uk.net  Fri Jan 15 09:19:54 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 08:19:54 +0000
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <4B50252A.1000106@jackpot.uk.net>

david at lang.hm wrote:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
> 
>> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
>>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
>>>> Michael Biebl wrote:
>>>>>> ===== :programname, contains, "NetworkManager"
>>>>>> /var/log/NetworkManager.log ~ :programname, contains,
>>>>>> "wpa_supplicant" /var/log/NetworkManager.log ~ =====
>>>> 
>>>> 1) rsyslogd always pukes on itself if the config file doesn't
>>>> parse. not new.
>>>> 
>>>> 2) Its documented that selectors syntax is not a stable API /
>>>> config file syntax, though, the maintainer should have noted it
>>>> in UPDATING. I would have hit this tomorrow myself.
>>>> 
>>> 
>>> You forgot the part, where I said that I remove those lines and 
>>> rsyslog still doesn't log anything.
>> 
>> And the fact that if it fails to parse, it should complain loudly
>> and not fail silently.
> 
> unfortunantly in my experiance it doesn't complain loudly :-(
> 
> in V5 there is a new option to tell it to exit if it can't read the
> config.

Regarding failure to parse the config:

If you have a config entry of this form:

*.*	-/var/log/syslog	# Send everything else to syslog

(i.e. with a trailing comment appended using hash), it doesn't work (on
4.5.6, at least - I've observed this with other versions, but I don't
have a list). The config line is silently ignored.

The manpage says:
     "Lines  starting with a hash mark ('#') and empty lines are
      ignored."

That's fair enough; but it doesn't mention that lines containing a
trailing comment will also be ignored (silently).

Incorrect config lines should elicit a complaint in rsyslogd's output.

-- 
Jack.


From mbiebl at gmail.com  Fri Jan 15 09:44:36 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:44:36 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <4B50252A.1000106@jackpot.uk.net>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
Message-ID: <aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>

Apparently it is that line in my config file, that make rsyslog unhappy:

daemon.*;mail.*;\
       news.err;\
       *.=debug;*.=info;\
       *.=notice;*.=warn       |/dev/xconsole


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From david at lang.hm  Fri Jan 15 09:48:35 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 15 Jan 2010 00:48:35 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>

On Fri, 15 Jan 2010, Michael Biebl wrote:

> Apparently it is that line in my config file, that make rsyslog unhappy:
>
> daemon.*;mail.*;\
>       news.err;\
>       *.=debug;*.=info;\
>       *.=notice;*.=warn       |/dev/xconsole

my ubuntu laptop doesn't have /dev/xconsole

also, I thought that | was used to execute a program and send the 
logmessage to stdin on that program

David Lang


From mbiebl at gmail.com  Fri Jan 15 09:59:13 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 09:59:13 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
	<alpine.DEB.2.00.1001150047000.4649@asgard.lang.hm>
Message-ID: <aa76cdac1001150059rd187152w95901c02dd37b6d9@mail.gmail.com>

2010/1/15  <david at lang.hm>:
> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> Apparently it is that line in my config file, that make rsyslog unhappy:
>>
>> daemon.*;mail.*;\
>> ? ? ? news.err;\
>> ? ? ? *.=debug;*.=info;\
>> ? ? ? *.=notice;*.=warn ? ? ? |/dev/xconsole
>
> my ubuntu laptop doesn't have /dev/xconsole

That's most likely because of the switch to native upstart jobs. The
old SysV init script had an explicit
mknod -m 640 /dev/xconsole p
line. The new upstart job apparently not anymore. That is arguably a
bug in the upstart job.
(I've CC Michael Vogt, as he is responsible for rsyslog during the lucid cycle)

> also, I thought that | was used to execute a program and send the
> logmessage to stdin on that program

We might argue about the usefulness of this, but it's mostly for
historical reasons. The original syslog.conf had this entry for ages.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:25:38 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:25:38 +0100
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6 (v5-beta)
	released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<4B50252A.1000106@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>

Hi folks,

jumping right in the middle and looking at one issue at the other ;)

Please note that nothing is silently ignored. Whenever rsyslog encounters a
problem, a message is generated. HOWEVER, almost nobody ever looks at the
messages emitted from the syslog facility and so the error messages are
"lost". See also:

http://blog.gerhards.net/2009/11/rsyslog-internal-messages.html

For this, the $AbortOnUnleanConfig directive has been introduced, which will
prevent rsyslog from starting if there is any problem. As the doc for that
directive

http://www.rsyslog.com/doc-rsconf1_abortonuncleanconfig.html

says, enabling it can have harsh consequences. There is a reason that rsyslog
by default does not abort - but rather emit an error message - and continue
to function for that part of the config that is OK. This usually is much
better than aborting.

Please note that this is a long-term issue. For example, see this blog post:

http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.html

Since I have written this post, rsyslog now has a config check action and
also emits error messages (if not disabled) to stderr during startup. I have
to admit I have no further clue how I can make sure people actually look at
the error messages... (it's quite frustrating for me). Any suggestions are
very welcome.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 9:20 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> david at lang.hm wrote:
> > On Fri, 15 Jan 2010, Michael Biebl wrote:
> >
> >> 2010/1/15 Michael Biebl <mbiebl at gmail.com>:
> >>> 2010/1/15 Philip M. Gollucci <pgollucci at p6m7g8.com>:
> >>>> Michael Biebl wrote:
> >>>>>> ===== :programname, contains, "NetworkManager"
> >>>>>> /var/log/NetworkManager.log ~ :programname, contains,
> >>>>>> "wpa_supplicant" /var/log/NetworkManager.log ~ =====
> >>>>
> >>>> 1) rsyslogd always pukes on itself if the config file doesn't
> >>>> parse. not new.
> >>>>
> >>>> 2) Its documented that selectors syntax is not a stable API /
> >>>> config file syntax, though, the maintainer should have noted it
> >>>> in UPDATING. I would have hit this tomorrow myself.
> >>>>
> >>>
> >>> You forgot the part, where I said that I remove those lines and
> >>> rsyslog still doesn't log anything.
> >>
> >> And the fact that if it fails to parse, it should complain loudly
> >> and not fail silently.
> >
> > unfortunantly in my experiance it doesn't complain loudly :-(
> >
> > in V5 there is a new option to tell it to exit if it can't read the
> > config.
> 
> Regarding failure to parse the config:
> 
> If you have a config entry of this form:
> 
> *.*	-/var/log/syslog	# Send everything else to syslog
> 
> (i.e. with a trailing comment appended using hash), it doesn't work (on
> 4.5.6, at least - I've observed this with other versions, but I don't
> have a list). The config line is silently ignored.
> 
> The manpage says:
>      "Lines  starting with a hash mark ('#') and empty lines are
>       ignored."
> 
> That's fair enough; but it doesn't mention that lines containing a
> trailing comment will also be ignored (silently).
> 
> Incorrect config lines should elicit a complaint in rsyslogd's output.
> 
> --
> Jack.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 14:28:03 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:28:03 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<alpine.DEB.2.00.1001142252150.4649@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D0@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 15, 2010 7:53 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> > Running rsyslogd -d -c4 I got
> 
> it may not matter, but I think you need to do -c5 with 5.x

No - if you specify -c4, it will start up with the v4 defaults, if you do
-c5, it will start up with the v5 defaults. Nothing else. That's what -c is
for (just the defaults). Note that currently -c4 and -c5, I think, are
aquivalent.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 14:30:14 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:30:14 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D1@GRFEXC.intern.adiscon.com>

> config. One thing that I frequently tripped over when switching back
> and
> forth was th HUPisRestart option. In V5 that's not a valid option
> anymore
> and needs to be removed.

Just FYI: if $HUPisRestart is present in a v5 config, it will generate an
error message, but that's it. No harsh effects (except, of course, if you set
rsyslog to abort on error ;)).




From rgerhards at hq.adiscon.com  Fri Jan 15 14:32:43 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:32:43 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 8:42 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15  <david at lang.hm>:
> 
> > I think I just realized the problem
> >
> > you have
> >
> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> > ~
> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> > ~
> >
> > when you should have
> >
> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
> > & ~
> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
> > & ~
> >
> > give that a shot.
> 
> The error message goes away but rsyslog still logs nothing.
> 
> Interesting fact is, that the above syntax worked fine with 4.4.2

I don't think so, you probably ignored (did not record?) the error message.
The tilde character is an action, and an action needs to be placed after a
filter. So a tilde character just on its own in a single line is definitely a
syntax error. The engine would not know what to do with such a line.

If it generated no error in v4.4.2, *that* was a bug (will verify later).

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 14:36:54 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:36:54 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><4B50252A.1000106@jackpot.uk.net>
	<aa76cdac1001150044k7afed80fkc6151b4cb14eaf45@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D3@GRFEXC.intern.adiscon.com>

ah, that's interesting. The code for pipes (and file output in general) has
been considerably changed, and there was a problem with pipes. I assume that
/dev/xconsole exists? If so, it may fill up and block further processing.

Just to verify, could you try the latest version from the master branch?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 9:45 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Apparently it is that line in my config file, that make rsyslog
> unhappy:
> 
> daemon.*;mail.*;\
>        news.err;\
>        *.=debug;*.=info;\
>        *.=notice;*.=warn       |/dev/xconsole
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 14:39:11 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:39:11 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 8:27 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15  <david at lang.hm>:
> > in V5 there is a new option to tell it to exit if it can't read the
> config.
> >
> > I suspect that the actual config error is significantly earlier in
> the
> > config. One thing that I frequently tripped over when switching back
> and
> > forth was th HUPisRestart option. In V5 that's not a valid option
> anymore
> > and needs to be removed.
> >
> > can you post your full config?
> 
> Here is the rsyslog.conf (default Debian install)
> http://paste.debian.net/56723/

If I am not mistaken, the default Debian config discards rsyslog error
messages - at least I have not spotted any rule that records syslog.err
messages anywhere...

Rainer


From mbiebl at gmail.com  Fri Jan 15 14:45:20 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 14:45:20 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> Sent: Friday, January 15, 2010 8:42 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> 2010/1/15 ?<david at lang.hm>:
>>
>> > I think I just realized the problem
>> >
>> > you have
>> >
>> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> > ~
>> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> > ~
>> >
>> > when you should have
>> >
>> > :programname, contains, "NetworkManager" /var/log/NetworkManager.log
>> > & ~
>> > :programname, contains, "wpa_supplicant" /var/log/NetworkManager.log
>> > & ~
>> >
>> > give that a shot.
>>
>> The error message goes away but rsyslog still logs nothing.
>>
>> Interesting fact is, that the above syntax worked fine with 4.4.2
>
> I don't think so, you probably ignored (did not record?) the error message.
> The tilde character is an action, and an action needs to be placed after a
> filter. So a tilde character just on its own in a single line is definitely a
> syntax error. The engine would not know what to do with such a line.
>
> If it generated no error in v4.4.2, *that* was a bug (will verify later).

It definitely worked with 4.4.2, i.e. the
NetworkManager/wpa_supplicant messages were discarded.

Will have to check if rsyslog wrote any error message in the syslog.

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:47:48 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:47:48 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D6@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 2:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> Sent: Friday, January 15, 2010 8:42 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> 2010/1/15 ?<david at lang.hm>:
> >>
> >> > I think I just realized the problem
> >> >
> >> > you have
> >> >
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > ~
> >> > :programname, contains, "wpa_supplicant"
> /var/log/NetworkManager.log
> >> > ~
> >> >
> >> > when you should have
> >> >
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > & ~
> >> > :programname, contains, "wpa_supplicant"
> /var/log/NetworkManager.log
> >> > & ~
> >> >
> >> > give that a shot.
> >>
> >> The error message goes away but rsyslog still logs nothing.
> >>
> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >
> > I don't think so, you probably ignored (did not record?) the error
> message.
> > The tilde character is an action, and an action needs to be placed
> after a
> > filter. So a tilde character just on its own in a single line is
> definitely a
> > syntax error. The engine would not know what to do with such a line.
> >
> > If it generated no error in v4.4.2, *that* was a bug (will verify
> later).
> 
> It definitely worked with 4.4.2, i.e. the
> NetworkManager/wpa_supplicant messages were discarded.
> 
> Will have to check if rsyslog wrote any error message in the syslog.

OK, thanks, will see where the bug in v4 is. I am right now setting up a new
Debian test env, it's probaly easiest to find the issues using the same
platform as you :)

Rainer


From mbiebl at gmail.com  Fri Jan 15 14:43:09 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 14:43:09 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>
	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>
	<4B501261.1070901@p6m7g8.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150543i47e9a0b2o7ebdc0e0a47dc22d@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>
>> Here is the rsyslog.conf (default Debian install)
>> http://paste.debian.net/56723/
>
> If I am not mistaken, the default Debian config discards rsyslog error
> messages - at least I have not spotted any rule that records syslog.err
> messages anywhere...

*.*;auth,authpriv.none		-/var/log/syslog

should catch syslog errors, right?


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 14:52:56 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 14:52:56 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D4@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150543i47e9a0b2o7ebdc0e0a47dc22d@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D7@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 2:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>
> >> Here is the rsyslog.conf (default Debian install)
> >> http://paste.debian.net/56723/
> >
> > If I am not mistaken, the default Debian config discards rsyslog
> error
> > messages - at least I have not spotted any rule that records
> syslog.err
> > messages anywhere...
> 
> *.*;auth,authpriv.none		-/var/log/syslog
> 
> should catch syslog errors, right?

Oops, I overlooked "*.*". And, indeed, it should catch them.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 15:23:34 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 15:23:34 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>

Michael,

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> > :programname, contains, "NetworkManager"
> /var/log/NetworkManager.log
> >> > ~
> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >
> > I don't think so, you probably ignored (did not record?) the error
> message.
> > The tilde character is an action, and an action needs to be placed
> after a
> > filter. So a tilde character just on its own in a single line is
> definitely a
> > syntax error. The engine would not know what to do with such a line.
> >
> > If it generated no error in v4.4.2, *that* was a bug (will verify
> later).
> 
> It definitely worked with 4.4.2, i.e. the
> NetworkManager/wpa_supplicant messages were discarded.

I used a Debian 5 I had available here, ran apt-get update/upgrade and
compiled rsyslog 4.4.2 from scratch. Then I entered the first line into the
config and restarted rsyslog.

After doing so, I had the relevant errors in /var/log/syslog.

Two observations: 

a) the commands were flagged as invalid by 4.4.2
b) error messages are logged (at least up to 4.4.2)

Note that I had the statements directly in my main config. Can you verify you
get the error messages, too, when you have them directly in the main config?

I'll now see if v5 does not emit the messages...

Rainer




From rgerhards at hq.adiscon.com  Fri Jan 15 15:33:10 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 15:33:10 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com><4B501261.1070901@p6m7g8.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036D9@GRFEXC.intern.adiscon.com>

Michael, 

I could reproduce the original bug report, now a bugzilla entry:

http://bugzilla.adiscon.com/show_bug.cgi?id=169

I guess you don't see any entries in /var/log/syslog simply because rsyslog
hangs and so is unable to process any further message. I suggest you
subscribe to the bug in bugzilla.

Rainer


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 3:24 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> Michael,
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > >> > :programname, contains, "NetworkManager"
> > /var/log/NetworkManager.log
> > >> > ~
> > >> Interesting fact is, that the above syntax worked fine with 4.4.2
> > >
> > > I don't think so, you probably ignored (did not record?) the error
> > message.
> > > The tilde character is an action, and an action needs to be placed
> > after a
> > > filter. So a tilde character just on its own in a single line is
> > definitely a
> > > syntax error. The engine would not know what to do with such a
> line.
> > >
> > > If it generated no error in v4.4.2, *that* was a bug (will verify
> > later).
> >
> > It definitely worked with 4.4.2, i.e. the
> > NetworkManager/wpa_supplicant messages were discarded.
> 
> I used a Debian 5 I had available here, ran apt-get update/upgrade and
> compiled rsyslog 4.4.2 from scratch. Then I entered the first line into
> the
> config and restarted rsyslog.
> 
> After doing so, I had the relevant errors in /var/log/syslog.
> 
> Two observations:
> 
> a) the commands were flagged as invalid by 4.4.2
> b) error messages are logged (at least up to 4.4.2)
> 
> Note that I had the statements directly in my main config. Can you
> verify you
> get the error messages, too, when you have them directly in the main
> config?
> 
> I'll now see if v5 does not emit the messages...
> 
> Rainer
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Fri Jan 15 16:02:30 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 16:02:30 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> >> > :programname, contains, "NetworkManager"
>> /var/log/NetworkManager.log
>> >> > ~
>> >> Interesting fact is, that the above syntax worked fine with 4.4.2
>> >
>> > I don't think so, you probably ignored (did not record?) the error
>> message.
>> > The tilde character is an action, and an action needs to be placed
>> after a
>> > filter. So a tilde character just on its own in a single line is
>> definitely a
>> > syntax error. The engine would not know what to do with such a line.
>> >
>> > If it generated no error in v4.4.2, *that* was a bug (will verify
>> later).
>>
>> It definitely worked with 4.4.2, i.e. the
>> NetworkManager/wpa_supplicant messages were discarded.
>
> I used a Debian 5 I had available here, ran apt-get update/upgrade and
> compiled rsyslog 4.4.2 from scratch. Then I entered the first line into the
> config and restarted rsyslog.
>
> After doing so, I had the relevant errors in /var/log/syslog.
>
> Two observations:
>
> a) the commands were flagged as invalid by 4.4.2
> b) error messages are logged (at least up to 4.4.2)

Yeah, false alarm from my side, sorry.

4.4.2 writes an error message about using incorrect syntax and the log
messages are not dropped when using a simple "~". Everything as it
should be :-)
So this was all a red herring.

The real problem, as you already noticed, the non-working pipe which
causes 5.3.6 to hang and not process any further message.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 16:37:15 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 16:37:15 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>

> Yeah, false alarm from my side, sorry.

No problem - much better a false alarm here and there than no alarm at all.
Thankfully, we could avoid propagating the pipe error into v4-stable, which I
consider very useful :)

Rainer

> 
> 4.4.2 writes an error message about using incorrect syntax and the log
> messages are not dropped when using a simple "~". Everything as it
> should be :-)
> So this was all a red herring.
> 
> The real problem, as you already noticed, the non-working pipe which
> causes 5.3.6 to hang and not process any further message.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Fri Jan 15 16:43:14 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 16:43:14 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001150743p70182910t144d10a068c8358b@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> Yeah, false alarm from my side, sorry.
>
> No problem - much better a false alarm here and there than no alarm at all.
> Thankfully, we could avoid propagating the pipe error into v4-stable, which I
> consider very useful :)

BTW, can you reproduce the problem, that -d no longer produces a
verbose output with 5.3.6?

Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Fri Jan 15 16:45:02 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 16:45:02 +0100
Subject: [rsyslog] -d doesn't work - RE:  rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DA@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150743p70182910t144d10a068c8358b@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DB@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 4:43 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> Yeah, false alarm from my side, sorry.
> >
> > No problem - much better a false alarm here and there than no alarm
> at all.
> > Thankfully, we could avoid propagating the pipe error into v4-stable,
> which I
> > consider very useful :)
> 
> BTW, can you reproduce the problem, that -d no longer produces a
> verbose output with 5.3.6?

Will look into that after the fix. I remember I had this issue and I think it
is fixed. Maybe I forgot to merge some change into v5-beta. This most
probably is a result of the improved runtime debugging support. Sorry I
forgot to comment on this one.

Rainer


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:04:58 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:04:58 +0000
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
Message-ID: <4B50922A.8090900@jackpot.uk.net>

Rainer Gerhards wrote:
> Hi folks,
> 
> jumping right in the middle and looking at one issue at the other ;)
> 
> Please note that nothing is silently ignored. Whenever rsyslog encounters a
> problem, a message is generated. HOWEVER, almost nobody ever looks at the
> messages emitted from the syslog facility and so the error messages are
> "lost". See also:

Rainer,

Consider please this single action line from a simple config:
   *.*                             /var/log/syslog

If that is modified as follows:
   *.*                             /var/log/syslog   # Comment goes here

then (1) no message goes to stdout; (2) nothing gets logged to
/var/log/syslog, because the action line specifying that action is
faulty. The service starts; but given that the defective action line is
the only one in the config, it might as well have failed to start,
because no log output will ever be produced. In particular, messages for 
the syslog facility will not be sent anywhere.

I call that "silent"; as far as I can see, there is absolutely no
message anywhere indicating that the service had any problems with the
config.

# rsyslogd -v
rsyslogd 4.5.6, compiled with:
         FEATURE_REGEXP:                         Yes
         FEATURE_LARGEFILE:                      Yes
         FEATURE_NETZIP (message compression):   Yes
         GSSAPI Kerberos 5 support:              No
         FEATURE_DEBUG (debug build, slow code): No
         Atomic operations supported:            Yes
         Runtime Instrumentation (slow code):    No

I think config parsing problems should be output unconditionally to
stdout; but what do I know :-) Anyway, relying on the logging service to
tell you about a problem with the logging service seems - umm -
over-confident.
-- 
Jack.



From rgerhards at hq.adiscon.com  Fri Jan 15 17:08:36 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:08:36 +0100
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
	(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
	<4B50922A.8090900@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 5:05 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] "silently ignored errors" - RE: rsyslog 5.3.6
> (v5-beta) released
> 
> Rainer Gerhards wrote:
> > Hi folks,
> >
> > jumping right in the middle and looking at one issue at the other ;)
> >
> > Please note that nothing is silently ignored. Whenever rsyslog
> encounters a
> > problem, a message is generated. HOWEVER, almost nobody ever looks at
> the
> > messages emitted from the syslog facility and so the error messages
> are
> > "lost". See also:
> 
> Rainer,
> 
> Consider please this single action line from a simple config:
>    *.*                             /var/log/syslog
> 
> If that is modified as follows:
>    *.*                             /var/log/syslog   # Comment goes
> here
> 
> then (1) no message goes to stdout; (2) nothing gets logged to
> /var/log/syslog, because the action line specifying that action is
> faulty. The service starts; but given that the defective action line is
> the only one in the config, it might as well have failed to start,
> because no log output will ever be produced. In particular, messages
> for
> the syslog facility will not be sent anywhere.
> 
> I call that "silent"; as far as I can see, there is absolutely no
> message anywhere indicating that the service had any problems with the
> config.

That's a problem with the current config syntax. Interestingly hard to fix.

> 
> # rsyslogd -v
> rsyslogd 4.5.6, compiled with:
>          FEATURE_REGEXP:                         Yes
>          FEATURE_LARGEFILE:                      Yes
>          FEATURE_NETZIP (message compression):   Yes
>          GSSAPI Kerberos 5 support:              No
>          FEATURE_DEBUG (debug build, slow code): No
>          Atomic operations supported:            Yes
>          Runtime Instrumentation (slow code):    No
> 
> I think config parsing problems should be output unconditionally to
> stdout; but what do I know :-) Anyway, relying on the logging service
> to
> tell you about a problem with the logging service seems - umm -
> over-confident.

Well, that's the meat of it. So what shall I do? I am asking this question
for roughly 20 months now, and so far obviously did not get a good answer,
nor do I have one. As I wrote, we can already output error messages to
stderr. Would it really help to add another option to send them to stdout as
well? 

All suggestions on how to handle error notifications are *very* welcome.

Rainer


From rgerhards at hq.adiscon.com  Fri Jan 15 17:18:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:18:24 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>

Michael,

Fix now in git, links at the bug tracker:

http://bugzilla.adiscon.com/show_bug.cgi?id=169

Please let me know if it works for you (the patch is a bit trickier than it
looks, so confirmations would be good).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 4:03 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> >> > :programname, contains, "NetworkManager"
> >> /var/log/NetworkManager.log
> >> >> > ~
> >> >> Interesting fact is, that the above syntax worked fine with 4.4.2
> >> >
> >> > I don't think so, you probably ignored (did not record?) the error
> >> message.
> >> > The tilde character is an action, and an action needs to be placed
> >> after a
> >> > filter. So a tilde character just on its own in a single line is
> >> definitely a
> >> > syntax error. The engine would not know what to do with such a
> line.
> >> >
> >> > If it generated no error in v4.4.2, *that* was a bug (will verify
> >> later).
> >>
> >> It definitely worked with 4.4.2, i.e. the
> >> NetworkManager/wpa_supplicant messages were discarded.
> >
> > I used a Debian 5 I had available here, ran apt-get update/upgrade
> and
> > compiled rsyslog 4.4.2 from scratch. Then I entered the first line
> into the
> > config and restarted rsyslog.
> >
> > After doing so, I had the relevant errors in /var/log/syslog.
> >
> > Two observations:
> >
> > a) the commands were flagged as invalid by 4.4.2
> > b) error messages are logged (at least up to 4.4.2)
> 
> Yeah, false alarm from my side, sorry.
> 
> 4.4.2 writes an error message about using incorrect syntax and the log
> messages are not dropped when using a simple "~". Everything as it
> should be :-)
> So this was all a red herring.
> 
> The real problem, as you already noticed, the non-working pipe which
> causes 5.3.6 to hang and not process any further message.
> 
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:23:10 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:23:10 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>
	<4B50922A.8090900@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>

 -----Original Message-----
> If that is modified as follows:
>    *.*                             /var/log/syslog   # Comment goes
> here
> 

It's even worse: data is written to the 
file "/var/log/syslog   # Comment goes here"!

Looks like I need to find a solution at least for omfile. Thanks for bringing
this issue up in this context ;)

Rainer


> then (1) no message goes to stdout; (2) nothing gets logged to
> /var/log/syslog, because the action line specifying that action is
> faulty. The service starts; but given that the defective action line is
> the only one in the config, it might as well have failed to start,
> because no log output will ever be produced. In particular, messages
> for
> the syslog facility will not be sent anywhere.
> 
> I call that "silent"; as far as I can see, there is absolutely no
> message anywhere indicating that the service had any problems with the
> config.
> 
> # rsyslogd -v
> rsyslogd 4.5.6, compiled with:
>          FEATURE_REGEXP:                         Yes
>          FEATURE_LARGEFILE:                      Yes
>          FEATURE_NETZIP (message compression):   Yes
>          GSSAPI Kerberos 5 support:              No
>          FEATURE_DEBUG (debug build, slow code): No
>          Atomic operations supported:            Yes
>          Runtime Instrumentation (slow code):    No
> 
> I think config parsing problems should be output unconditionally to
> stdout; but what do I know :-) Anyway, relying on the logging service
> to
> tell you about a problem with the logging service seems - umm -
> over-confident.
> --
> Jack.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:27:00 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:27:00 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com><4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036DF@GRFEXC.intern.adiscon.com>

bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=170

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 5:23 PM
> To: rsyslog-users
> Subject: [rsyslog] comments in file actions
> 
>  -----Original Message-----
> > If that is modified as follows:
> >    *.*                             /var/log/syslog   # Comment goes
> > here
> >
> 
> It's even worse: data is written to the
> file "/var/log/syslog   # Comment goes here"!
> 
> Looks like I need to find a solution at least for omfile. Thanks for
> bringing
> this issue up in this context ;)
> 
> Rainer
> 
> 
> > then (1) no message goes to stdout; (2) nothing gets logged to
> > /var/log/syslog, because the action line specifying that action is
> > faulty. The service starts; but given that the defective action line
> is
> > the only one in the config, it might as well have failed to start,
> > because no log output will ever be produced. In particular, messages
> > for
> > the syslog facility will not be sent anywhere.
> >
> > I call that "silent"; as far as I can see, there is absolutely no
> > message anywhere indicating that the service had any problems with
> the
> > config.
> >
> > # rsyslogd -v
> > rsyslogd 4.5.6, compiled with:
> >          FEATURE_REGEXP:                         Yes
> >          FEATURE_LARGEFILE:                      Yes
> >          FEATURE_NETZIP (message compression):   Yes
> >          GSSAPI Kerberos 5 support:              No
> >          FEATURE_DEBUG (debug build, slow code): No
> >          Atomic operations supported:            Yes
> >          Runtime Instrumentation (slow code):    No
> >
> > I think config parsing problems should be output unconditionally to
> > stdout; but what do I know :-) Anyway, relying on the logging service
> > to
> > tell you about a problem with the logging service seems - umm -
> > over-confident.
> > --
> > Jack.
> >
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 15 17:36:28 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 Jan 2010 17:36:28 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com><4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DF@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E0@GRFEXC.intern.adiscon.com>

mhh... general question: would anybody object if I would not permit spaces
inside file names? (one could introduce them by using dynafiles with a clever
template if absolutely needed...).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Friday, January 15, 2010 5:27 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=170
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Friday, January 15, 2010 5:23 PM
> > To: rsyslog-users
> > Subject: [rsyslog] comments in file actions
> >
> >  -----Original Message-----
> > > If that is modified as follows:
> > >    *.*                             /var/log/syslog   # Comment goes
> > > here
> > >
> >
> > It's even worse: data is written to the
> > file "/var/log/syslog   # Comment goes here"!
> >
> > Looks like I need to find a solution at least for omfile. Thanks for
> > bringing
> > this issue up in this context ;)
> >
> > Rainer
> >
> >
> > > then (1) no message goes to stdout; (2) nothing gets logged to
> > > /var/log/syslog, because the action line specifying that action is
> > > faulty. The service starts; but given that the defective action
> line
> > is
> > > the only one in the config, it might as well have failed to start,
> > > because no log output will ever be produced. In particular,
> messages
> > > for
> > > the syslog facility will not be sent anywhere.
> > >
> > > I call that "silent"; as far as I can see, there is absolutely no
> > > message anywhere indicating that the service had any problems with
> > the
> > > config.
> > >
> > > # rsyslogd -v
> > > rsyslogd 4.5.6, compiled with:
> > >          FEATURE_REGEXP:                         Yes
> > >          FEATURE_LARGEFILE:                      Yes
> > >          FEATURE_NETZIP (message compression):   Yes
> > >          GSSAPI Kerberos 5 support:              No
> > >          FEATURE_DEBUG (debug build, slow code): No
> > >          Atomic operations supported:            Yes
> > >          Runtime Instrumentation (slow code):    No
> > >
> > > I think config parsing problems should be output unconditionally to
> > > stdout; but what do I know :-) Anyway, relying on the logging
> service
> > > to
> > > tell you about a problem with the logging service seems - umm -
> > > over-confident.
> > > --
> > > Jack.
> > >
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:54:50 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:54:50 +0000
Subject: [rsyslog] "silently ignored errors" - RE: rsyslog
 5.3.6	(v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DC@GRFEXC.intern.adiscon.com>
Message-ID: <4B509DDA.9070207@jackpot.uk.net>

Rainer Gerhards wrote:
>> 
>> I think config parsing problems should be output unconditionally to
>>  stdout; but what do I know :-) Anyway, relying on the logging
>> service to tell you about a problem with the logging service seems
>> - umm - over-confident.
> 
> Well, that's the meat of it. So what shall I do? I am asking this
> question for roughly 20 months now, and so far obviously did not get
> a good answer, nor do I have one. As I wrote, we can already output
> error messages to stderr. Would it really help to add another option
> to send them to stdout as well?

It outputs to stderr? I don't seem to be able to make it do that (as far
as I know, both stderr and stdout should be going to the console).

With the config file containing the invalid action line, I tried this:

# rsyslogd -c4 -N1
rsyslogd: version 4.5.6, config validation run (level 1), master config
/etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

Shouldn't it at least THEN have told me there was something wrong with
the conf? I would expect anything that tells me it's a "config
validation run" to output any errors in the config to the same channel
that message gets printed on.

> 
> All suggestions on how to handle error notifications are *very*
> welcome.
> 

Output to stderr would be fine with me; but I'm not convinced it does that.


-- 
Jack.


From mrdemeanour at jackpot.uk.net  Fri Jan 15 17:58:40 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Fri, 15 Jan 2010 16:58:40 +0000
Subject: [rsyslog] comments in file actions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
Message-ID: <4B509EC0.3050504@jackpot.uk.net>

Rainer Gerhards wrote:
> -----Original Message-----
>> If that is modified as follows: *.*
>> /var/log/syslog   # Comment goes here
>> 
> 
> It's even worse: data is written to the file "/var/log/syslog   #
> Comment goes here"!

Aaaaahhh - that finally explains some odd files lurking in /var/log!

That also explains why there's no error message on stdout/stdderr -
there's no error, as far as rsyslog is concerned. [/me feels stupid]

I would really like to be able to put comments on the ends of config
lines, as I can with many other packages. But now I know what's going on
here. Thanks!

-- 
Jack.


From mbiebl at gmail.com  Fri Jan 15 23:57:08 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Fri, 15 Jan 2010 23:57:08 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>

2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> Fix now in git, links at the bug tracker:
>
> http://bugzilla.adiscon.com/show_bug.cgi?id=169
>
> Please let me know if it works for you (the patch is a bit trickier than it
> looks, so confirmations would be good).

I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
But now I'm getting a crash when rsyslog encounters the xconsole pipe
config.

Michael
-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From sanelson at gmail.com  Sun Jan 17 00:46:59 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sat, 16 Jan 2010 23:46:59 +0000
Subject: [rsyslog] Difference between versions
Message-ID: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>

Hi there,

I've spent a long time picking through changelogs, but I'm afraid I
don't have a clear understanding of what to choose between versions of
rsyslog.

My platform is RHEL 5 - the distro ships with 2.0 which seems to be
both ancient and deprecated.

If I have to build a new package, it would be good to understand which
version I should choose.  For example. Rawhide has a 4.x package,
which would be the obvious starting point.

My overall objective is to be able to aggregate syslog and also Drupal
watchdog logs in a central location, and index them with Solr, to
produce a data mart/wharehouse.

I look forward to enlightenment!

TIA,

S,

-- 
Stephen Nelson-Smith
Technical Director
Atalanta Systems Ltd
www.atalanta-systems.com


From david at lang.hm  Sun Jan 17 01:19:01 2010
From: david at lang.hm (david at lang.hm)
Date: Sat, 16 Jan 2010 16:19:01 -0800 (PST)
Subject: [rsyslog] Difference between versions
In-Reply-To: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>

On Sat, 16 Jan 2010, Stephen Nelson-Smith wrote:

> Hi there,
>
> I've spent a long time picking through changelogs, but I'm afraid I
> don't have a clear understanding of what to choose between versions of
> rsyslog.

2 is ancient, it's only in RHEL because that is the version that was out 
when RHEL5 was released and they never upgrade software (by policy)

3 was the stable about a year ago. This is in Debian 5

4 is after a bunch of rapid developement, it's starting to appear in some 
distros

5 is the current version, it is _much_ faster than previous version.

unfortunantly the current 5.2 'stable' release is known to be very buggy. 
5.3.6 was released a week ago, and it is believed to be the best version. 
several of us are testing it (I put in it production on a couple dozen 
machines in friday, so I should find anything that affects my environment 
by monday). The expectation is that this will replace the broken 5.2 very 
shortly.

so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you 
find anything that doesn't work, post here and you will probably get a fix 
quickly (note that the main developer is in germany, so you do have the 
time zone lag to deal with)

David Lang

> My platform is RHEL 5 - the distro ships with 2.0 which seems to be
> both ancient and deprecated.
>
> If I have to build a new package, it would be good to understand which
> version I should choose.  For example. Rawhide has a 4.x package,
> which would be the obvious starting point.
>
> My overall objective is to be able to aggregate syslog and also Drupal
> watchdog logs in a central location, and index them with Solr, to
> produce a data mart/wharehouse.
>
> I look forward to enlightenment!
>
> TIA,
>
> S,
>
>


From sanelson at gmail.com  Sun Jan 17 09:56:51 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sun, 17 Jan 2010 08:56:51 +0000
Subject: [rsyslog] Difference between versions
In-Reply-To: <alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
Message-ID: <b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>

Hi there,

> 4 is after a bunch of rapid developement, it's starting to appear in some
> distros

So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?

> 5 is the current version, it is _much_ faster than previous version.

Fast is good!

> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you
> find anything that doesn't work, post here and you will probably get a fix
> quickly (note that the main developer is in germany, so you do have the
> time zone lag to deal with)

Right - I'll get to work on a spec file.  Anyone got any gotchas to share?

S.


From david at lang.hm  Sun Jan 17 10:07:37 2010
From: david at lang.hm (david at lang.hm)
Date: Sun, 17 Jan 2010 01:07:37 -0800 (PST)
Subject: [rsyslog] Difference between versions
In-Reply-To: <b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
	<b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
Message-ID: <alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>

On Sun, 17 Jan 2010, Stephen Nelson-Smith wrote:

> Hi there,
>
>> 4 is after a bunch of rapid developement, it's starting to appear in some
>> distros
>
> So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?

4 will go into stable mode like 3 is now. 4 and 5 were under development 
at the same time, but the changes for 5 were so drastic that Rainer didn't 
feel comfortable doing them in the normal development version. 4 settled 
down a few months ago, it looks like 5 is settling down now. I think we 
have already hit one bug that may not end up getting fixed in 4 as the fix 
would be too invasive (when Rainer declares a version stable he is _very_ 
careful about changes to it, even if that means leaving something broken 
to avoid a substantial risk of breaking other things)

people are finding more bugs in rsyslog in recent months, most of the bugs 
that they have been finding are not new bugs, but are instead the result 
of more people useing rsyslog in more different ways (the fact that most 
distros have switched to rsyslog for their next release, if not their last 
release, has drasticly increased it's use)

>> 5 is the current version, it is _much_ faster than previous version.
>
> Fast is good!
>
>> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if you
>> find anything that doesn't work, post here and you will probably get a fix
>> quickly (note that the main developer is in germany, so you do have the
>> time zone lag to deal with)
>
> Right - I'll get to work on a spec file.  Anyone got any gotchas to share?

the big thing is that it is sensitive to config file errors, make sure 
that your startup script doesn't hide such errors from the user.

David Lang


From rgerhards at hq.adiscon.com  Sun Jan 17 11:43:18 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:43:18 +0100
Subject: [rsyslog] FW:  RHEL5 rsyslog 4 rpms
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E2@GRFEXC.intern.adiscon.com>

Hi Stephen,

this message (below) may be useful for you. Maybe you can join forces...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Daniel Anson
> Sent: Thursday, January 07, 2010 7:44 PM
> To: rsyslog-users
> Subject: [rsyslog] RHEL5 rsyslog 4 rpms
> 
> If anyone is interested, an RPM engineer I know has packaged RHEL5
> rsyslog4 rpms.  These are available for public download and testing @
> http://dl.iuscommunity.org/pub/ius  Any comments can be emailed
> directly to him at ius-coredev at lists.launchpad.net
> 
> rpms are regularly packaged by him so let him know what you think.  I
> believe you just have to add the yum repo.
> 
> --Daniel M. Anson
> --Linux Systems Engineer
> 
> 
> 
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use
> of the
> individual or entity to which this message is addressed, and unless
> otherwise
> expressly indicated, is confidential and privileged information of
> Rackspace.
> Any dissemination, distribution or copying of the enclosed material is
> prohibited.
> If you receive this transmission in error, please notify us immediately
> by e-mail
> at abuse at rackspace.com, and delete the original message.
> Your cooperation is appreciated.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Sun Jan 17 11:43:36 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:43:36 +0100
Subject: [rsyslog] Difference between versions
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E3@GRFEXC.intern.adiscon.com>

David,

thanks, that's an excellent summary :)

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Sunday, January 17, 2010 1:19 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Difference between versions
> 
> On Sat, 16 Jan 2010, Stephen Nelson-Smith wrote:
> 
> > Hi there,
> >
> > I've spent a long time picking through changelogs, but I'm afraid I
> > don't have a clear understanding of what to choose between versions
> of
> > rsyslog.
> 
> 2 is ancient, it's only in RHEL because that is the version that was
> out
> when RHEL5 was released and they never upgrade software (by policy)
> 
> 3 was the stable about a year ago. This is in Debian 5
> 
> 4 is after a bunch of rapid developement, it's starting to appear in
> some
> distros
> 
> 5 is the current version, it is _much_ faster than previous version.
> 
> unfortunantly the current 5.2 'stable' release is known to be very
> buggy.
> 5.3.6 was released a week ago, and it is believed to be the best
> version.
> several of us are testing it (I put in it production on a couple dozen
> machines in friday, so I should find anything that affects my
> environment
> by monday). The expectation is that this will replace the broken 5.2
> very
> shortly.
> 
> so if you are compiling anyway, I would suggest giving 5.3.6 a try, if
> you
> find anything that doesn't work, post here and you will probably get a
> fix
> quickly (note that the main developer is in germany, so you do have the
> time zone lag to deal with)
> 
> David Lang
> 
> > My platform is RHEL 5 - the distro ships with 2.0 which seems to be
> > both ancient and deprecated.
> >
> > If I have to build a new package, it would be good to understand
> which
> > version I should choose.  For example. Rawhide has a 4.x package,
> > which would be the obvious starting point.
> >
> > My overall objective is to be able to aggregate syslog and also
> Drupal
> > watchdog logs in a central location, and index them with Solr, to
> > produce a data mart/wharehouse.
> >
> > I look forward to enlightenment!
> >
> > TIA,
> >
> > S,
> >
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Sun Jan 17 11:47:25 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:47:25 +0100
Subject: [rsyslog] Difference between versions
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com><alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm><b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
	<alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Sunday, January 17, 2010 10:08 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Difference between versions
> 
> On Sun, 17 Jan 2010, Stephen Nelson-Smith wrote:
> 
> > Hi there,
> >
> >> 4 is after a bunch of rapid developement, it's starting to appear in
> some
> >> distros
> >
> > So is 4 likely to be discontinued, or will 4 leapfrog 5 and become 6?
> 
> 4 will go into stable mode like 3 is now. 4 and 5 were under
> development
> at the same time, but the changes for 5 were so drastic that Rainer
> didn't
> feel comfortable doing them in the normal development version. 4
> settled
> down a few months ago, it looks like 5 is settling down now. I think we
> have already hit one bug that may not end up getting fixed in 4 as the
> fix
> would be too invasive (when Rainer declares a version stable he is
> _very_
> careful about changes to it, even if that means leaving something
> broken
> to avoid a substantial risk of breaking other things)

Let me elaborate a bit on the v4 bug. There are some situations in the v4
queue engine, that will lead to an unclean shutdown, maybe even a hang
condition (based on the configuration). To fix this, I would need to rewrite
the v4 queue engine very much in the same way as the v5 engine is (minus some
things, but it is a *very* substantial change). Rather than spending time on
that, I accept this issue as it is, and recommend to move to v5 for those few
that are affected. Thankfully, with 5.3.6 we will have a real stable v5 soon.
Note that the v4 bug is *very unlikely* to show up - you need many queues,
various queing params (I don't know all of them out of my head) and it will
happen only very occasionally.

Rainer


From rgerhards at hq.adiscon.com  Sun Jan 17 11:51:16 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 17 Jan 2010 11:51:16 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Friday, January 15, 2010 11:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > Fix now in git, links at the bug tracker:
> >
> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >
> > Please let me know if it works for you (the patch is a bit trickier
> than it
> > looks, so confirmations would be good).
> 
> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> But now I'm getting a crash when rsyslog encounters the xconsole pipe
> config.

I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
stock Debian config?

Rainer

> 
> Michael
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From sanelson at gmail.com  Sun Jan 17 12:26:42 2010
From: sanelson at gmail.com (Stephen Nelson-Smith)
Date: Sun, 17 Jan 2010 11:26:42 +0000
Subject: [rsyslog] Difference between versions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>
References: <b6131fdc1001161546gd002499sf2438a7647122a0f@mail.gmail.com>
	<alpine.DEB.2.00.1001161612540.4831@asgard.lang.hm>
	<b6131fdc1001170056x185718ffh1f0f2498bc8d0ac4@mail.gmail.com>
	<alpine.DEB.2.00.1001170101250.28895@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036E4@GRFEXC.intern.adiscon.com>
Message-ID: <b6131fdc1001170326g64142ed9ta2395883695f45d4@mail.gmail.com>

Hi,
> Thankfully, with 5.3.6 we will have a real stable v5 soon.

OK - you'll have to excuse my weak git fu:

git clone git://git.adiscon.com/git/rsyslog.git
git checkout -b v5.3.6rpm v5.3.6

Is this the place to start from?  I also want to make sure I can pull
in any patches that emerge while I'm working on the package.  Do I do
that with rebase?  But will that rebase from origin/master?  Do I need
to have checked out 5.3.6-devel first?

S.


From mbiebl at gmail.com  Sun Jan 17 12:48:59 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 17 Jan 2010 12:48:59 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>

2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>> Sent: Friday, January 15, 2010 11:57 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>> > Michael,
>> >
>> > Fix now in git, links at the bug tracker:
>> >
>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
>> >
>> > Please let me know if it works for you (the patch is a bit trickier
>> than it
>> > looks, so confirmations would be good).
>>
>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
>> But now I'm getting a crash when rsyslog encounters the xconsole pipe
>> config.
>
> I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
> stock Debian config?

Yes. As said, I just downloaded the 5.3.6 tarball applied the
5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
got the crash. I use the default rsyslog.conf from the official debian
package.
I attached a backtrace. Hope that helps

(gdb) run -c4 -d
Starting program: /usr/sbin/rsyslogd -c4 -d
[Thread debugging using libthread_db enabled]
[New Thread 0xb7df2b70 (LWP 5162)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7df2b70 (LWP 5162)]
qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
pUsr=0x80b72c0) at queue.c:2256
2256            if(pThis->qType != QUEUETYPE_DIRECT) {
(gdb) bt full
#0  qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
pUsr=0x80b72c0) at queue.c:2256
        iRet = <value optimized out>
        iCancelStateSave = <value optimized out>
#1  0x0807cd5b in actionWriteToAction (pAction=0x80ac8d8) at
../action.c:1169
        pMsgSave = 0x0
        iRet = <value optimized out>
#2  0x0807d4fe in doActionCallAction (pAction=0x80ac8d8,
pMsg=0x80b72c0) at ../action.c:1244
No locals.
#3  actionCallAction (pAction=0x80ac8d8, pMsg=0x80b72c0) at ../action.c:1274
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113864, 0, -1210114056, 1464119912, -422385385},
              __mask_was_saved = 0}}, __pad = {0xb7df2240, 0x0,
0xb7df20d0, 0xb7feb27f}}
        __cancel_arg = 0x80ad0a0
        not_first_call = <value optimized out>
        iRet = -1210113864
#4  0x080794d7 in processMsgDoActions (pData=0xfffff815,
pParam=0xb7df20b8) at rule.c:113
        iRet = <value optimized out>
        iRetMod = <value optimized out>
#5  0x080627ba in llExecFunc (pThis=0x80ac9a0, pFunc=0x8079480
<processMsgDoActions>, pParam=0xb7df20b8) at linkedlist.c:391
        iRet = <value optimized out>
        iRetLL = <value optimized out>
        pData = 0x80ac8d8
        llCookie = 0x80ac508
        llCookiePrev = 0x0
#6  0x08079007 in processMsg (pThis=0x80ac968, pMsg=0x80b72c0) at rule.c:299
        bProcessMsg = 1
        DoActData = {bPrevWasSuspended = 0, pMsg = 0x80b72c0}
        iRet = RS_RET_OK
#7  0x080781ba in processMsgDoRules (pData=0x80ac968,
pParam=0x80b72c0) at ruleset.c:145
        iRet = <value optimized out>
#8  0x080627ba in llExecFunc (pThis=0x809eb68, pFunc=0x8078190
<processMsgDoRules>, pParam=0x80b72c0) at linkedlist.c:391
        iRet = <value optimized out>
        iRetLL = <value optimized out>
        pData = 0x80ac968
        llCookie = 0x80ac478
        llCookiePrev = 0x80ac4f8
#9  0x0807876d in processMsg (pMsg=0x80b72c0) at ruleset.c:164
        pThis = <value optimized out>
        iRet = <value optimized out>
#10 0x080506ab in msgConsumer (notNeeded=0x0, pBatch=0x809ecc8,
pbShutdownImmediate=0x80ad348) at syslogd.c:614
        i = 0
        pMsg = 0x80b72c0
        localRet = RS_RET_IO_ERROR
#11 0x08077ca4 in ConsumerReg (pThis=0x80ad338, pWti=0x809ecb0) at queue.c:1638
        iCancelStateSave = 1
        iRet = <value optimized out>
#12 0x08070526 in wtiWorker (pThis=0x809ecb0) at wti.c:286
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113200, 0, -1210113384, 1463800424, -411898089},
              __mask_was_saved = 0}}, __pad = {0xb7df2350, 0x0, 0x0, 0x809002c}}
        not_first_call = <value optimized out>
        pWtp = 0x809ebd8
        bInactivityTOOccured = <value optimized out>
        localRet = <value optimized out>
        terminateRet = RS_RET_OK
        iCancelStateSave = <value optimized out>
#13 0x08070074 in wtpWorker (arg=0x809ecb0) at wtp.c:356
        __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
{134807308, -1210113152, 0, -1210113096, 1463726696, -411345641},
              __mask_was_saved = 0}}, __pad = {0xb7df2460, 0x0,
0xb7feff7b, 0xb7fe0cb0}}
        not_first_call = <value optimized out>
        pszDbgHdr = <value optimized out>
        thrdName = "rs:main Q:Reg", '\000' <repeats 18 times>
---Type <return> to continue, or q <return> to quit---
        pThis = 0x809ebd8
        sigSet = {__val = {2147483647, 4294967294, 4294967295 <repeats
30 times>}}
#14 0xb7f9c585 in start_thread () from /lib/i686/cmov/libpthread.so.0
No symbol table info available.
#15 0xb7f1026e in clone () from /lib/i686/cmov/libc.so.6
No symbol table info available.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From mbiebl at gmail.com  Sun Jan 17 12:51:37 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Sun, 17 Jan 2010 12:51:37 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
Message-ID: <aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>

2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
>>> Sent: Friday, January 15, 2010 11:57 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>>
>>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
>>> > Michael,
>>> >
>>> > Fix now in git, links at the bug tracker:
>>> >
>>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
>>> >
>>> > Please let me know if it works for you (the patch is a bit trickier
>>> than it
>>> > looks, so confirmations would be good).
>>>
>>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
>>> But now I'm getting a crash when rsyslog encounters the xconsole pipe
>>> config.
>>
>> I am a bit puzzled, but will try to reproduce that on my Debian box. I assume
>> stock Debian config?
>
> Yes. As said, I just downloaded the 5.3.6 tarball applied the
> 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> got the crash. I use the default rsyslog.conf from the official debian
> package.

As an additonal hint: If I start xconsole (a process reading from
/dev/xconsole) before I start rsyslogd, then the crash does not occur.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From pgollucci at p6m7g8.com  Sun Jan 17 12:56:07 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Sun, 17 Jan 2010 06:56:07 -0500
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
Message-ID: <4B52FAD7.305@p6m7g8.com>

On 1/17/2010 6:51 AM, Michael Biebl wrote:
> As an additonal hint: If I start xconsole (a process reading from
> /dev/xconsole) before I start rsyslogd, then the crash does not occur.
Possibly related, on FreeBSD in a jail if rsyslog ever tries to write to
/dev/console, it loops in a extremely tightly loop consuming 100% of the
core its on.  [see -dn output, possibly with ktrace/kdump]

Eventually it will consume all the memory on the box and it will go boom.


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Sr. System Admin,                 Ridecharge Inc.
Consultant,                       P6M7G8 Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From ralph at crongeyer.com  Sun Jan 17 23:50:10 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Sun, 17 Jan 2010 17:50:10 -0500
Subject: [rsyslog] fromhost-ip
Message-ID: <4B539422.3020709@crongeyer.com>

Hello list,
I'm trying to send my IPOCop Firewall logs to my rsyslog server like this:

# Firewall logs #
$template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
*.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall

But I just getting this error in /var/log/syslog:

Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
Jan 17 16:49:47 log rsyslogd: the last error occured in 
/etc/rsyslog.d/remote-logs.conf, line 10
Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
will be discarded
Jan 17 16:49:47 log rsyslogd: the last error occured in 
/etc/rsyslog.conf, line 48
Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

I'm trying to log all logs from my IPCop host to 
"/var/log/server-logs/firewall/%HOSTNAME%.log" .

Can someone help me out with this?

Thanks,
Ralph

-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From pgollucci at p6m7g8.com  Mon Jan 18 00:09:22 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Sun, 17 Jan 2010 18:09:22 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B539422.3020709@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>
Message-ID: <4B5398A2.4020604@p6m7g8.com>

On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
> # Firewall logs #
> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
> 
> But I just getting this error in /var/log/syslog:
> 
> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
> swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> /etc/rsyslog.d/remote-logs.conf, line 10
> Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
> will be discarded
> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> /etc/rsyslog.conf, line 48
> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
> master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
> 
> I'm trying to log all logs from my IPCop host to 
> "/var/log/server-logs/firewall/%HOSTNAME%.log" .

I tried for 1.5 days to figure this out cutting and pasting examples
left and right.  Finally I came up with the following with works well
for me, you should be able to tweak it slightly for yourself.


$template by_prog,"/var/log/rws/%programname%.log"

:programname, regex, "^pxy.*rc\."  ?by_prog
& :omrelp:cl.dca1.rws:2514
& ~

Just sub out %programname% for %HOSTNAME%



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Sr. System Admin,                 Ridecharge Inc.
Consultant,                       P6M7G8 Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From ralph at crongeyer.com  Mon Jan 18 16:37:22 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 10:37:22 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B5398A2.4020604@p6m7g8.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
Message-ID: <4B548032.60807@crongeyer.com>

Hi Phillip,
Thanks for the response.
The %HOSTNAME% part works fine here if I do this:
$template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
*.*    -?DynFwall

However if I try to filter by IP using the "fromhost-ip" like this:
*.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

It fails to capture logs in the DynFwall template file.

I've tried to do this with the "fromhost" and the "fromhost-ip" and 
neither seem to work?

I want to have it so that a specific host IP uses a specific template.

It looks like the fromhost and the fromhost-ip arn't working at all? Or 
my config is wrong.

Dose anyone on the list have "fromhost-ip" working?

Thanks,
Ralph

Philip M. Gollucci wrote:
> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>   
>> # Firewall logs #
>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>
>> But I just getting this error in /var/log/syslog:
>>
>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
>> swVersion="4.4.2" x-pid="12540" x-info="http://www.rsyslog.com"] (re)start
>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>> /etc/rsyslog.d/remote-logs.conf, line 10
>> Jan 17 16:49:47 log rsyslogd: warning: selector line without actions 
>> will be discarded
>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>> /etc/rsyslog.conf, line 48
>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not interpret 
>> master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
>>
>> I'm trying to log all logs from my IPCop host to 
>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>     
>
> I tried for 1.5 days to figure this out cutting and pasting examples
> left and right.  Finally I came up with the following with works well
> for me, you should be able to tweak it slightly for yourself.
>
>
> $template by_prog,"/var/log/rws/%programname%.log"
>
> :programname, regex, "^pxy.*rc\."  ?by_prog
> & :omrelp:cl.dca1.rws:2514
> & ~
>
> Just sub out %programname% for %HOSTNAME%
>
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Mon Jan 18 17:24:20 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 17:24:20 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph 
> Crongeyer
> Sent: Monday, January 18, 2010 4:37 PM
> To: Philip M. Gollucci
> Cc: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> Hi Phillip,
> Thanks for the response.
> The %HOSTNAME% part works fine here if I do this:
> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> *.*    -?DynFwall

Phillip suggested the rigth thing. 
> 
> However if I try to filter by IP using the "fromhost-ip" like this:
> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

The issue is that the config is wrong. "*.*" and ":fromhost..." are both
filters. There can only be one filter in front of an action. As *.* maeans
all messages, I assume ou actually wanted to do this:

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall

Which filters alls messages based on fromhost-ip.

The config format is clumpsy. I am currently talking with some folks at
Adiscon, and we will probably create a cookbook-type doc that provides
samples for some common scenarios. I guess that would be useful. Any feedback
on that effort would be welcome.

Rainer

> 
> It fails to capture logs in the DynFwall template file.
> 
> I've tried to do this with the "fromhost" and the "fromhost-ip" and 
> neither seem to work?
> 
> I want to have it so that a specific host IP uses a specific template.
> 
> It looks like the fromhost and the fromhost-ip arn't working 
> at all? Or 
> my config is wrong.
> 
> Dose anyone on the list have "fromhost-ip" working?
> 
> Thanks,
> Ralph
> 
> Philip M. Gollucci wrote:
> > On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
> >   
> >> # Firewall logs #
> >> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
> >> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
> >>
> >> But I just getting this error in /var/log/syslog:
> >>
> >> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
> >> swVersion="4.4.2" x-pid="12540" 
> x-info="http://www.rsyslog.com"] (re)start
> >> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> >> /etc/rsyslog.d/remote-logs.conf, line 10
> >> Jan 17 16:49:47 log rsyslogd: warning: selector line 
> without actions 
> >> will be discarded
> >> Jan 17 16:49:47 log rsyslogd: the last error occured in 
> >> /etc/rsyslog.conf, line 48
> >> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not 
> interpret 
> >> master config file '/etc/rsyslog.conf'. [try 
> http://www.rsyslog.com/e/2124 ]
> >>
> >> I'm trying to log all logs from my IPCop host to 
> >> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
> >>     
> >
> > I tried for 1.5 days to figure this out cutting and pasting examples
> > left and right.  Finally I came up with the following with 
> works well
> > for me, you should be able to tweak it slightly for yourself.
> >
> >
> > $template by_prog,"/var/log/rws/%programname%.log"
> >
> > :programname, regex, "^pxy.*rc\."  ?by_prog
> > & :omrelp:cl.dca1.rws:2514
> > & ~
> >
> > Just sub out %programname% for %HOSTNAME%
> >
> >
> >
> >   
> 
> 
> -- 
> Reminds me of my expedition into the wilds of Afghanistan. We 
> lost our 
> corkscrew and were compelled to live on food and water for 
> several days. - 
> WC Fields
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ralph at crongeyer.com  Mon Jan 18 18:18:18 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 12:18:18 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
Message-ID: <4B5497DA.9070405@crongeyer.com>

Hi Rainer,
Thanks for the explanation, that helps me understand how it's working.

That works, the logs are going to the correct file, however they are 
also being sent to /var/log/syslog? How can I make all the logs from my 
host "192.168.1.1" go only to the "-?DynFwall" template file?

I would like to give feedback on the cookbook let me know how I can help.

Thanks all, for your help with this.
Ralph

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com 
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph 
>> Crongeyer
>> Sent: Monday, January 18, 2010 4:37 PM
>> To: Philip M. Gollucci
>> Cc: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> Hi Phillip,
>> Thanks for the response.
>> The %HOSTNAME% part works fine here if I do this:
>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>> *.*    -?DynFwall
>>     
>
> Phillip suggested the rigth thing. 
>   
>> However if I try to filter by IP using the "fromhost-ip" like this:
>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>     
>
> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
> filters. There can only be one filter in front of an action. As *.* maeans
> all messages, I assume ou actually wanted to do this:
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>
> Which filters alls messages based on fromhost-ip.
>
> The config format is clumpsy. I am currently talking with some folks at
> Adiscon, and we will probably create a cookbook-type doc that provides
> samples for some common scenarios. I guess that would be useful. Any feedback
> on that effort would be welcome.
>
> Rainer
>
>   
>> It fails to capture logs in the DynFwall template file.
>>
>> I've tried to do this with the "fromhost" and the "fromhost-ip" and 
>> neither seem to work?
>>
>> I want to have it so that a specific host IP uses a specific template.
>>
>> It looks like the fromhost and the fromhost-ip arn't working 
>> at all? Or 
>> my config is wrong.
>>
>> Dose anyone on the list have "fromhost-ip" working?
>>
>> Thanks,
>> Ralph
>>
>> Philip M. Gollucci wrote:
>>     
>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>   
>>>       
>>>> # Firewall logs #
>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>
>>>> But I just getting this error in /var/log/syslog:
>>>>
>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd" 
>>>> swVersion="4.4.2" x-pid="12540" 
>>>>         
>> x-info="http://www.rsyslog.com"] (re)start
>>     
>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line 
>>>>         
>> without actions 
>>     
>>>> will be discarded
>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in 
>>>> /etc/rsyslog.conf, line 48
>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not 
>>>>         
>> interpret 
>>     
>>>> master config file '/etc/rsyslog.conf'. [try 
>>>>         
>> http://www.rsyslog.com/e/2124 ]
>>     
>>>> I'm trying to log all logs from my IPCop host to 
>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>     
>>>>         
>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>> left and right.  Finally I came up with the following with 
>>>       
>> works well
>>     
>>> for me, you should be able to tweak it slightly for yourself.
>>>
>>>
>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>
>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>> & :omrelp:cl.dca1.rws:2514
>>> & ~
>>>
>>> Just sub out %programname% for %HOSTNAME%
>>>
>>>
>>>
>>>   
>>>       
>> -- 
>> Reminds me of my expedition into the wilds of Afghanistan. We 
>> lost our 
>> corkscrew and were compelled to live on food and water for 
>> several days. - 
>> WC Fields
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 18:29:02 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 09:29:02 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B5497DA.9070405@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Hi Rainer,
> Thanks for the explanation, that helps me understand how it's working.
>
> That works, the logs are going to the correct file, however they are
> also being sent to /var/log/syslog? How can I make all the logs from my
> host "192.168.1.1" go only to the "-?DynFwall" template file?

after you tell rsyslog to put the logs in that file, you then need to tell 
rsyslog to throw the log away.

so you would do something like

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
& ~

which is logicly the same as

:fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
:fromhost-ip,isequal,"192.168.1.1"    ~

David Lang


> I would like to give feedback on the cookbook let me know how I can help.
>
> Thanks all, for your help with this.
> Ralph
>
> Rainer Gerhards wrote:
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>> Crongeyer
>>> Sent: Monday, January 18, 2010 4:37 PM
>>> To: Philip M. Gollucci
>>> Cc: rsyslog-users
>>> Subject: Re: [rsyslog] fromhost-ip
>>>
>>> Hi Phillip,
>>> Thanks for the response.
>>> The %HOSTNAME% part works fine here if I do this:
>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>> *.*    -?DynFwall
>>>
>>
>> Phillip suggested the rigth thing.
>>
>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>
>>
>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>> filters. There can only be one filter in front of an action. As *.* maeans
>> all messages, I assume ou actually wanted to do this:
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>
>> Which filters alls messages based on fromhost-ip.
>>
>> The config format is clumpsy. I am currently talking with some folks at
>> Adiscon, and we will probably create a cookbook-type doc that provides
>> samples for some common scenarios. I guess that would be useful. Any feedback
>> on that effort would be welcome.
>>
>> Rainer
>>
>>
>>> It fails to capture logs in the DynFwall template file.
>>>
>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>> neither seem to work?
>>>
>>> I want to have it so that a specific host IP uses a specific template.
>>>
>>> It looks like the fromhost and the fromhost-ip arn't working
>>> at all? Or
>>> my config is wrong.
>>>
>>> Dose anyone on the list have "fromhost-ip" working?
>>>
>>> Thanks,
>>> Ralph
>>>
>>> Philip M. Gollucci wrote:
>>>
>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>
>>>>
>>>>> # Firewall logs #
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>
>>>>> But I just getting this error in /var/log/syslog:
>>>>>
>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>
>>> x-info="http://www.rsyslog.com"] (re)start
>>>
>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>
>>> without actions
>>>
>>>>> will be discarded
>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>> /etc/rsyslog.conf, line 48
>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>
>>> interpret
>>>
>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>
>>> http://www.rsyslog.com/e/2124 ]
>>>
>>>>> I'm trying to log all logs from my IPCop host to
>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>
>>>>>
>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>> left and right.  Finally I came up with the following with
>>>>
>>> works well
>>>
>>>> for me, you should be able to tweak it slightly for yourself.
>>>>
>>>>
>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>
>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>> & :omrelp:cl.dca1.rws:2514
>>>> & ~
>>>>
>>>> Just sub out %programname% for %HOSTNAME%
>>>>
>>>>
>>>>
>>>>
>>>>
>>> --
>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>> lost our
>>> corkscrew and were compelled to live on food and water for
>>> several days. -
>>> WC Fields
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 18:47:03 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 12:47:03 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
Message-ID: <4B549E97.8030108@crongeyer.com>

Oh,
I tried that but I had it on the same line. So that has to be on a 
separate line?

Thanks again for the explanation that really helps me understand how 
it's working.

Thanks again for all your help with this.

Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Hi Rainer,
>> Thanks for the explanation, that helps me understand how it's working.
>>
>> That works, the logs are going to the correct file, however they are
>> also being sent to /var/log/syslog? How can I make all the logs from my
>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>     
>
> after you tell rsyslog to put the logs in that file, you then need to tell 
> rsyslog to throw the log away.
>
> so you would do something like
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
> & ~
>
> which is logicly the same as
>
> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
> :fromhost-ip,isequal,"192.168.1.1"    ~
>
> David Lang
>
>
>   
>> I would like to give feedback on the cookbook let me know how I can help.
>>
>> Thanks all, for your help with this.
>> Ralph
>>
>> Rainer Gerhards wrote:
>>     
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>> Crongeyer
>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>> To: Philip M. Gollucci
>>>> Cc: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> Hi Phillip,
>>>> Thanks for the response.
>>>> The %HOSTNAME% part works fine here if I do this:
>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>> *.*    -?DynFwall
>>>>
>>>>         
>>> Phillip suggested the rigth thing.
>>>
>>>       
>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>>         
>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>> filters. There can only be one filter in front of an action. As *.* maeans
>>> all messages, I assume ou actually wanted to do this:
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>
>>> Which filters alls messages based on fromhost-ip.
>>>
>>> The config format is clumpsy. I am currently talking with some folks at
>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>> on that effort would be welcome.
>>>
>>> Rainer
>>>
>>>
>>>       
>>>> It fails to capture logs in the DynFwall template file.
>>>>
>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>> neither seem to work?
>>>>
>>>> I want to have it so that a specific host IP uses a specific template.
>>>>
>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>> at all? Or
>>>> my config is wrong.
>>>>
>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>
>>>> Thanks,
>>>> Ralph
>>>>
>>>> Philip M. Gollucci wrote:
>>>>
>>>>         
>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> # Firewall logs #
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>
>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>
>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>
>>>>>>             
>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>
>>>>         
>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>
>>>>>>             
>>>> without actions
>>>>
>>>>         
>>>>>> will be discarded
>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>> /etc/rsyslog.conf, line 48
>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>
>>>>>>             
>>>> interpret
>>>>
>>>>         
>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>
>>>>>>             
>>>> http://www.rsyslog.com/e/2124 ]
>>>>
>>>>         
>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>
>>>>>>
>>>>>>             
>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>> left and right.  Finally I came up with the following with
>>>>>
>>>>>           
>>>> works well
>>>>
>>>>         
>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>
>>>>>
>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>
>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>> & :omrelp:cl.dca1.rws:2514
>>>>> & ~
>>>>>
>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> --
>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>> lost our
>>>> corkscrew and were compelled to live on food and water for
>>>> several days. -
>>>> WC Fields
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 19:15:49 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 13:15:49 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B549E97.8030108@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com>
Message-ID: <4B54A555.9010007@crongeyer.com>

Ok one more question.
I have:
$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
mail.*  -?DynMail

Which logs all mail to the %HOSTNAME%.mail.log.

My guess would be:
$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail

But as Rainer explained these are both filters which won't work.

So how do I use "fromhost-ip" to send only "mail.*" logs from a 
specified host IP to the "DynMail" template?

Thanks,
Ralph

Ralph Crongeyer wrote:
> Oh,
> I tried that but I had it on the same line. So that has to be on a 
> separate line?
>
> Thanks again for the explanation that really helps me understand how 
> it's working.
>
> Thanks again for all your help with this.
>
> Ralph
>
> david at lang.hm wrote:
>   
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>   
>>     
>>> Hi Rainer,
>>> Thanks for the explanation, that helps me understand how it's working.
>>>
>>> That works, the logs are going to the correct file, however they are
>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>     
>>>       
>> after you tell rsyslog to put the logs in that file, you then need to tell 
>> rsyslog to throw the log away.
>>
>> so you would do something like
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> & ~
>>
>> which is logicly the same as
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>
>> David Lang
>>
>>
>>   
>>     
>>> I would like to give feedback on the cookbook let me know how I can help.
>>>
>>> Thanks all, for your help with this.
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>     
>>>       
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>> Crongeyer
>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>> To: Philip M. Gollucci
>>>>> Cc: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> Hi Phillip,
>>>>> Thanks for the response.
>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.*    -?DynFwall
>>>>>
>>>>>         
>>>>>           
>>>> Phillip suggested the rigth thing.
>>>>
>>>>       
>>>>         
>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>>         
>>>>>           
>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>> all messages, I assume ou actually wanted to do this:
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>> Which filters alls messages based on fromhost-ip.
>>>>
>>>> The config format is clumpsy. I am currently talking with some folks at
>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>> on that effort would be welcome.
>>>>
>>>> Rainer
>>>>
>>>>
>>>>       
>>>>         
>>>>> It fails to capture logs in the DynFwall template file.
>>>>>
>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>> neither seem to work?
>>>>>
>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>
>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>> at all? Or
>>>>> my config is wrong.
>>>>>
>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>
>>>>> Thanks,
>>>>> Ralph
>>>>>
>>>>> Philip M. Gollucci wrote:
>>>>>
>>>>>         
>>>>>           
>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>           
>>>>>>             
>>>>>>> # Firewall logs #
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>
>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>
>>>>>         
>>>>>           
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> without actions
>>>>>
>>>>>         
>>>>>           
>>>>>>> will be discarded
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> interpret
>>>>>
>>>>>         
>>>>>           
>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>
>>>>>         
>>>>>           
>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>
>>>>>>>
>>>>>>>             
>>>>>>>               
>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>> left and right.  Finally I came up with the following with
>>>>>>
>>>>>>           
>>>>>>             
>>>>> works well
>>>>>
>>>>>         
>>>>>           
>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>
>>>>>>
>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>
>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>> & ~
>>>>>>
>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>           
>>>>>>             
>>>>> --
>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>> lost our
>>>>> corkscrew and were compelled to live on food and water for
>>>>> several days. -
>>>>> WC Fields
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>       
>>>>         
>>>     
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 20:14:36 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 14:14:36 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54A555.9010007@crongeyer.com>
References: <4B539422.3020709@crongeyer.com>	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>
Message-ID: <4B54B31C.80109@crongeyer.com>

Is it possible to use "fromhost-ip" to send only "mail.*" logs from a 
specified host IP to the "DynMail" template?

I tried this:
:fromhost-ip,isequal,"192.168.1.1" & mail.* -?DynMail

But that didn't work.

How can I accomplish this?

Thanks,
Ralph
 
Ralph Crongeyer wrote:
> Ok one more question.
> I have:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  -?DynMail
>
> Which logs all mail to the %HOSTNAME%.mail.log.
>
> My guess would be:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>
> But as Rainer explained these are both filters which won't work.
>
> So how do I use "fromhost-ip" to send only "mail.*" logs from a 
> specified host IP to the "DynMail" template?
>
> Thanks,
> Ralph
>
> Ralph Crongeyer wrote:
>   
>> Oh,
>> I tried that but I had it on the same line. So that has to be on a 
>> separate line?
>>
>> Thanks again for the explanation that really helps me understand how 
>> it's working.
>>
>> Thanks again for all your help with this.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>   
>>     
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>   
>>>     
>>>       
>>>> Hi Rainer,
>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>
>>>> That works, the logs are going to the correct file, however they are
>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>     
>>>>       
>>>>         
>>> after you tell rsyslog to put the logs in that file, you then need to tell 
>>> rsyslog to throw the log away.
>>>
>>> so you would do something like
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> & ~
>>>
>>> which is logicly the same as
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>
>>> David Lang
>>>
>>>
>>>   
>>>     
>>>       
>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>
>>>> Thanks all, for your help with this.
>>>> Ralph
>>>>
>>>> Rainer Gerhards wrote:
>>>>     
>>>>       
>>>>         
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>> Crongeyer
>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>> To: Philip M. Gollucci
>>>>>> Cc: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> Hi Phillip,
>>>>>> Thanks for the response.
>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.*    -?DynFwall
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> Phillip suggested the rigth thing.
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>> all messages, I assume ou actually wanted to do this:
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>> Which filters alls messages based on fromhost-ip.
>>>>>
>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>> on that effort would be welcome.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>
>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>> neither seem to work?
>>>>>>
>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>
>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>> at all? Or
>>>>>> my config is wrong.
>>>>>>
>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>
>>>>>> Thanks,
>>>>>> Ralph
>>>>>>
>>>>>> Philip M. Gollucci wrote:
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>>>> # Firewall logs #
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>
>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> without actions
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> will be discarded
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> interpret
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>
>>>>>>>>
>>>>>>>>             
>>>>>>>>               
>>>>>>>>                 
>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> works well
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>
>>>>>>>
>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>
>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>> & ~
>>>>>>>
>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>           
>>>>>>>             
>>>>>>>               
>>>>>> --
>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>> lost our
>>>>>> corkscrew and were compelled to live on food and water for
>>>>>> several days. -
>>>>>> WC Fields
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>         
>>>>>>           
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>       
>>>>>         
>>>>>           
>>>>     
>>>>       
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>   
>>>     
>>>       
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 20:49:39 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 11:49:39 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B549E97.8030108@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181149010.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Oh,
> I tried that but I had it on the same line. So that has to be on a
> separate line?

yes, one line is a filter plus an action

haveing two filters on a line (like you initially tried) doesn't work, 
neither does having two actions on a line.

David Lang

> Thanks again for the explanation that really helps me understand how
> it's working.
>
> Thanks again for all your help with this.
>
> Ralph
>
> david at lang.hm wrote:
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>
>>> Hi Rainer,
>>> Thanks for the explanation, that helps me understand how it's working.
>>>
>>> That works, the logs are going to the correct file, however they are
>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>
>>
>> after you tell rsyslog to put the logs in that file, you then need to tell
>> rsyslog to throw the log away.
>>
>> so you would do something like
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> & ~
>>
>> which is logicly the same as
>>
>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>
>> David Lang
>>
>>
>>
>>> I would like to give feedback on the cookbook let me know how I can help.
>>>
>>> Thanks all, for your help with this.
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>> Crongeyer
>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>> To: Philip M. Gollucci
>>>>> Cc: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> Hi Phillip,
>>>>> Thanks for the response.
>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>> *.*    -?DynFwall
>>>>>
>>>>>
>>>> Phillip suggested the rigth thing.
>>>>
>>>>
>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>>
>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>> all messages, I assume ou actually wanted to do this:
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>
>>>> Which filters alls messages based on fromhost-ip.
>>>>
>>>> The config format is clumpsy. I am currently talking with some folks at
>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>> on that effort would be welcome.
>>>>
>>>> Rainer
>>>>
>>>>
>>>>
>>>>> It fails to capture logs in the DynFwall template file.
>>>>>
>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>> neither seem to work?
>>>>>
>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>
>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>> at all? Or
>>>>> my config is wrong.
>>>>>
>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>
>>>>> Thanks,
>>>>> Ralph
>>>>>
>>>>> Philip M. Gollucci wrote:
>>>>>
>>>>>
>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> # Firewall logs #
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>
>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>
>>>>>>>
>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>
>>>>>
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>
>>>>>>>
>>>>> without actions
>>>>>
>>>>>
>>>>>>> will be discarded
>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>
>>>>>>>
>>>>> interpret
>>>>>
>>>>>
>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>
>>>>>>>
>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>
>>>>>
>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>> left and right.  Finally I came up with the following with
>>>>>>
>>>>>>
>>>>> works well
>>>>>
>>>>>
>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>
>>>>>>
>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>
>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>> & ~
>>>>>>
>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>> lost our
>>>>> corkscrew and were compelled to live on food and water for
>>>>> several days. -
>>>>> WC Fields
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From david at lang.hm  Mon Jan 18 20:57:41 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 11:57:41 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54A555.9010007@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Ok one more question.
> I have:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  -?DynMail
>
> Which logs all mail to the %HOSTNAME%.mail.log.
>
> My guess would be:
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>
> But as Rainer explained these are both filters which won't work.
>
> So how do I use "fromhost-ip" to send only "mail.*" logs from a
> specified host IP to the "DynMail" template?

you need to use the more powerful/complex

if ((condition) and (condition)) action

line format

David Lang

> Thanks,
> Ralph
>
> Ralph Crongeyer wrote:
>> Oh,
>> I tried that but I had it on the same line. So that has to be on a
>> separate line?
>>
>> Thanks again for the explanation that really helps me understand how
>> it's working.
>>
>> Thanks again for all your help with this.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>
>>>
>>>> Hi Rainer,
>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>
>>>> That works, the logs are going to the correct file, however they are
>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>
>>>>
>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>> rsyslog to throw the log away.
>>>
>>> so you would do something like
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> & ~
>>>
>>> which is logicly the same as
>>>
>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>
>>> David Lang
>>>
>>>
>>>
>>>
>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>
>>>> Thanks all, for your help with this.
>>>> Ralph
>>>>
>>>> Rainer Gerhards wrote:
>>>>
>>>>
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>> Crongeyer
>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>> To: Philip M. Gollucci
>>>>>> Cc: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> Hi Phillip,
>>>>>> Thanks for the response.
>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>> *.*    -?DynFwall
>>>>>>
>>>>>>
>>>>>>
>>>>> Phillip suggested the rigth thing.
>>>>>
>>>>>
>>>>>
>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>>
>>>>>>
>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>> all messages, I assume ou actually wanted to do this:
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>
>>>>> Which filters alls messages based on fromhost-ip.
>>>>>
>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>> on that effort would be welcome.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>
>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>> neither seem to work?
>>>>>>
>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>
>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>> at all? Or
>>>>>> my config is wrong.
>>>>>>
>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>
>>>>>> Thanks,
>>>>>> Ralph
>>>>>>
>>>>>> Philip M. Gollucci wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> # Firewall logs #
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>
>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> without actions
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> will be discarded
>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> interpret
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> works well
>>>>>>
>>>>>>
>>>>>>
>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>
>>>>>>>
>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>
>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>> & ~
>>>>>>>
>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>> lost our
>>>>>> corkscrew and were compelled to live on food and water for
>>>>>> several days. -
>>>>>> WC Fields
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 21:37:03 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 15:37:03 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
Message-ID: <4B54C66F.80506@crongeyer.com>

Thanks David,
Ok so now I'm trying this:

$template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then 
?DynMail

After a restart of rsyslog there are no errors in /var/log/syslog 
however no logs are being collected?

Thanks for your help with this David.

Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Ok one more question.
>> I have:
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> mail.*  -?DynMail
>>
>> Which logs all mail to the %HOSTNAME%.mail.log.
>>
>> My guess would be:
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>
>> But as Rainer explained these are both filters which won't work.
>>
>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>> specified host IP to the "DynMail" template?
>>     
>
> you need to use the more powerful/complex
>
> if ((condition) and (condition)) action
>
> line format
>
> David Lang
>
>   
>> Thanks,
>> Ralph
>>
>> Ralph Crongeyer wrote:
>>     
>>> Oh,
>>> I tried that but I had it on the same line. So that has to be on a
>>> separate line?
>>>
>>> Thanks again for the explanation that really helps me understand how
>>> it's working.
>>>
>>> Thanks again for all your help with this.
>>>
>>> Ralph
>>>
>>> david at lang.hm wrote:
>>>
>>>       
>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Hi Rainer,
>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>
>>>>> That works, the logs are going to the correct file, however they are
>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>
>>>>>
>>>>>           
>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>> rsyslog to throw the log away.
>>>>
>>>> so you would do something like
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>> & ~
>>>>
>>>> which is logicly the same as
>>>>
>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>>
>>>>         
>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>
>>>>> Thanks all, for your help with this.
>>>>> Ralph
>>>>>
>>>>> Rainer Gerhards wrote:
>>>>>
>>>>>
>>>>>           
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>> Crongeyer
>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>> To: Philip M. Gollucci
>>>>>>> Cc: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>
>>>>>>> Hi Phillip,
>>>>>>> Thanks for the response.
>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>> *.*    -?DynFwall
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Phillip suggested the rigth thing.
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>
>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>
>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>> on that effort would be welcome.
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>
>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>> neither seem to work?
>>>>>>>
>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>
>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>> at all? Or
>>>>>>> my config is wrong.
>>>>>>>
>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ralph
>>>>>>>
>>>>>>> Philip M. Gollucci wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> # Firewall logs #
>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>
>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> without actions
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> will be discarded
>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> interpret
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> works well
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>
>>>>>>>>
>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>
>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>> & ~
>>>>>>>>
>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> --
>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>> lost our
>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>> several days. -
>>>>>>> WC Fields
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 21:41:26 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 12:41:26 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54C66F.80506@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
	<4B54C66F.80506@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> Thanks David,
> Ok so now I'm trying this:
>
> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
> if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then
> ?DynMail

you can't use single quotes, you must use double quotes (apparently the 
config language uses single quotes for something else, I don't know what)

I've tripped over this several times now.

David Lang

> After a restart of rsyslog there are no errors in /var/log/syslog
> however no logs are being collected?
>
> Thanks for your help with this David.
>
> Ralph
>
> david at lang.hm wrote:
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>
>>> Ok one more question.
>>> I have:
>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>> mail.*  -?DynMail
>>>
>>> Which logs all mail to the %HOSTNAME%.mail.log.
>>>
>>> My guess would be:
>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>>
>>> But as Rainer explained these are both filters which won't work.
>>>
>>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>>> specified host IP to the "DynMail" template?
>>>
>>
>> you need to use the more powerful/complex
>>
>> if ((condition) and (condition)) action
>>
>> line format
>>
>> David Lang
>>
>>
>>> Thanks,
>>> Ralph
>>>
>>> Ralph Crongeyer wrote:
>>>
>>>> Oh,
>>>> I tried that but I had it on the same line. So that has to be on a
>>>> separate line?
>>>>
>>>> Thanks again for the explanation that really helps me understand how
>>>> it's working.
>>>>
>>>> Thanks again for all your help with this.
>>>>
>>>> Ralph
>>>>
>>>> david at lang.hm wrote:
>>>>
>>>>
>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi Rainer,
>>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>>
>>>>>> That works, the logs are going to the correct file, however they are
>>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>>
>>>>>>
>>>>>>
>>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>>> rsyslog to throw the log away.
>>>>>
>>>>> so you would do something like
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>> & ~
>>>>>
>>>>> which is logicly the same as
>>>>>
>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>>
>>>>> David Lang
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>>
>>>>>> Thanks all, for your help with this.
>>>>>> Ralph
>>>>>>
>>>>>> Rainer Gerhards wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>>> Crongeyer
>>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>>> To: Philip M. Gollucci
>>>>>>>> Cc: rsyslog-users
>>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>>
>>>>>>>> Hi Phillip,
>>>>>>>> Thanks for the response.
>>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>> *.*    -?DynFwall
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Phillip suggested the rigth thing.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>>
>>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>
>>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>>
>>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>>> on that effort would be welcome.
>>>>>>>
>>>>>>> Rainer
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>>
>>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>>> neither seem to work?
>>>>>>>>
>>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>>
>>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>>> at all? Or
>>>>>>>> my config is wrong.
>>>>>>>>
>>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ralph
>>>>>>>>
>>>>>>>> Philip M. Gollucci wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> # Firewall logs #
>>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>>
>>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>>
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> without actions
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> will be discarded
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> interpret
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> works well
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>>
>>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>>> & ~
>>>>>>>>>
>>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>>> lost our
>>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>>> several days. -
>>>>>>>> WC Fields
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 21:52:32 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 15:52:32 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>	<4B54C66F.80506@crongeyer.com>
	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
Message-ID: <4B54CA10.2060103@crongeyer.com>

When I switched to double quotes I get the error in /var/log/syslog and 
no logs are collected?
I switched back to single quots and restart and no error but still no logs?

What else may I be doing wrong?

Thanks,
Ralph

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> Thanks David,
>> Ok so now I'm trying this:
>>
>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>> if $fromhost-ip == '192.168.1.1' and $syslogfacility-text == 'mail' then
>> ?DynMail
>>     
>
> you can't use single quotes, you must use double quotes (apparently the 
> config language uses single quotes for something else, I don't know what)
>
> I've tripped over this several times now.
>
> David Lang
>
>   
>> After a restart of rsyslog there are no errors in /var/log/syslog
>> however no logs are being collected?
>>
>> Thanks for your help with this David.
>>
>> Ralph
>>
>> david at lang.hm wrote:
>>     
>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>
>>>
>>>       
>>>> Ok one more question.
>>>> I have:
>>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>>> mail.*  -?DynMail
>>>>
>>>> Which logs all mail to the %HOSTNAME%.mail.log.
>>>>
>>>> My guess would be:
>>>> $template DynMail,"/var/log/server-logs/mail/%HOSTNAME%.mail.log"
>>>> mail.*  :fromhost-ip,isequal,"192.168.1.1" -?DynMail
>>>>
>>>> But as Rainer explained these are both filters which won't work.
>>>>
>>>> So how do I use "fromhost-ip" to send only "mail.*" logs from a
>>>> specified host IP to the "DynMail" template?
>>>>
>>>>         
>>> you need to use the more powerful/complex
>>>
>>> if ((condition) and (condition)) action
>>>
>>> line format
>>>
>>> David Lang
>>>
>>>
>>>       
>>>> Thanks,
>>>> Ralph
>>>>
>>>> Ralph Crongeyer wrote:
>>>>
>>>>         
>>>>> Oh,
>>>>> I tried that but I had it on the same line. So that has to be on a
>>>>> separate line?
>>>>>
>>>>> Thanks again for the explanation that really helps me understand how
>>>>> it's working.
>>>>>
>>>>> Thanks again for all your help with this.
>>>>>
>>>>> Ralph
>>>>>
>>>>> david at lang.hm wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Hi Rainer,
>>>>>>> Thanks for the explanation, that helps me understand how it's working.
>>>>>>>
>>>>>>> That works, the logs are going to the correct file, however they are
>>>>>>> also being sent to /var/log/syslog? How can I make all the logs from my
>>>>>>> host "192.168.1.1" go only to the "-?DynFwall" template file?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> after you tell rsyslog to put the logs in that file, you then need to tell
>>>>>> rsyslog to throw the log away.
>>>>>>
>>>>>> so you would do something like
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>> & ~
>>>>>>
>>>>>> which is logicly the same as
>>>>>>
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>> :fromhost-ip,isequal,"192.168.1.1"    ~
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> I would like to give feedback on the cookbook let me know how I can help.
>>>>>>>
>>>>>>> Thanks all, for your help with this.
>>>>>>> Ralph
>>>>>>>
>>>>>>> Rainer Gerhards wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Ralph
>>>>>>>>> Crongeyer
>>>>>>>>> Sent: Monday, January 18, 2010 4:37 PM
>>>>>>>>> To: Philip M. Gollucci
>>>>>>>>> Cc: rsyslog-users
>>>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>>>
>>>>>>>>> Hi Phillip,
>>>>>>>>> Thanks for the response.
>>>>>>>>> The %HOSTNAME% part works fine here if I do this:
>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>> *.*    -?DynFwall
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Phillip suggested the rigth thing.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> However if I try to filter by IP using the "fromhost-ip" like this:
>>>>>>>>> *.*    :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> The issue is that the config is wrong. "*.*" and ":fromhost..." are both
>>>>>>>> filters. There can only be one filter in front of an action. As *.* maeans
>>>>>>>> all messages, I assume ou actually wanted to do this:
>>>>>>>>
>>>>>>>> :fromhost-ip,isequal,"192.168.1.1"    -?DynFwall
>>>>>>>>
>>>>>>>> Which filters alls messages based on fromhost-ip.
>>>>>>>>
>>>>>>>> The config format is clumpsy. I am currently talking with some folks at
>>>>>>>> Adiscon, and we will probably create a cookbook-type doc that provides
>>>>>>>> samples for some common scenarios. I guess that would be useful. Any feedback
>>>>>>>> on that effort would be welcome.
>>>>>>>>
>>>>>>>> Rainer
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> It fails to capture logs in the DynFwall template file.
>>>>>>>>>
>>>>>>>>> I've tried to do this with the "fromhost" and the "fromhost-ip" and
>>>>>>>>> neither seem to work?
>>>>>>>>>
>>>>>>>>> I want to have it so that a specific host IP uses a specific template.
>>>>>>>>>
>>>>>>>>> It looks like the fromhost and the fromhost-ip arn't working
>>>>>>>>> at all? Or
>>>>>>>>> my config is wrong.
>>>>>>>>>
>>>>>>>>> Dose anyone on the list have "fromhost-ip" working?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ralph
>>>>>>>>>
>>>>>>>>> Philip M. Gollucci wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> On 1/17/2010 5:50 PM, Ralph Crongeyer wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> # Firewall logs #
>>>>>>>>>>> $template DynFwall,"/var/log/server-logs/firewall/%HOSTNAME%.log"
>>>>>>>>>>> *.* :fromhost-ip, isequal, "192.168.1.1" -?DynFwall
>>>>>>>>>>>
>>>>>>>>>>> But I just getting this error in /var/log/syslog:
>>>>>>>>>>>
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: [origin software="rsyslogd"
>>>>>>>>>>> swVersion="4.4.2" x-pid="12540"
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> x-info="http://www.rsyslog.com"] (re)start
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>>> /etc/rsyslog.d/remote-logs.conf, line 10
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: warning: selector line
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> without actions
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> will be discarded
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd: the last error occured in
>>>>>>>>>>> /etc/rsyslog.conf, line 48
>>>>>>>>>>> Jan 17 16:49:47 log rsyslogd-2124: CONFIG ERROR: could not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> interpret
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> master config file '/etc/rsyslog.conf'. [try
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>> http://www.rsyslog.com/e/2124 ]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>>> I'm trying to log all logs from my IPCop host to
>>>>>>>>>>> "/var/log/server-logs/firewall/%HOSTNAME%.log" .
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> I tried for 1.5 days to figure this out cutting and pasting examples
>>>>>>>>>> left and right.  Finally I came up with the following with
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> works well
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> for me, you should be able to tweak it slightly for yourself.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> $template by_prog,"/var/log/rws/%programname%.log"
>>>>>>>>>>
>>>>>>>>>> :programname, regex, "^pxy.*rc\."  ?by_prog
>>>>>>>>>> & :omrelp:cl.dca1.rws:2514
>>>>>>>>>> & ~
>>>>>>>>>>
>>>>>>>>>> Just sub out %programname% for %HOSTNAME%
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> --
>>>>>>>>> Reminds me of my expedition into the wilds of Afghanistan. We
>>>>>>>>> lost our
>>>>>>>>> corkscrew and were compelled to live on food and water for
>>>>>>>>> several days. -
>>>>>>>>> WC Fields
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> rsyslog mailing list
>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>>> http://www.rsyslog.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> _______________________________________________
>>>>>>>> rsyslog mailing list
>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>>> http://www.rsyslog.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>           
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 21:56:30 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 12:56:30 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54CA10.2060103@crongeyer.com>
References: <4B539422.3020709@crongeyer.com> <4B5398A2.4020604@p6m7g8.com>
	<4B548032.60807@crongeyer.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>
	<4B5497DA.9070405@crongeyer.com>
	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>
	<4B549E97.8030108@crongeyer.com> <4B54A555.9010007@crongeyer.com>
	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>
	<4B54C66F.80506@crongeyer.com>
	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>
	<4B54CA10.2060103@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> When I switched to double quotes I get the error in /var/log/syslog and
> no logs are collected?

what was the error you got this time?

David Lang



From rgerhards at hq.adiscon.com  Mon Jan 18 21:59:54 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 21:59:54 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>

David,

Single quotes are right in the scripting engine (double quotes are reserved
for future use - they shall provide the capability to extend macros, e.g.
$A="BC" => '$A' is the string "$A", while "$A" is supposed to be the string
"BC").

I don't have an idea what may be wrong, but running rsyslog in debug mode
will most probably pinpoint it.

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 18, 2010 9:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> 
> > When I switched to double quotes I get the error in 
> /var/log/syslog and
> > no logs are collected?
> 
> what was the error you got this time?
> 
> David Lang
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Mon Jan 18 22:02:04 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 13:02:04 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>

On Mon, 18 Jan 2010, Rainer Gerhards wrote:

> David,
>
> Single quotes are right in the scripting engine (double quotes are reserved
> for future use - they shall provide the capability to extend macros, e.g.
> $A="BC" => '$A' is the string "$A", while "$A" is supposed to be the string
> "BC").

that is the normal behavior of single vs double quotes, but in such 
situations it's normal for 'ABC' and "ABC" to be equivalent, it's only 
when you have variables involved that there would be a difference.

David Lang

> I don't have an idea what may be wrong, but running rsyslog in debug mode
> will most probably pinpoint it.
>
> Rainer
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 18, 2010 9:57 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>
>>> When I switched to double quotes I get the error in
>> /var/log/syslog and
>>> no logs are collected?
>>
>> what was the error you got this time?
>>
>> David Lang
>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From ralph at crongeyer.com  Mon Jan 18 22:02:27 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:02:27 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com>
	<4B5398A2.4020604@p6m7g8.com>	<4B548032.60807@crongeyer.com>	<9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com>	<4B5497DA.9070405@crongeyer.com>	<alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm>	<4B549E97.8030108@crongeyer.com>
	<4B54A555.9010007@crongeyer.com>	<alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm>	<4B54C66F.80506@crongeyer.com>	<alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm>	<4B54CA10.2060103@crongeyer.com>
	<alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm>
Message-ID: <4B54CC63.3010103@crongeyer.com>

With double quots I get this in /var/log/syslog:

Jan 18 16:00:22 log rsyslogd: [origin software="rsyslogd" 
swVersion="4.4.2" x-pid="15703" x-info="http://www.rsyslog.com"] (re)start
Jan 18 16:00:22 log rsyslogd: the last error occured in 
/etc/rsyslog.d/remote-logs.conf, line 6
Jan 18 16:00:22 log rsyslogd: warning: selector line without actions 
will be discarded
Jan 18 16:00:22 log rsyslogd: the last error occured in 
/etc/rsyslog.conf, line 48
Jan 18 16:00:22 log rsyslogd-2124: CONFIG ERROR: could not interpret 
master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]

david at lang.hm wrote:
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> When I switched to double quotes I get the error in /var/log/syslog and
>> no logs are collected?
>>     
>
> what was the error you got this time?
>
> David Lang
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Mon Jan 18 22:03:50 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 18 Jan 2010 22:03:50 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com 
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Monday, January 18, 2010 10:02 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> 
> > David,
> >
> > Single quotes are right in the scripting engine (double 
> quotes are reserved
> > for future use - they shall provide the capability to 
> extend macros, e.g.
> > $A="BC" => '$A' is the string "$A", while "$A" is supposed 
> to be the string
> > "BC").
> 
> that is the normal behavior of single vs double quotes, but in such 
> situations it's normal for 'ABC' and "ABC" to be equivalent, 
> it's only 
> when you have variables involved that there would be a difference.

Jup, that's right - but double quotes are not yet implemented ;)

Rainer
> 
> David Lang
> 
> > I don't have an idea what may be wrong, but running rsyslog 
> in debug mode
> > will most probably pinpoint it.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com
> >> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
> david at lang.hm
> >> Sent: Monday, January 18, 2010 9:57 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] fromhost-ip
> >>
> >> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >>
> >>> When I switched to double quotes I get the error in
> >> /var/log/syslog and
> >>> no logs are collected?
> >>
> >> what was the error you got this time?
> >>
> >> David Lang
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From ralph at crongeyer.com  Mon Jan 18 22:27:49 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:27:49 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
Message-ID: <4B54D255.30505@crongeyer.com>

Here's the debug output when configured with single quotes.
I'm sending this off the list to Rainer.
David, let me know if you want this also.

Thanks guys,
Ralph

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com 
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Monday, January 18, 2010 10:02 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>>
>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>
>>     
>>> David,
>>>
>>> Single quotes are right in the scripting engine (double 
>>>       
>> quotes are reserved
>>     
>>> for future use - they shall provide the capability to 
>>>       
>> extend macros, e.g.
>>     
>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed 
>>>       
>> to be the string
>>     
>>> "BC").
>>>       
>> that is the normal behavior of single vs double quotes, but in such 
>> situations it's normal for 'ABC' and "ABC" to be equivalent, 
>> it's only 
>> when you have variables involved that there would be a difference.
>>     
>
> Jup, that's right - but double quotes are not yet implemented ;)
>
> Rainer
>   
>> David Lang
>>
>>     
>>> I don't have an idea what may be wrong, but running rsyslog 
>>>       
>> in debug mode
>>     
>>> will most probably pinpoint it.
>>>
>>> Rainer
>>>
>>>       
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
>>>>         
>> david at lang.hm
>>     
>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>
>>>>         
>>>>> When I switched to double quotes I get the error in
>>>>>           
>>>> /var/log/syslog and
>>>>         
>>>>> no logs are collected?
>>>>>           
>>>> what was the error you got this time?
>>>>
>>>> David Lang
>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From ralph at crongeyer.com  Mon Jan 18 22:47:53 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 16:47:53 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54D255.30505@crongeyer.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
	<4B54D255.30505@crongeyer.com>
Message-ID: <4B54D709.4050408@crongeyer.com>

This ma be of help:

0928.085091536:imrelp.c: Message has legacy syslog format.
0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
0928.085443830:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
0928.085812887:imrelp.c: tcpSend returns 17
0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
0928.086029125:imrelp.c: relp engine is dispatching frame with command 
'syslog'
0928.086053430:imrelp.c: in 'syslog' command handler
0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg 
2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost 
connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
0928.086124392:imrelp.c: Message has legacy syslog format.
0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
0928.086514402:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
0928.087044659:imrelp.c: tcpSend returns 17
0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
0928.087110313:imrelp.c: relp engine is dispatching frame with command 
'syslog'
0928.087131545:imrelp.c: in 'syslog' command handler
0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg 
2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect 
from 81-64-60-151.rev.numericable.fr[81.64.60.151]
0928.087200552:imrelp.c: Message has legacy syslog format.
0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0, 
size now 0 entries
0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
0928.087609280:main queue:Reg/w0: Filter: check for property 
'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE, 
waiting for work.
0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
0928.088020802:imrelp.c: tcpSend returns 17
0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
0928.088099586:imrelp.c: ***<librelp> calling select, active file 
descriptors (max 23): 6 7 23
0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout, 
worker terminating...
0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08, 
terminated, num workers now 0
0988.088339377:main queue:Reg/w0: destructor for debug call stack 
0x9bd1260 called


Ralph Crongeyer wrote:
> Here's the debug output when configured with single quotes.
> I'm sending this off the list to Rainer.
> David, let me know if you want this also.
>
> Thanks guys,
> Ralph
>
> Rainer Gerhards wrote:
>   
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com 
>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Monday, January 18, 2010 10:02 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] fromhost-ip
>>>
>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>
>>>     
>>>       
>>>> David,
>>>>
>>>> Single quotes are right in the scripting engine (double 
>>>>       
>>>>         
>>> quotes are reserved
>>>     
>>>       
>>>> for future use - they shall provide the capability to 
>>>>       
>>>>         
>>> extend macros, e.g.
>>>     
>>>       
>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed 
>>>>       
>>>>         
>>> to be the string
>>>     
>>>       
>>>> "BC").
>>>>       
>>>>         
>>> that is the normal behavior of single vs double quotes, but in such 
>>> situations it's normal for 'ABC' and "ABC" to be equivalent, 
>>> it's only 
>>> when you have variables involved that there would be a difference.
>>>     
>>>       
>> Jup, that's right - but double quotes are not yet implemented ;)
>>
>> Rainer
>>   
>>     
>>> David Lang
>>>
>>>     
>>>       
>>>> I don't have an idea what may be wrong, but running rsyslog 
>>>>       
>>>>         
>>> in debug mode
>>>     
>>>       
>>>> will most probably pinpoint it.
>>>>
>>>> Rainer
>>>>
>>>>       
>>>>         
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of 
>>>>>         
>>>>>           
>>> david at lang.hm
>>>     
>>>       
>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>
>>>>>         
>>>>>           
>>>>>> When I switched to double quotes I get the error in
>>>>>>           
>>>>>>             
>>>>> /var/log/syslog and
>>>>>         
>>>>>           
>>>>>> no logs are collected?
>>>>>>           
>>>>>>             
>>>>> what was the error you got this time?
>>>>>
>>>>> David Lang
>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>       
>>>>         
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>     
>>>       
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>   
>>     
>
>
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From david at lang.hm  Mon Jan 18 22:52:32 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 18 Jan 2010 13:52:32 -0800 (PST)
Subject: [rsyslog] fromhost-ip
In-Reply-To: <4B54D709.4050408@crongeyer.com>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>
	<4B54D255.30505@crongeyer.com> <4B54D709.4050408@crongeyer.com>
Message-ID: <alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>

Ok, this says that fromhost-ip is not being set in your case.

I think I ran into a similar problem before, are you starting with -x to 
disable name lookups?

try changing from fromhost-ip to fromhost

David Lang

On Mon, 18 Jan 2010, Ralph Crongeyer wrote:

> This ma be of help:
>
> 0928.085091536:imrelp.c: Message has legacy syslog format.
> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
> 0928.085443830:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.085812887:imrelp.c: tcpSend returns 17
> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> 0928.086029125:imrelp.c: relp engine is dispatching frame with command
> 'syslog'
> 0928.086053430:imrelp.c: in 'syslog' command handler
> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
> connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> 0928.086124392:imrelp.c: Message has legacy syslog format.
> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
> 0928.086514402:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.087044659:imrelp.c: tcpSend returns 17
> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> 0928.087110313:imrelp.c: relp engine is dispatching frame with command
> 'syslog'
> 0928.087131545:imrelp.c: in 'syslog' command handler
> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect
> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> 0928.087200552:imrelp.c: Message has legacy syslog format.
> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0,
> size now 0 entries
> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
> 0928.087609280:main queue:Reg/w0: Filter: check for property
> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> waiting for work.
> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
> 0928.088020802:imrelp.c: tcpSend returns 17
> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
> descriptors (max 23): 6 7 23
> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout,
> worker terminating...
> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08,
> terminated, num workers now 0
> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> 0x9bd1260 called
>
>
> Ralph Crongeyer wrote:
>> Here's the debug output when configured with single quotes.
>> I'm sending this off the list to Rainer.
>> David, let me know if you want this also.
>>
>> Thanks guys,
>> Ralph
>>
>> Rainer Gerhards wrote:
>>
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com
>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>> Sent: Monday, January 18, 2010 10:02 PM
>>>> To: rsyslog-users
>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>
>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>>
>>>>
>>>>
>>>>> David,
>>>>>
>>>>> Single quotes are right in the scripting engine (double
>>>>>
>>>>>
>>>> quotes are reserved
>>>>
>>>>
>>>>> for future use - they shall provide the capability to
>>>>>
>>>>>
>>>> extend macros, e.g.
>>>>
>>>>
>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>>>>>
>>>>>
>>>> to be the string
>>>>
>>>>
>>>>> "BC").
>>>>>
>>>>>
>>>> that is the normal behavior of single vs double quotes, but in such
>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>>>> it's only
>>>> when you have variables involved that there would be a difference.
>>>>
>>>>
>>> Jup, that's right - but double quotes are not yet implemented ;)
>>>
>>> Rainer
>>>
>>>
>>>> David Lang
>>>>
>>>>
>>>>
>>>>> I don't have an idea what may be wrong, but running rsyslog
>>>>>
>>>>>
>>>> in debug mode
>>>>
>>>>
>>>>> will most probably pinpoint it.
>>>>>
>>>>> Rainer
>>>>>
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>>>>>>
>>>>>>
>>>> david at lang.hm
>>>>
>>>>
>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>>> To: rsyslog-users
>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>
>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> When I switched to double quotes I get the error in
>>>>>>>
>>>>>>>
>>>>>> /var/log/syslog and
>>>>>>
>>>>>>
>>>>>>> no logs are collected?
>>>>>>>
>>>>>>>
>>>>>> what was the error you got this time?
>>>>>>
>>>>>> David Lang
>>>>>>
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>>>
>>>
>>
>>
>>
>
>
>


From ralph at crongeyer.com  Mon Jan 18 23:12:24 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Mon, 18 Jan 2010 17:12:24 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>	<4B54D255.30505@crongeyer.com>
	<4B54D709.4050408@crongeyer.com>
	<alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
Message-ID: <4B54DCC8.3020504@crongeyer.com>

No, I'm starting with -c4.

I'll give it a try but ultimately  I need to filter in IP.

I'll try it when I get back from dinner......

Thanks again for your help with this guys.

david at lang.hm wrote:
> Ok, this says that fromhost-ip is not being set in your case.
>
> I think I ran into a similar problem before, are you starting with -x to 
> disable name lookups?
>
> try changing from fromhost-ip to fromhost
>
> David Lang
>
> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>
>   
>> This ma be of help:
>>
>> 0928.085091536:imrelp.c: Message has legacy syslog format.
>> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.085443830:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.085812887:imrelp.c: tcpSend returns 17
>> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
>> 0928.086029125:imrelp.c: relp engine is dispatching frame with command
>> 'syslog'
>> 0928.086053430:imrelp.c: in 'syslog' command handler
>> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
>> connection after RCPT from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> 0928.086124392:imrelp.c: Message has legacy syslog format.
>> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.086514402:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.087044659:imrelp.c: tcpSend returns 17
>> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
>> 0928.087110313:imrelp.c: relp engine is dispatching frame with command
>> 'syslog'
>> 0928.087131545:imrelp.c: in 'syslog' command handler
>> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]: disconnect
>> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> 0928.087200552:imrelp.c: Message has legacy syslog format.
>> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
>> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
>> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state 0,
>> size now 0 entries
>> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
>> 0928.087609280:main queue:Reg/w0: Filter: check for property
>> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> waiting for work.
>> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
>> 0928.088020802:imrelp.c: tcpSend returns 17
>> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
>> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
>> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
>> descriptors (max 23): 6 7 23
>> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity timeout,
>> worker terminating...
>> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving command 1
>> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker terminating
>> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread 9bb5a08,
>> terminated, num workers now 0
>> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
>> 0x9bd1260 called
>>
>>
>> Ralph Crongeyer wrote:
>>     
>>> Here's the debug output when configured with single quotes.
>>> I'm sending this off the list to Rainer.
>>> David, let me know if you want this also.
>>>
>>> Thanks guys,
>>> Ralph
>>>
>>> Rainer Gerhards wrote:
>>>
>>>       
>>>>> -----Original Message-----
>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>>> Sent: Monday, January 18, 2010 10:02 PM
>>>>> To: rsyslog-users
>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>
>>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> David,
>>>>>>
>>>>>> Single quotes are right in the scripting engine (double
>>>>>>
>>>>>>
>>>>>>             
>>>>> quotes are reserved
>>>>>
>>>>>
>>>>>           
>>>>>> for future use - they shall provide the capability to
>>>>>>
>>>>>>
>>>>>>             
>>>>> extend macros, e.g.
>>>>>
>>>>>
>>>>>           
>>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>>>>>>
>>>>>>
>>>>>>             
>>>>> to be the string
>>>>>
>>>>>
>>>>>           
>>>>>> "BC").
>>>>>>
>>>>>>
>>>>>>             
>>>>> that is the normal behavior of single vs double quotes, but in such
>>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>>>>> it's only
>>>>> when you have variables involved that there would be a difference.
>>>>>
>>>>>
>>>>>           
>>>> Jup, that's right - but double quotes are not yet implemented ;)
>>>>
>>>> Rainer
>>>>
>>>>
>>>>         
>>>>> David Lang
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> I don't have an idea what may be wrong, but running rsyslog
>>>>>>
>>>>>>
>>>>>>             
>>>>> in debug mode
>>>>>
>>>>>
>>>>>           
>>>>>> will most probably pinpoint it.
>>>>>>
>>>>>> Rainer
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> -----Original Message-----
>>>>>>> From: rsyslog-bounces at lists.adiscon.com
>>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>> david at lang.hm
>>>>>
>>>>>
>>>>>           
>>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>>>>>>> To: rsyslog-users
>>>>>>> Subject: Re: [rsyslog] fromhost-ip
>>>>>>>
>>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> When I switched to double quotes I get the error in
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> /var/log/syslog and
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> no logs are collected?
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> what was the error you got this time?
>>>>>>>
>>>>>>> David Lang
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> rsyslog mailing list
>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>>> http://www.rsyslog.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> _______________________________________________
>>>>>> rsyslog mailing list
>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>> http://www.rsyslog.com
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>> http://www.rsyslog.com
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>> http://www.rsyslog.com
>>>>
>>>>
>>>>         
>>>
>>>       
>>
>>     
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>   


-- 
Reminds me of my expedition into the wilds of Afghanistan. We lost our 
corkscrew and were compelled to live on food and water for several days. - 
WC Fields



From rgerhards at hq.adiscon.com  Tue Jan 19 10:44:04 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 10:44:04 +0100
Subject: [rsyslog] fromhost-ip
References: <4B539422.3020709@crongeyer.com><4B5398A2.4020604@p6m7g8.com><4B548032.60807@crongeyer.com><9B6E2A8877C38245BFB15CC491A11DA71036F2@GRFEXC.intern.adiscon.com><4B5497DA.9070405@crongeyer.com><alpine.DEB.2.00.1001180927180.11108@asgard.lang.hm><4B549E97.8030108@crongeyer.com><4B54A555.9010007@crongeyer.com><alpine.DEB.2.00.1001181156550.7748@asgard.lang.hm><4B54C66F.80506@crongeyer.com><alpine.DEB.2.00.1001181240360.7748@asgard.lang.hm><4B54CA10.2060103@crongeyer.com><alpine.DEB.2.00.1001181255260.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA71036F6@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001181300580.7748@asgard.lang.hm>	<9B6E2A8877C38245BFB15CC491A11DA71036F7@GRFEXC.intern.adiscon.com>	<4B54D255.30505@crongeyer.com><4B54D709.4050408@crongeyer.com><alpine.DEB.2.00.1001181351220.7748@asgard.lang.hm>
	<4B54DCC8.3020504@crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036F9@GRFEXC.intern.adiscon.com>

RELP did not provide fromhost-ip until recently. You need to use the most
recent development version of the git master branch (to be released soon)
TOGETHER with the most recent version of librelp to get that information.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> Sent: Monday, January 18, 2010 11:12 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
> No, I'm starting with -c4.
> 
> I'll give it a try but ultimately  I need to filter in IP.
> 
> I'll try it when I get back from dinner......
> 
> Thanks again for your help with this guys.
> 
> david at lang.hm wrote:
> > Ok, this says that fromhost-ip is not being set in your case.
> >
> > I think I ran into a similar problem before, are you starting with -x
> to
> > disable name lookups?
> >
> > try changing from fromhost-ip to fromhost
> >
> > David Lang
> >
> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >
> >
> >> This ma be of help:
> >>
> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.085812887:imrelp.c: tcpSend returns 17
> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
> command
> >> 'syslog'
> >> 0928.086053430:imrelp.c: in 'syslog' command handler
> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: lost
> >> connection after RCPT from 81-64-60-
> 151.rev.numericable.fr[81.64.60.151]
> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.087044659:imrelp.c: tcpSend returns 17
> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
> command
> >> 'syslog'
> >> 0928.087131545:imrelp.c: in 'syslog' command handler
> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
> disconnect
> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state
> 0,
> >> size now 0 entries
> >> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> waiting for work.
> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
> >> 0928.088020802:imrelp.c: tcpSend returns 17
> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> >> 0928.088099586:imrelp.c: ***<librelp> calling select, active file
> >> descriptors (max 23): 6 7 23
> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
> timeout,
> >> worker terminating...
> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
> command 1
> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
> terminating
> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
> 9bb5a08,
> >> terminated, num workers now 0
> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> >> 0x9bd1260 called
> >>
> >>
> >> Ralph Crongeyer wrote:
> >>
> >>> Here's the debug output when configured with single quotes.
> >>> I'm sending this off the list to Rainer.
> >>> David, let me know if you want this also.
> >>>
> >>> Thanks guys,
> >>> Ralph
> >>>
> >>> Rainer Gerhards wrote:
> >>>
> >>>
> >>>>> -----Original Message-----
> >>>>> From: rsyslog-bounces at lists.adiscon.com
> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> david at lang.hm
> >>>>> Sent: Monday, January 18, 2010 10:02 PM
> >>>>> To: rsyslog-users
> >>>>> Subject: Re: [rsyslog] fromhost-ip
> >>>>>
> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> David,
> >>>>>>
> >>>>>> Single quotes are right in the scripting engine (double
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> quotes are reserved
> >>>>>
> >>>>>
> >>>>>
> >>>>>> for future use - they shall provide the capability to
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> extend macros, e.g.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> to be the string
> >>>>>
> >>>>>
> >>>>>
> >>>>>> "BC").
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> that is the normal behavior of single vs double quotes, but in
> such
> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
> >>>>> it's only
> >>>>> when you have variables involved that there would be a
> difference.
> >>>>>
> >>>>>
> >>>>>
> >>>> Jup, that's right - but double quotes are not yet implemented ;)
> >>>>
> >>>> Rainer
> >>>>
> >>>>
> >>>>
> >>>>> David Lang
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> I don't have an idea what may be wrong, but running rsyslog
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> in debug mode
> >>>>>
> >>>>>
> >>>>>
> >>>>>> will most probably pinpoint it.
> >>>>>>
> >>>>>> Rainer
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>> david at lang.hm
> >>>>>
> >>>>>
> >>>>>
> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
> >>>>>>> To: rsyslog-users
> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
> >>>>>>>
> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> When I switched to double quotes I get the error in
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> /var/log/syslog and
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> no logs are collected?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>> what was the error you got this time?
> >>>>>>>
> >>>>>>> David Lang
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> rsyslog mailing list
> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>>> http://www.rsyslog.com
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> rsyslog mailing list
> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>>> http://www.rsyslog.com
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>> _______________________________________________
> >>>>> rsyslog mailing list
> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>>> http://www.rsyslog.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>>> http://www.rsyslog.com
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> 
> --
> Reminds me of my expedition into the wilds of Afghanistan. We lost our
> corkscrew and were compelled to live on food and water for several
> days. -
> WC Fields
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 14:53:44 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 14:53:44 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>

Michael,

I tried to reproduce, but I can not get to this error. Could you provide me a
debug log of the failed startup?

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, January 17, 2010 12:49 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >> Sent: Friday, January 15, 2010 11:57 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >> > Michael,
> >> >
> >> > Fix now in git, links at the bug tracker:
> >> >
> >> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >> >
> >> > Please let me know if it works for you (the patch is a bit
> trickier
> >> than it
> >> > looks, so confirmations would be good).
> >>
> >> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> >> But now I'm getting a crash when rsyslog encounters the xconsole
> pipe
> >> config.
> >
> > I am a bit puzzled, but will try to reproduce that on my Debian box.
> I assume
> > stock Debian config?
> 
> Yes. As said, I just downloaded the 5.3.6 tarball applied the
> 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> got the crash. I use the default rsyslog.conf from the official debian
> package.
> I attached a backtrace. Hope that helps
> 
> (gdb) run -c4 -d
> Starting program: /usr/sbin/rsyslogd -c4 -d
> [Thread debugging using libthread_db enabled]
> [New Thread 0xb7df2b70 (LWP 5162)]
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0xb7df2b70 (LWP 5162)]
> qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
> pUsr=0x80b72c0) at queue.c:2256
> 2256            if(pThis->qType != QUEUETYPE_DIRECT) {
> (gdb) bt full
> #0  qqueueEnqObj (pThis=0x1, flowCtlType=eFLOWCTL_LIGHT_DELAY,
> pUsr=0x80b72c0) at queue.c:2256
>         iRet = <value optimized out>
>         iCancelStateSave = <value optimized out>
> #1  0x0807cd5b in actionWriteToAction (pAction=0x80ac8d8) at
> ../action.c:1169
>         pMsgSave = 0x0
>         iRet = <value optimized out>
> #2  0x0807d4fe in doActionCallAction (pAction=0x80ac8d8,
> pMsg=0x80b72c0) at ../action.c:1244
> No locals.
> #3  actionCallAction (pAction=0x80ac8d8, pMsg=0x80b72c0) at
> ../action.c:1274
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113864, 0, -1210114056, 1464119912, -422385385},
>               __mask_was_saved = 0}}, __pad = {0xb7df2240, 0x0,
> 0xb7df20d0, 0xb7feb27f}}
>         __cancel_arg = 0x80ad0a0
>         not_first_call = <value optimized out>
>         iRet = -1210113864
> #4  0x080794d7 in processMsgDoActions (pData=0xfffff815,
> pParam=0xb7df20b8) at rule.c:113
>         iRet = <value optimized out>
>         iRetMod = <value optimized out>
> #5  0x080627ba in llExecFunc (pThis=0x80ac9a0, pFunc=0x8079480
> <processMsgDoActions>, pParam=0xb7df20b8) at linkedlist.c:391
>         iRet = <value optimized out>
>         iRetLL = <value optimized out>
>         pData = 0x80ac8d8
>         llCookie = 0x80ac508
>         llCookiePrev = 0x0
> #6  0x08079007 in processMsg (pThis=0x80ac968, pMsg=0x80b72c0) at
> rule.c:299
>         bProcessMsg = 1
>         DoActData = {bPrevWasSuspended = 0, pMsg = 0x80b72c0}
>         iRet = RS_RET_OK
> #7  0x080781ba in processMsgDoRules (pData=0x80ac968,
> pParam=0x80b72c0) at ruleset.c:145
>         iRet = <value optimized out>
> #8  0x080627ba in llExecFunc (pThis=0x809eb68, pFunc=0x8078190
> <processMsgDoRules>, pParam=0x80b72c0) at linkedlist.c:391
>         iRet = <value optimized out>
>         iRetLL = <value optimized out>
>         pData = 0x80ac968
>         llCookie = 0x80ac478
>         llCookiePrev = 0x80ac4f8
> #9  0x0807876d in processMsg (pMsg=0x80b72c0) at ruleset.c:164
>         pThis = <value optimized out>
>         iRet = <value optimized out>
> #10 0x080506ab in msgConsumer (notNeeded=0x0, pBatch=0x809ecc8,
> pbShutdownImmediate=0x80ad348) at syslogd.c:614
>         i = 0
>         pMsg = 0x80b72c0
>         localRet = RS_RET_IO_ERROR
> #11 0x08077ca4 in ConsumerReg (pThis=0x80ad338, pWti=0x809ecb0) at
> queue.c:1638
>         iCancelStateSave = 1
>         iRet = <value optimized out>
> #12 0x08070526 in wtiWorker (pThis=0x809ecb0) at wti.c:286
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113200, 0, -1210113384, 1463800424, -411898089},
>               __mask_was_saved = 0}}, __pad = {0xb7df2350, 0x0, 0x0,
> 0x809002c}}
>         not_first_call = <value optimized out>
>         pWtp = 0x809ebd8
>         bInactivityTOOccured = <value optimized out>
>         localRet = <value optimized out>
>         terminateRet = RS_RET_OK
>         iCancelStateSave = <value optimized out>
> #13 0x08070074 in wtpWorker (arg=0x809ecb0) at wtp.c:356
>         __cancel_buf = {__cancel_jmp_buf = {{__cancel_jmp_buf =
> {134807308, -1210113152, 0, -1210113096, 1463726696, -411345641},
>               __mask_was_saved = 0}}, __pad = {0xb7df2460, 0x0,
> 0xb7feff7b, 0xb7fe0cb0}}
>         not_first_call = <value optimized out>
>         pszDbgHdr = <value optimized out>
>         thrdName = "rs:main Q:Reg", '\000' <repeats 18 times>
> ---Type <return> to continue, or q <return> to quit---
>         pThis = 0x809ebd8
>         sigSet = {__val = {2147483647, 4294967294, 4294967295 <repeats
> 30 times>}}
> #14 0xb7f9c585 in start_thread () from /lib/i686/cmov/libpthread.so.0
> No symbol table info available.
> #15 0xb7f1026e in clone () from /lib/i686/cmov/libc.so.6
> No symbol table info available.
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 14:55:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 14:55:59 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<4B52FAD7.305@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FD@GRFEXC.intern.adiscon.com>

Philip,

I wil try to set up this as well. In the mean time, could you tell me if it
happens with the plain 5.3.6 or with the newer git tree (with the patch).
Without the patch, I can already see why it can happen, with it, I do not yet
have a clear understanding of the issue.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Sunday, January 17, 2010 12:56 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On 1/17/2010 6:51 AM, Michael Biebl wrote:
> > As an additonal hint: If I start xconsole (a process reading from
> > /dev/xconsole) before I start rsyslogd, then the crash does not
> occur.
> Possibly related, on FreeBSD in a jail if rsyslog ever tries to write
> to
> /dev/console, it loops in a extremely tightly loop consuming 100% of
> the
> core its on.  [see -dn output, possibly with ktrace/kdump]
> 
> Eventually it will consume all the memory on the box and it will go
> boom.
> 
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Sr. System Admin,                 Ridecharge Inc.
> Consultant,                       P6M7G8 Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Tue Jan 19 14:58:50 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 19 Jan 2010 14:58:50 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001190558k138e1868y2ec8763e8a574a00@mail.gmail.com>

2010/1/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> Michael,
>
> I tried to reproduce, but I can not get to this error. Could you provide me a
> debug log of the failed startup?

There is no debug output of rsyslog before it crashes. All I can get
is the gdb output I already attached

The missing debug output when using -d is another bug I already
mentioned in this thread.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Tue Jan 19 15:20:03 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:20:03 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>
	<4B509EC0.3050504@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>

Jack,

I have written this small patch for v4:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c89ae6b
41d1c5838b9a42b0

I assume it will apply without problems in v5 as well, but I have not yet
tried as I am doing some more work on v4 first (hoping to be able to save a
merge or two, which clutter up the git history...).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Friday, January 15, 2010 5:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> Rainer Gerhards wrote:
> > -----Original Message-----
> >> If that is modified as follows: *.*
> >> /var/log/syslog   # Comment goes here
> >>
> >
> > It's even worse: data is written to the file "/var/log/syslog   #
> > Comment goes here"!
> 
> Aaaaahhh - that finally explains some odd files lurking in /var/log!
> 
> That also explains why there's no error message on stdout/stdderr -
> there's no error, as far as rsyslog is concerned. [/me feels stupid]
> 
> I would really like to be able to put comments on the ends of config
> lines, as I can with many other packages. But now I know what's going
> on
> here. Thanks!
> 
> --
> Jack.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 19 15:50:42 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:50:42 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036FC@GRFEXC.intern.adiscon.com>
	<aa76cdac1001190558k138e1868y2ec8763e8a574a00@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71036FF@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Tuesday, January 19, 2010 2:59 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/19 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > Michael,
> >
> > I tried to reproduce, but I can not get to this error. Could you
> provide me a
> > debug log of the failed startup?
> 
> There is no debug output of rsyslog before it crashes. All I can get
> is the gdb output I already attached
> 
> The missing debug output when using -d is another bug I already
> mentioned in this thread.

slipped my mind, I should have opened a bug tracker. As I thought, it was a
regression from "debug on demand" mode. Patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bd03b86c6322c82fc9f66712
2f4365e339f28ccc

Rainer

> 
> Cheers,
> Michael
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mrdemeanour at jackpot.uk.net  Tue Jan 19 15:50:17 2010
From: mrdemeanour at jackpot.uk.net (Mr. Demeanour)
Date: Tue, 19 Jan 2010 14:50:17 +0000
Subject: [rsyslog] comments in file actions
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>	<4B509EC0.3050504@jackpot.uk.net>
	<9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
Message-ID: <4B55C6A9.5010008@jackpot.uk.net>

Rainer Gerhards wrote:
> Jack,
> 
> I have written this small patch for v4:
> 
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c89ae6b
>  41d1c5838b9a42b0
> 
> I assume it will apply without problems in v5 as well, but I have not
> yet tried as I am doing some more work on v4 first (hoping to be able
> to save a merge or two, which clutter up the git history...).

OK - have to go out now, but I will try this tomorrow and report back.

-- 
Jack.



From rgerhards at hq.adiscon.com  Tue Jan 19 15:51:33 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 15:51:33 +0100
Subject: [rsyslog] comments in file actions
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>	<aa76cdac1001142248n643efbd6w63b33c811fba5cb1@mail.gmail.com>	<aa76cdac1001142252j1806a272k762f9a0f91f539cc@mail.gmail.com>	<4B501261.1070901@p6m7g8.com>	<aa76cdac1001142315o5cd71c66o4ac24e704fed7c64@mail.gmail.com>	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>	<4B50252A.1000106@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036CF@GRFEXC.intern.adiscon.com>	<4B50922A.8090900@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036DE@GRFEXC.intern.adiscon.com>	<4B509EC0.3050504@jackpot.uk.net><9B6E2A8877C38245BFB15CC491A11DA71036FE@GRFEXC.intern.adiscon.com>
	<4B55C6A9.5010008@jackpot.uk.net>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103700@GRFEXC.intern.adiscon.com>


> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Mr. Demeanour
> Sent: Tuesday, January 19, 2010 3:50 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] comments in file actions
> 
> Rainer Gerhards wrote:
> > Jack,
> >
> > I have written this small patch for v4:
> >
> >
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=9cfa072caa0ba1863c
> 89ae6b
> >  41d1c5838b9a42b0
> >
> > I assume it will apply without problems in v5 as well, but I have not
> > yet tried as I am doing some more work on v4 first (hoping to be able
> > to save a merge or two, which clutter up the git history...).
> 
> OK - have to go out now, but I will try this tomorrow and report back.

excellent - thanks!

Rainer

> 
> --
> Jack.
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ralph at crongeyer.com  Tue Jan 19 16:22:14 2010
From: ralph at crongeyer.com (Ralph Crongeyer)
Date: Tue, 19 Jan 2010 10:22:14 -0500
Subject: [rsyslog] fromhost-ip
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036F9@GRFEXC.intern.adiscon.com>
Message-ID: <5fa6e0144d8003c7c72edff17f9f1675@webmail.crongeyer.com>

 Ok.
I'll try it with TCP (@@).
This weekend I'll build a deb of the latest rsyslog and relp and check it
out.

Would I ned the latest on both the rsyslog server and the client or just the
server?

Thanks,
Ralph
 
----------------original message-----------------
From: "Rainer Gerhards" rgerhards at hq.adiscon.com
To: "rsyslog-users" rsyslog at lists.adiscon.com
Date: Tue, 19 Jan 2010 10:44:04 +0100
-------------------------------------------------
 
 
> RELP did not provide fromhost-ip until recently. You need to use the most
> recent development version of the git master branch (to be released soon)
> TOGETHER with the most recent version of librelp to get that information.
> 
> Rainer
> 
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
>> Sent: Monday, January 18, 2010 11:12 PM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] fromhost-ip
>> 
>> No, I'm starting with -c4.
>> 
>> I'll give it a try but ultimately I need to filter in IP.
>> 
>> I'll try it when I get back from dinner......
>> 
>> Thanks again for your help with this guys.
>> 
>> david at lang.hm wrote:
>> > Ok, this says that fromhost-ip is not being set in your case.
>> >
>> > I think I ran into a similar problem before, are you starting with -x
>> to
>> > disable name lookups?
>> >
>> > try changing from fromhost-ip to fromhost
>> >
>> > David Lang
>> >
>> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>> >
>> >
>> >> This ma be of help:
>> >>
>> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
>> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.085416731:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.085812887:imrelp.c: tcpSend returns 17
>> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
>> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
>> command
>> >> 'syslog'
>> >> 0928.086053430:imrelp.c: in 'syslog' command handler
>> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]: 
>> lost
>> >> connection after RCPT from 81-64-60-
>> 151.rev.numericable.fr[81.64.60.151]
>> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
>> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.086486185:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.087044659:imrelp.c: tcpSend returns 17
>> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
>> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
>> command
>> >> 'syslog'
>> >> 0928.087131545:imrelp.c: in 'syslog' command handler
>> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5', msg
>> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
>> disconnect
>> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
>> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
>> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1 entries
>> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
>> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted, state
>> 0,
>> >> size now 0 entries
>> >> 0928.087581622:main queue:Reg/w0: result of expression evaluation: 0
>> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
>> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
>> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
>> >> waiting for work.
>> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker start
>> >> 0928.088020802:imrelp.c: tcpSend returns 17
>> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
>> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
>> >> 0928.088099586:imrelp.c: ***
 calling select, active file
>> >> descriptors (max 23): 6 7 23
>> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
>> timeout,
>> >> worker terminating...
>> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
>> command 1
>> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
>> terminating
>> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
>> 9bb5a08,
>> >> terminated, num workers now 0
>> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
>> >> 0x9bd1260 called
>> >>
>> >>
>> >> Ralph Crongeyer wrote:
>> >>
>> >>> Here's the debug output when configured with single quotes.
>> >>> I'm sending this off the list to Rainer.
>> >>> David, let me know if you want this also.
>> >>>
>> >>> Thanks guys,
>> >>> Ralph
>> >>>
>> >>> Rainer Gerhards wrote:
>> >>>
>> >>>
>> >>>>> -----Original Message-----
>> >>>>> From: rsyslog-bounces at lists.adiscon.com
>> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>> david at lang.hm
>> >>>>> Sent: Monday, January 18, 2010 10:02 PM
>> >>>>> To: rsyslog-users
>> >>>>> Subject: Re: [rsyslog] fromhost-ip
>> >>>>>
>> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> David,
>> >>>>>>
>> >>>>>> Single quotes are right in the scripting engine (double
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> quotes are reserved
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> for future use - they shall provide the capability to
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> extend macros, e.g.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> to be the string
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> "BC").
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> that is the normal behavior of single vs double quotes, but in
>> such
>> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
>> >>>>> it's only
>> >>>>> when you have variables involved that there would be a
>> difference.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>> Jup, that's right - but double quotes are not yet implemented ;)
>> >>>>
>> >>>> Rainer
>> >>>>
>> >>>>
>> >>>>
>> >>>>> David Lang
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> I don't have an idea what may be wrong, but running rsyslog
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> in debug mode
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> will most probably pinpoint it.
>> >>>>>>
>> >>>>>> Rainer
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> -----Original Message-----
>> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
>> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>> david at lang.hm
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
>> >>>>>>> To: rsyslog-users
>> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
>> >>>>>>>
>> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> When I switched to double quotes I get the error in
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>> /var/log/syslog and
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> no logs are collected?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>> what was the error you got this time?
>> >>>>>>>
>> >>>>>>> David Lang
>> >>>>>>>
>> >>>>>>> _______________________________________________
>> >>>>>>> rsyslog mailing list
>> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>>>> http://www.rsyslog.com
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>> _______________________________________________
>> >>>>>> rsyslog mailing list
>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>>> http://www.rsyslog.com
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>> _______________________________________________
>> >>>>> rsyslog mailing list
>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>>> http://www.rsyslog.com
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>> _______________________________________________
>> >>>> rsyslog mailing list
>> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> >>>> http://www.rsyslog.com
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>>
>> >>
>> >>
>> > _______________________________________________
>> > rsyslog mailing list
>> > http://lists.adiscon.net/mailman/listinfo/rsyslog
>> > http://www.rsyslog.com
>> >
>> 
>> 
>> --
>> Reminds me of my expedition into the wilds of Afghanistan. We lost our
>> corkscrew and were compelled to live on food and water for several
>> days. -
>> WC Fields
>> 
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 




From rgerhards at hq.adiscon.com  Tue Jan 19 16:28:01 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 19 Jan 2010 16:28:01 +0100
Subject: [rsyslog] fromhost-ip
References: <5fa6e0144d8003c7c72edff17f9f1675@webmail.crongeyer.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103702@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> Sent: Tuesday, January 19, 2010 4:22 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] fromhost-ip
> 
>  Ok.
> I'll try it with TCP (@@).
> This weekend I'll build a deb of the latest rsyslog and relp and check
> it
> out.
> 
> Would I ned the latest on both the rsyslog server and the client or
> just the
> server?

The server should be sufficient. The issue is that librelp < 1.0.0 has the
information, but does not pass it down to the call (imrelp in rsyslog case).
So imrelp decided to use "[unset]" instead of anything else (librelp actually
passes down the hostname twice). In librelp >= 1.0.0 this is corrected, it
now provides the ip address. However, you also need the new imrelp, as it now
needs to use that property. All of this, however, is done on the server, so
no dependency on the client should exist.

I have done these changes in early december 2009 as a side-activity for
something else relp related. My memory has a bit vanished since them, but I
think I conveyed the right information (but you now know I may be wrong in
case something works other than expected - in that case, ask here first
before getting nuts ;)).

Rainer

> 
> Thanks,
> Ralph
> 
> ----------------original message-----------------
> From: "Rainer Gerhards" rgerhards at hq.adiscon.com
> To: "rsyslog-users" rsyslog at lists.adiscon.com
> Date: Tue, 19 Jan 2010 10:44:04 +0100
> -------------------------------------------------
> 
> 
> > RELP did not provide fromhost-ip until recently. You need to use the
> most
> > recent development version of the git master branch (to be released
> soon)
> > TOGETHER with the most recent version of librelp to get that
> information.
> >
> > Rainer
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Ralph Crongeyer
> >> Sent: Monday, January 18, 2010 11:12 PM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] fromhost-ip
> >>
> >> No, I'm starting with -c4.
> >>
> >> I'll give it a try but ultimately I need to filter in IP.
> >>
> >> I'll try it when I get back from dinner......
> >>
> >> Thanks again for your help with this guys.
> >>
> >> david at lang.hm wrote:
> >> > Ok, this says that fromhost-ip is not being set in your case.
> >> >
> >> > I think I ran into a similar problem before, are you starting with
> -x
> >> to
> >> > disable name lookups?
> >> >
> >> > try changing from fromhost-ip to fromhost
> >> >
> >> > David Lang
> >> >
> >> > On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >> >
> >> >
> >> >> This ma be of help:
> >> >>
> >> >> 0928.085091536:imrelp.c: Message has legacy syslog format.
> >> >> 0928.085124502:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.085150205:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.085355268:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.085416731:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.085443830:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.085582122:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.085693593:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.085812887:imrelp.c: tcpSend returns 17
> >> >> 0928.085841383:imrelp.c: in destructor: sendbuf 0x9bc9228
> >> >> 0928.086029125:imrelp.c: relp engine is dispatching frame with
> >> command
> >> >> 'syslog'
> >> >> 0928.086053430:imrelp.c: in 'syslog' command handler
> >> >> 0928.086100366:imrelp.c: logmsg: flags 20, from '192.168.1.5',
> msg
> >> >> 2010-01-18T16:41:14.104596-05:00 spoonie postfix/smtpd[7528]:
> >> lost
> >> >> connection after RCPT from 81-64-60-
> >> 151.rev.numericable.fr[81.64.60.151]
> >> >> 0928.086124392:imrelp.c: Message has legacy syslog format.
> >> >> 0928.086157638:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.086202059:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.086419414:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.086486185:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.086514402:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.086771149:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.086895193:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.087044659:imrelp.c: tcpSend returns 17
> >> >> 0928.087074832:imrelp.c: in destructor: sendbuf 0x9bc9e10
> >> >> 0928.087110313:imrelp.c: relp engine is dispatching frame with
> >> command
> >> >> 'syslog'
> >> >> 0928.087131545:imrelp.c: in 'syslog' command handler
> >> >> 0928.087176805:imrelp.c: logmsg: flags 20, from '192.168.1.5',
> msg
> >> >> 2010-01-18T16:41:14.104922-05:00 spoonie postfix/smtpd[7528]:
> >> disconnect
> >> >> from 81-64-60-151.rev.numericable.fr[81.64.60.151]
> >> >> 0928.087200552:imrelp.c: Message has legacy syslog format.
> >> >> 0928.087232959:imrelp.c: main queue: entry added, size now 1
> entries
> >> >> 0928.087286600:imrelp.c: wtpAdviseMaxWorkers signals busy
> >> >> 0928.087482163:main queue:Reg/w0: main queue: entry deleted,
> state
> >> 0,
> >> >> size now 0 entries
> >> >> 0928.087581622:main queue:Reg/w0: result of expression
> evaluation: 0
> >> >> 0928.087609280:main queue:Reg/w0: Filter: check for property
> >> >> 'fromhost-ip' (value '[unset]') isequal '192.168.1.1': FALSE
> >> >> 0928.087783052:main queue:Reg/w0: main queue:Reg/w0: worker IDLE,
> >> >> waiting for work.
> >> >> 0928.087897597:imrelp.c: main queue: EnqueueMsg advised worker
> start
> >> >> 0928.088020802:imrelp.c: tcpSend returns 17
> >> >> 0928.088049857:imrelp.c: in destructor: sendbuf 0x9bc9d58
> >> >> 0928.088078912:imrelp.c: relpSendqIsEmpty() returns 1
> >> >> 0928.088099586:imrelp.c: ***
>  calling select, active file
> >> >> descriptors (max 23): 6 7 23
> >> >> 0988.087889021:main queue:Reg/w0: main queue:Reg/w0: inactivity
> >> timeout,
> >> >> worker terminating...
> >> >> 0988.088192704:main queue:Reg/w0: main queue:Reg/w0: receiving
> >> command 1
> >> >> 0988.088222318:main queue:Reg/w0: main queue:Reg/w0: worker
> >> terminating
> >> >> 0988.088247741:main queue:Reg/w0: main queue:Reg: Worker thread
> >> 9bb5a08,
> >> >> terminated, num workers now 0
> >> >> 0988.088339377:main queue:Reg/w0: destructor for debug call stack
> >> >> 0x9bd1260 called
> >> >>
> >> >>
> >> >> Ralph Crongeyer wrote:
> >> >>
> >> >>> Here's the debug output when configured with single quotes.
> >> >>> I'm sending this off the list to Rainer.
> >> >>> David, let me know if you want this also.
> >> >>>
> >> >>> Thanks guys,
> >> >>> Ralph
> >> >>>
> >> >>> Rainer Gerhards wrote:
> >> >>>
> >> >>>
> >> >>>>> -----Original Message-----
> >> >>>>> From: rsyslog-bounces at lists.adiscon.com
> >> >>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >> david at lang.hm
> >> >>>>> Sent: Monday, January 18, 2010 10:02 PM
> >> >>>>> To: rsyslog-users
> >> >>>>> Subject: Re: [rsyslog] fromhost-ip
> >> >>>>>
> >> >>>>> On Mon, 18 Jan 2010, Rainer Gerhards wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> David,
> >> >>>>>>
> >> >>>>>> Single quotes are right in the scripting engine (double
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> quotes are reserved
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> for future use - they shall provide the capability to
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> extend macros, e.g.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> $A="BC" => '$A' is the string "$A", while "$A" is supposed
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> to be the string
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> "BC").
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> that is the normal behavior of single vs double quotes, but in
> >> such
> >> >>>>> situations it's normal for 'ABC' and "ABC" to be equivalent,
> >> >>>>> it's only
> >> >>>>> when you have variables involved that there would be a
> >> difference.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>> Jup, that's right - but double quotes are not yet implemented
> ;)
> >> >>>>
> >> >>>> Rainer
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>> David Lang
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> I don't have an idea what may be wrong, but running rsyslog
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> in debug mode
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>> will most probably pinpoint it.
> >> >>>>>>
> >> >>>>>> Rainer
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> -----Original Message-----
> >> >>>>>>> From: rsyslog-bounces at lists.adiscon.com
> >> >>>>>>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>> david at lang.hm
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>>> Sent: Monday, January 18, 2010 9:57 PM
> >> >>>>>>> To: rsyslog-users
> >> >>>>>>> Subject: Re: [rsyslog] fromhost-ip
> >> >>>>>>>
> >> >>>>>>> On Mon, 18 Jan 2010, Ralph Crongeyer wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> When I switched to double quotes I get the error in
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>> /var/log/syslog and
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>> no logs are collected?
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>> what was the error you got this time?
> >> >>>>>>>
> >> >>>>>>> David Lang
> >> >>>>>>>
> >> >>>>>>> _______________________________________________
> >> >>>>>>> rsyslog mailing list
> >> >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>>>> http://www.rsyslog.com
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>> _______________________________________________
> >> >>>>>> rsyslog mailing list
> >> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>>> http://www.rsyslog.com
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>> _______________________________________________
> >> >>>>> rsyslog mailing list
> >> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>>> http://www.rsyslog.com
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>> _______________________________________________
> >> >>>> rsyslog mailing list
> >> >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> >>>> http://www.rsyslog.com
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>
> >> >>>
> >> >>
> >> >>
> >> > _______________________________________________
> >> > rsyslog mailing list
> >> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> > http://www.rsyslog.com
> >> >
> >>
> >>
> >> --
> >> Reminds me of my expedition into the wilds of Afghanistan. We lost
> our
> >> corkscrew and were compelled to live on food and water for several
> >> days. -
> >> WC Fields
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> 
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From epiphani at gmail.com  Tue Jan 19 19:55:55 2010
From: epiphani at gmail.com (Aaron Wiebe)
Date: Tue, 19 Jan 2010 13:55:55 -0500
Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
Message-ID: <e7ca40f71001191055k308bfa4dw2682e4fc702fefa8@mail.gmail.com>

Greetings,

I'm trying to sort out applying rulesets to IMUDP, and there is not
module-specific documentation for imudp as there is with imtcp.

What is the equivilent for udp input of:

$InputTCPServerInputName
$InputTCPServerBindRuleSet

?

I want to be able to apply rules to specific ports in the same way I
can with tcp...  Changing TCP to UDP doesn't seem to work.

-Aaron


From david at lang.hm  Wed Jan 20 00:26:15 2010
From: david at lang.hm (david at lang.hm)
Date: Tue, 19 Jan 2010 15:26:15 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com>
	<alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm>
	<aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com>
	<alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm>
	<aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com>
	<alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm>
	<aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com>
	<alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm>
	<aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com>
	<aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com>
	<alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
Message-ID: <alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>

I've now had v5.3.6 running on my production environemnets since friday 
with no problems

one side effect of the cleanups is that previously when I had multiple 
filters write to one file I was getting lots of corrupt lines, but the 
change to have omfile write each transaction rather than just as the 
buffer filled up seems to have eliminated this (it went from 10's of 
thousands of corrupted lines/day to none over the weekend and monday, 
tonight's report will be the acid test to see if it's fully cleaned up)

I realize there is still a window for corruption (if two output threads 
running at the same time both decide they need to write at the same time), 
but it seems that in practice it's effectively gone.

David Lang

On Fri, 15 Jan 2010, david at lang.hm wrote:

> On Fri, 15 Jan 2010, Michael Biebl wrote:
>
>> BTW, I'm actually surprised that you don't encounter those problems 
>> yourself.
>
> I'm running 5.3.5 still, I haven't had time to build a new version (hopefully 
> tomorrow)
>
> David Lang
>


From rgerhards at hq.adiscon.com  Wed Jan 20 12:09:18 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 12:09:18 +0100
Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
References: <e7ca40f71001191055k308bfa4dw2682e4fc702fefa8@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103710@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Aaron Wiebe
> Sent: Tuesday, January 19, 2010 7:56 PM
> To: rsyslog-users
> Subject: [rsyslog] Rulesets with UDP (in 4.5.7)
> 
> Greetings,
> 
> I'm trying to sort out applying rulesets to IMUDP, and there is not
> module-specific documentation for imudp as there is with imtcp.
> 
> What is the equivilent for udp input of:
> 
> $InputTCPServerInputName
> $InputTCPServerBindRuleSet
> 
> ?
> 
> I want to be able to apply rules to specific ports in the same way I
> can with tcp...  Changing TCP to UDP doesn't seem to work.

In v4, imudp has considerable less functionality than imtcp has. I think I
changed that only in v5. For example, you can NOT bind a ruleset to a
listener in imudp.

Rainer


From rgerhards at hq.adiscon.com  Wed Jan 20 16:19:46 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 16:19:46 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>

David,

thanks for the feedback - and a quick note.

With the new engine, you can do ruleset inclusion (via omruleset[1]). That is
probably *the* method to handle files that are written to by multiple
actions. Of course, no need to change if all works in default config. But you
can gain some extra performance by using buffered mode (for busy files) and
to use that, you need to have only one action write to each file. This is
where ruleset inclusion enters the game.

Rainer

[1] http://www.rsyslog.com/doc-omruleset.html

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 12:26 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> I've now had v5.3.6 running on my production environemnets since friday
> with no problems
> 
> one side effect of the cleanups is that previously when I had multiple
> filters write to one file I was getting lots of corrupt lines, but the
> change to have omfile write each transaction rather than just as the
> buffer filled up seems to have eliminated this (it went from 10's of
> thousands of corrupted lines/day to none over the weekend and monday,
> tonight's report will be the acid test to see if it's fully cleaned up)
> 
> I realize there is still a window for corruption (if two output threads
> running at the same time both decide they need to write at the same
> time),
> but it seems that in practice it's effectively gone.
> 
> David Lang
> 
> On Fri, 15 Jan 2010, david at lang.hm wrote:
> 
> > On Fri, 15 Jan 2010, Michael Biebl wrote:
> >
> >> BTW, I'm actually surprised that you don't encounter those problems
> >> yourself.
> >
> > I'm running 5.3.5 still, I haven't had time to build a new version
> (hopefully
> > tomorrow)
> >
> > David Lang
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 20 16:22:12 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 16:22:12 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>

I forgot to mention:

> I realize there is still a window for corruption (if two output threads
> running at the same time both decide they need to write at the same
> time),
> but it seems that in practice it's effectively gone.

The current code writes a single line with a single API call. I guess that
call is rather atomic from an OS point of view, so the window of corruption
probably don't even exists with current rsyslog and linux code.

Rainer


From david at lang.hm  Wed Jan 20 17:57:02 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 08:57:02 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001200851350.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Rainer Gerhards wrote:

> David,
>
> thanks for the feedback - and a quick note.
>
> With the new engine, you can do ruleset inclusion (via omruleset[1]). That is
> probably *the* method to handle files that are written to by multiple
> actions. Of course, no need to change if all works in default config. But you
> can gain some extra performance by using buffered mode (for busy files) and
> to use that, you need to have only one action write to each file. This is
> where ruleset inclusion enters the game.

thanks for this, I was thinking about how this could be improved, but this 
looks like it deals with the issue.

on my central box I currently have all the logs written to one file, roll 
that every 5 min, and then at night split this into 45 different files 
based on 100 simplified program names (where I strip out versions so that 
blah-2.3[123] and blah-2.4[123] end up in the same file). I was thinking 
of experimenting to see what happened if I did this in rsyslog instead. 
This is a very good pointer to what I would need to do.

David Lang

> Rainer
>
> [1] http://www.rsyslog.com/doc-omruleset.html
>
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> Sent: Wednesday, January 20, 2010 12:26 AM
>> To: rsyslog-users
>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
>>
>> I've now had v5.3.6 running on my production environemnets since friday
>> with no problems
>>
>> one side effect of the cleanups is that previously when I had multiple
>> filters write to one file I was getting lots of corrupt lines, but the
>> change to have omfile write each transaction rather than just as the
>> buffer filled up seems to have eliminated this (it went from 10's of
>> thousands of corrupted lines/day to none over the weekend and monday,
>> tonight's report will be the acid test to see if it's fully cleaned up)
>>
>> I realize there is still a window for corruption (if two output threads
>> running at the same time both decide they need to write at the same
>> time),
>> but it seems that in practice it's effectively gone.
>>
>> David Lang
>>
>> On Fri, 15 Jan 2010, david at lang.hm wrote:
>>
>>> On Fri, 15 Jan 2010, Michael Biebl wrote:
>>>
>>>> BTW, I'm actually surprised that you don't encounter those problems
>>>> yourself.
>>>
>>> I'm running 5.3.5 still, I haven't had time to build a new version
>> (hopefully
>>> tomorrow)
>>>
>>> David Lang
>>>
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Wed Jan 20 18:00:51 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 18:00:51 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm><alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710371B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001200851350.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371D@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 5:57 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On Wed, 20 Jan 2010, Rainer Gerhards wrote:
> 
> > David,
> >
> > thanks for the feedback - and a quick note.
> >
> > With the new engine, you can do ruleset inclusion (via omruleset[1]).
> That is
> > probably *the* method to handle files that are written to by multiple
> > actions. Of course, no need to change if all works in default config.
> But you
> > can gain some extra performance by using buffered mode (for busy
> files) and
> > to use that, you need to have only one action write to each file.
> This is
> > where ruleset inclusion enters the game.
> 
> thanks for this, I was thinking about how this could be improved, but
> this
> looks like it deals with the issue.
> 
> on my central box I currently have all the logs written to one file,
> roll
> that every 5 min, and then at night split this into 45 different files
> based on 100 simplified program names (where I strip out versions so
> that
> blah-2.3[123] and blah-2.4[123] end up in the same file). I was
> thinking
> of experimenting to see what happened if I did this in rsyslog instead.
> This is a very good pointer to what I would need to do.

I would be quite interested in feedback on omruleset. I doubt anyone has put
it into production yet, at least in a demanding environment (aka "bugs to be
expected" ;)). Note that this functionality is very hard to configure with
the current config language... (it was omruleset that made me believe that
finally something must be done to improve that part of the system).

Rainer

> 
> David Lang
> 
> > Rainer
> >
> > [1] http://www.rsyslog.com/doc-omruleset.html
> >
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >> Sent: Wednesday, January 20, 2010 12:26 AM
> >> To: rsyslog-users
> >> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>
> >> I've now had v5.3.6 running on my production environemnets since
> friday
> >> with no problems
> >>
> >> one side effect of the cleanups is that previously when I had
> multiple
> >> filters write to one file I was getting lots of corrupt lines, but
> the
> >> change to have omfile write each transaction rather than just as the
> >> buffer filled up seems to have eliminated this (it went from 10's of
> >> thousands of corrupted lines/day to none over the weekend and
> monday,
> >> tonight's report will be the acid test to see if it's fully cleaned
> up)
> >>
> >> I realize there is still a window for corruption (if two output
> threads
> >> running at the same time both decide they need to write at the same
> >> time),
> >> but it seems that in practice it's effectively gone.
> >>
> >> David Lang
> >>
> >> On Fri, 15 Jan 2010, david at lang.hm wrote:
> >>
> >>> On Fri, 15 Jan 2010, Michael Biebl wrote:
> >>>
> >>>> BTW, I'm actually surprised that you don't encounter those
> problems
> >>>> yourself.
> >>>
> >>> I'm running 5.3.5 still, I haven't had time to build a new version
> >> (hopefully
> >>> tomorrow)
> >>>
> >>> David Lang
> >>>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Wed Jan 20 18:00:56 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 09:00:56 -0800 (PST)
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm>
	<alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001200857260.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Rainer Gerhards wrote:

> I forgot to mention:
>
>> I realize there is still a window for corruption (if two output threads
>> running at the same time both decide they need to write at the same
>> time),
>> but it seems that in practice it's effectively gone.
>
> The current code writes a single line with a single API call. I guess that
> call is rather atomic from an OS point of view, so the window of corruption
> probably don't even exists with current rsyslog and linux code.

even when things are batched? with 5.3.5 I was very definantly 
experianceing problems with lines getting combined in the writes when I 
had multiple outputs to the same file (using different formats to fix up 
bad input)

David Lang


From rgerhards at hq.adiscon.com  Wed Jan 20 18:02:22 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 20 Jan 2010 18:02:22 +0100
Subject: [rsyslog] rsyslog 5.3.6 (v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142319l31c7694ar4e288ec19ca63ec8@mail.gmail.com><alpine.DEB.2.00.1001142321490.4649@asgard.lang.hm><aa76cdac1001142327w7704d70aj7c8afa794d1dbf69@mail.gmail.com><alpine.DEB.2.00.1001142333130.4649@asgard.lang.hm><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><alpine.DEB.2.00.1001142345300.4649@asgard.lang.hm><aa76cdac1001142355i667cdb9aq69effd4629b39ef2@mail.gmail.com><alpine.DEB.2.00.1001150001210.4649@asgard.lang.hm><aa76cdac1001150007s6153e5fet592ee33d7d52fc7e@mail.gmail.com><aa76cdac1001150008v352905e3y734a1c3db9ad3c80@mail.gmail.com><alpine.DEB.2.00.1001150010340.4649@asgard.lang.hm><alpine.DEB.2.00.1001191523180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710371C@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001200857260.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710371E@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Wednesday, January 20, 2010 6:01 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> On Wed, 20 Jan 2010, Rainer Gerhards wrote:
> 
> > I forgot to mention:
> >
> >> I realize there is still a window for corruption (if two output
> threads
> >> running at the same time both decide they need to write at the same
> >> time),
> >> but it seems that in practice it's effectively gone.
> >
> > The current code writes a single line with a single API call. I guess
> that
> > call is rather atomic from an OS point of view, so the window of
> corruption
> > probably don't even exists with current rsyslog and linux code.
> 
> even when things are batched? with 5.3.5 I was very definantly
> experianceing problems with lines getting combined in the writes when I
> had multiple outputs to the same file (using different formats to fix
> up
> bad input)

good point. No, you are right. With batches, buffered mode is used by
default, with a flush at the end of batch.

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From sur5r at sur5r.net  Wed Jan 20 19:20:31 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Wed, 20 Jan 2010 19:20:31 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
Message-ID: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 6 Jan 2010 16:14:59 +0100
Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:

> which encoding should be chosen for the database when using postgres?

As far as I understand the syslog protocol (at least the legacy one), it has
no concept of character encodings at all.  So if you simply want to make sure
that everything ends up in the database "as is", then choose SQL_ASCII.

> My rsyslog version is 4.4.3.
> 
> Which client_encoding does rsyslog use in ompgsql?

Right now, it does net set an encoding by itself, so the database default
applies. If I'm not mistaken, you can even set that per user from inside of
postgres. So I would rather vote against another configuration parameter here.

> I currently have set UTF-8 on the database. It worked for a while until
> some special message arrived at the server where postgres denies the INSERT:
> 
> 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for 
> encoding "UTF8": 0xd220
> 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen if
> the byte sequence does not match the encoding expected by the server, which
> is controlled by "client_encoding".

Were you able to isolate the message? Or find out which program was sending
it?

> Now rsyslog is not able to log anything... it is currently spooling to disk 
> because it "hangs" at this message not being accepted by postgres.

This is bad, because if the machine is an open syslog server that simply
collects everything it gets, we have a potential DoS vector here.

I can think of three options:

* Drop the message and report that we did so. That would be rather easy,
  but might not be what people want.

* Re-insert the message after converting it from ASCII to UTF-8 or whatever
  the DB encoding is. But this might/will produce garbage if the input is not
  ASCII. It also creates more load on the system if these messages are
  frequent. Guessing the input encoding is hard or even impossible, depending
  on the set you guess from.

* Make the database SQL_ASCII. This will silently accept anything but will
  create nonsense from UTF/UCS encoded messages. Also might create trouble
  for programs like phplogcon that analyze the logs.

For me, this sums up to one question:

Can we make ompgsql UTF/UCS-clean and at the same time not choke on non-UTF8
strings? Everyone is trying to be UTF-8 clean these days, so it would be bad
if ompgsql could not keep up.

Comments please.

Regards,
Jakab Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktXSW8ACgkQ1YAhDic+adbqXACeIJcx6GW6PhSXFO1YF72PafJG
7t8AoLNwnJYMZ4bssqMZt/nkTIPWs0LI
=vuWN
-----END PGP SIGNATURE-----

From david at lang.hm  Wed Jan 20 19:44:42 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 20 Jan 2010 10:44:42 -0800 (PST)
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>

On Wed, 20 Jan 2010, Jakob Haufe wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> This is bad, because if the machine is an open syslog server that simply
> collects everything it gets, we have a potential DoS vector here.
>
> I can think of three options:
>
> * Drop the message and report that we did so. That would be rather easy,
>  but might not be what people want.
>
> * Re-insert the message after converting it from ASCII to UTF-8 or whatever
>  the DB encoding is. But this might/will produce garbage if the input is not
>  ASCII. It also creates more load on the system if these messages are
>  frequent. Guessing the input encoding is hard or even impossible, depending
>  on the set you guess from.
>
> * Make the database SQL_ASCII. This will silently accept anything but will
>  create nonsense from UTF/UCS encoded messages. Also might create trouble
>  for programs like phplogcon that analyze the logs.
>
> For me, this sums up to one question:
>
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on non-UTF8
> strings? Everyone is trying to be UTF-8 clean these days, so it would be bad
> if ompgsql could not keep up.

my thought is that just like we have a filter to change control characters 
to escape sequences, it would be good to have a filter to escape non-ascii 
characters. this will mangle other character sets, but they are unlikly to 
go through cleanly anyway.

David Lang


From marc.schiffbauer at mightycare.de  Thu Jan 21 01:49:47 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Thu, 21 Jan 2010 01:49:47 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <201001210149.48041.marc.schiffbauer@mightycare.de>

Am Mittwoch, 20. Januar 2010 19:20:31 schrieb Jakob Haufe:
> On Wed, 6 Jan 2010 16:14:59 +0100
> 
> Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:
> > which encoding should be chosen for the database when using postgres?
> 
> As far as I understand the syslog protocol (at least the legacy one), it
> has no concept of character encodings at all.  So if you simply want to
> make sure that everything ends up in the database "as is", then choose
> SQL_ASCII.

This is what I did in the end. And it works good now.

> 
> > My rsyslog version is 4.4.3.
> > 
> > Which client_encoding does rsyslog use in ompgsql?
> 
> Right now, it does net set an encoding by itself, so the database default
> applies. If I'm not mistaken, you can even set that per user from inside of
> postgres. So I would rather vote against another configuration parameter
> here.

ACK

> 
> > I currently have set UTF-8 on the database. It worked for a while until
> > some special message arrived at the server where postgres denies the
> > INSERT:
> > 
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also happen
> > if the byte sequence does not match the encoding expected by the server,
> > which is controlled by "client_encoding".
> 
> Were you able to isolate the message? Or find out which program was sending
> it?

I was able to identify it: Some servers sent data about strings found in 
system BIOS (read by dmidecode so something like that)
It was just some strange charcters in a model or device name string set by a 
hardware vendor (compaq IIRC)


> 
> > Now rsyslog is not able to log anything... it is currently spooling to
> > disk because it "hangs" at this message not being accepted by postgres.
> 
> This is bad, because if the machine is an open syslog server that simply
> collects everything it gets, we have a potential DoS vector here.
> 

True.


> I can think of three options:
> 
> * Drop the message and report that we did so. That would be rather easy,
>   but might not be what people want.
> 

But this might be the best option I guess. Maybe the original message could 
then be written to a special logfile on disk.

> * Re-insert the message after converting it from ASCII to UTF-8 or whatever
>   the DB encoding is. But this might/will produce garbage if the input is
> not ASCII. It also creates more load on the system if these messages are
> frequent. Guessing the input encoding is hard or even impossible,
> depending on the set you guess from.

Yes but this would be an option. I would vote for creating a warning message 
in these cases as well.

> 
> * Make the database SQL_ASCII. This will silently accept anything but will
>   create nonsense from UTF/UCS encoded messages. Also might create trouble
>   for programs like phplogcon that analyze the logs.
> 

This is what I did. And phplogcon had no problems at all displaying everything 
as expected. Even those strange messages that were not accepted by postgres 
look as in the original message that came via syslog.

This might only work if apache and the browser all "speak" UTF-8.



> For me, this sums up to one question:
> 
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> non-UTF8 strings? Everyone is trying to be UTF-8 clean these days, so it
> would be bad if ompgsql could not keep up.

I think this is a special case because rsyslog is not the originator of those 
messages. It "just" transports them. And because the syslog-Protocol does not 
define something like encoding in any way the best thing to do is just leave 
those strings "as-is" and make the database behind it do so as well with 
SQL_ASCII.

I thing everythign else will be error prone in some way. The Documentation of 
rsyslog should bring a big fat NOTE that the database must be SQL_ASCII as 
other wise thesrings might not be accepted.


-Marc

> 
> Comments please.
> 
> Regards,
> Jakab Haufe (sur5r)


From xkubina at fi.muni.cz  Thu Jan 21 11:21:27 2010
From: xkubina at fi.muni.cz (Tomas Kubina)
Date: Thu, 21 Jan 2010 11:21:27 +0100
Subject: [rsyslog] How to add new configuration option
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>	<4B4DCF31.6090105@fi.muni.cz>
	<9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
Message-ID: <4B582AA7.3040906@fi.muni.cz>

Rainer Gerhards wrote:
> Thanks for the code. Unfortunately, adding the config switch to it is not
> quite easy in that case (good I asked for the actual code). I'd say that you
> best do it similar to the other config directives, like the authentication
> mode. They actual directives are in the upper level code (imtcp/omfwd).
> There, they are shuffled over to the instance data, which goes along with
> each of the configured listeners/sender. Then, when a new network stream is
> created, the params are passed down to the generic stream interface and there
> passed down to the selected stream driver, which finally stores and acts on
> them. It's clumpsy and quite some work, but that is what is needed for the
> old config system. You probably need to add around 50 to 100 lines of code
> altogether to the various files. It's not complex, but easy to forget
> something. Best start by a directive (like $..AuthMode), see how it is
> handled (and passed down) in imtcp and work your way down the stack ;)
>
> Rainer 
>
>   
Hi Rainer,

I have added some code that I have thought was necessary, but
I am stuck now. In nsd_gtls.c is added function:

static rsRetVal
SetAddClientCN(nsd_t *pNsd, int mode)
{
    DEFiRet;
    nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;

    ISOBJ_TYPE_assert((pThis), nsd_gtls);
    if(mode != 0 && mode != 1) {
        errmsg.LogError(0, RS_RET_INVALID_DRVR_MODE, "error: driver mode 
%d not supported by "
                "gtls netstream driver", mode);
        ABORT_FINALIZE(RS_RET_INVALID_DRVR_MODE);
    }
       
    pThis->iAddClientCN = mode;
        dbgprintf("GTLS:%d\n", pThis->iAddClientCN);
finalize_it:
    RETiRet;
}

The "dbgprintf" shows correct value in pThis, but if I check 
pThis->iAddClientCN
later in function:

static rsRetVal
Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
{
    DEFiRet;
    ssize_t iBytesCopy; /* how many bytes are to be copied to the client 
buffer? */
    nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
    ISOBJ_TYPE_assert(pThis, nsd_gtls);

        cstr_t *pstrCN = NULL;
        const gnutls_datum *cert_list;
    unsigned int cert_list_size = 0;
        gnutls_x509_crt cert;       
        int len = 0;       
        char *buf_temp;

    if(pThis->bAbortConn)
        ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);

    if(pThis->iMode == 0) {
        CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
        FINALIZE;
    }

    /* --- in TLS mode now --- */

    /* Buffer logic applies only if we are in TLS mode. Here we
     * assume that we will switch from plain to TLS, but never back. This
     * assumption may be unsafe, but it is the model for the time being 
and I
     * do not see any valid reason why we should switch back to plain 
TCP after
     * we were in TLS mode. However, in that case we may lose something that
     * is already in the receive buffer ... risk accepted. -- rgerhards, 
2008-06-23
     */

    if(pThis->pszRcvBuf == NULL) {
        /* we have no buffer, so we need to malloc one */
        CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
        pThis->lenRcvBuf = -1;
    }

    /* now check if we have something in our buffer. If so, we satisfy
     * the request from buffer contents.
     */
    if(pThis->lenRcvBuf == -1) { /* no data present, must read */
        CHKiRet(gtlsRecordRecv(pThis));
    }

    if(pThis->lenRcvBuf == 0) { /* EOS */
        *pLenBuf = 0;
        /* in this case, we also need to free the receive buffer, if we
         * allocated one. -- rgerhards, 2008-12-03
         */
        if(pThis->pszRcvBuf != NULL) {
            free(pThis->pszRcvBuf);
            pThis->pszRcvBuf = NULL;
        }
        ABORT_FINALIZE(RS_RET_CLOSED);
    }

    /* if we reach this point, data is present in the buffer and must be 
copied */
    iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
    if(iBytesCopy > *pLenBuf) {
        iBytesCopy = *pLenBuf;
    } else {
        pThis->lenRcvBuf = -1; /* buffer will be emptied below */
    }

        dbgprintf("!!!!!!!!!!!%d!!!!!!!!!!!!!!\n\n", pThis->iAddClientCN);
        if (pThis->iAddClientCN)
        {
          if (pThis->clientCNValid != 1)
          {
            cert_list = gnutls_certificate_get_peers(pThis->sess, 
&cert_list_size);

        if(cert_list_size > 0)
            {
          // we only print information about the first certificate
          gnutls_x509_crt_init(&cert);
          gnutls_x509_crt_import(cert, &cert_list[0], GNUTLS_X509_FMT_DER);

              CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));

              len = snprintf(NULL, 0, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
              if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char))) )
                return -1;

              snprintf(pThis->clientCN, len + 1, "CN:%s ", 
(char*)cstrGetSzStr(pstrCN));
              pThis->clientCN[len] = '\0';               
              pThis->clientCNLen = len + 1;

              pThis->clientCNValid = 1;
            }
          }
       
          iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ? 
iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;

          buf_temp = (char*)malloc(iBytesCopy);
       
          if (buf_temp)
          {
            memset(buf_temp, 0, iBytesCopy);
            strncpy(buf_temp, pThis->clientCN, iBytesCopy);
            buf_temp[strlen(buf_temp)] ='\0';
            strncat(buf_temp, pThis->pszRcvBuf, iBytesCopy - 
strlen(buf_temp));
            buf_temp[strlen(buf_temp)] ='\0';
          }        
       
          memset(pBuf, 0, *pLenBuf);
      memcpy(pBuf, buf_temp + pThis->ptrRcvBuf, iBytesCopy);

          if (buf_temp)
            free(buf_temp);
        }
        else
        {
          memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf, iBytesCopy);
    }

        pThis->ptrRcvBuf += iBytesCopy;
    *pLenBuf = iBytesCopy;

finalize_it:
    dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf 
%d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
    RETiRet;
}

The value is zero. Can you help me what I have to check in the sources code?

Thanks.

Regards,

Tomas


From sur5r at sur5r.net  Thu Jan 21 21:33:00 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Thu, 21 Jan 2010 21:33:00 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
Message-ID: <20100121213300.2abb07bf@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 20 Jan 2010 10:44:42 -0800 (PST)
david at lang.hm wrote:

> my thought is that just like we have a filter to change control characters 
> to escape sequences, it would be good to have a filter to escape non-ascii 
> characters. this will mangle other character sets, but they are unlikly to 
> go through cleanly anyway.

This is not an escaping issue, but an issue of byte sequences that are not
valid UTF8. That's why PostgreSQL rejects them. So we either need to make
ompgsql set SQL_ASCII as a client encoding (which will result in extended
characters being transcoded to UTF-8, which results in garbage) or make the
database SQL_ASCII.

Regards,
Jakob Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktYuf0ACgkQ1YAhDic+adY60QCbBqyEzDJtaEiWmg1cqKlMEJ2N
PnwAn2wAfPIpGlCOx2LdPJivrElU83Bu
=eTVw
-----END PGP SIGNATURE-----

From sur5r at sur5r.net  Thu Jan 21 22:26:26 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Thu, 21 Jan 2010 22:26:26 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<201001210149.48041.marc.schiffbauer@mightycare.de>
Message-ID: <20100121222626.083c7a49@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 21 Jan 2010 01:49:47 +0100
Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:

> Am Mittwoch, 20. Januar 2010 19:20:31 schrieb Jakob Haufe:
> > * Drop the message and report that we did so. That would be rather easy,
> >   but might not be what people want.
> > 
> 
> But this might be the best option I guess. Maybe the original message could 
> then be written to a special logfile on disk.

And then you have to check every now and then whether something ended up
there? That's not nice, and rather complex to implement as well (file name
should be configurable, maybe size limited, rotated, whatever)

> > For me, this sums up to one question:
> > 
> > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days, so it
> > would be bad if ompgsql could not keep up.
> 
> I think this is a special case because rsyslog is not the originator of
> those messages. It "just" transports them. And because the syslog-Protocol
> does not define something like encoding in any way the best thing to do is
> just leave those strings "as-is" and make the database behind it do so as
> well with SQL_ASCII.

I like the idea of seeing rsyslog as some kind of transport only. This is the
best argument for switching to SQL_ASCII altogether so far.

Rainer, do you have any thoughts on this?

> I thing everythign else will be error prone in some way. The Documentation
> of rsyslog should bring a big fat NOTE that the database must be SQL_ASCII
> as other wise thesrings might not be accepted.

Yes, and the createDB.sql for ompgsql should be changed as well.

Regards,
Jakob Haufe (sur5r)


- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
=1POP
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Fri Jan 22 10:51:41 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 10:51:41 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>

V5 has the capability to discard messages that cause an action failure.
However, this is mostly untested yet, AND the action must support it by
providing proper status information - it must differentiate between
system-induced errors (which can be retried) and message-induced errors
(which need the discard). ompgsql currently does not provide that status
information.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Wednesday, January 20, 2010 7:21 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Wed, 6 Jan 2010 16:14:59 +0100
> Marc Schiffbauer <marc.schiffbauer at mightycare.de> wrote:
> 
> > which encoding should be chosen for the database when using postgres?
> 
> As far as I understand the syslog protocol (at least the legacy one),
> it has
> no concept of character encodings at all.  So if you simply want to
> make sure
> that everything ends up in the database "as is", then choose SQL_ASCII.
> 
> > My rsyslog version is 4.4.3.
> >
> > Which client_encoding does rsyslog use in ompgsql?
> 
> Right now, it does net set an encoding by itself, so the database
> default
> applies. If I'm not mistaken, you can even set that per user from
> inside of
> postgres. So I would rather vote against another configuration
> parameter here.
> 
> > I currently have set UTF-8 on the database. It worked for a while
> until
> > some special message arrived at the server where postgres denies the
> INSERT:
> >
> > 2010-01-06 16:13:11 CET syslog syslog ERROR:  invalid byte sequence
> for
> > encoding "UTF8": 0xd220
> > 2010-01-06 16:13:11 CET syslog syslog HINT:  This error can also
> happen if
> > the byte sequence does not match the encoding expected by the server,
> which
> > is controlled by "client_encoding".
> 
> Were you able to isolate the message? Or find out which program was
> sending
> it?
> 
> > Now rsyslog is not able to log anything... it is currently spooling
> to disk
> > because it "hangs" at this message not being accepted by postgres.
> 
> This is bad, because if the machine is an open syslog server that
> simply
> collects everything it gets, we have a potential DoS vector here.
> 
> I can think of three options:
> 
> * Drop the message and report that we did so. That would be rather
> easy,
>   but might not be what people want.
> 
> * Re-insert the message after converting it from ASCII to UTF-8 or
> whatever
>   the DB encoding is. But this might/will produce garbage if the input
> is not
>   ASCII. It also creates more load on the system if these messages are
>   frequent. Guessing the input encoding is hard or even impossible,
> depending
>   on the set you guess from.
> 
> * Make the database SQL_ASCII. This will silently accept anything but
> will
>   create nonsense from UTF/UCS encoded messages. Also might create
> trouble
>   for programs like phplogcon that analyze the logs.
> 
> For me, this sums up to one question:
> 
> Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> non-UTF8
> strings? Everyone is trying to be UTF-8 clean these days, so it would
> be bad
> if ompgsql could not keep up.
> 
> Comments please.
> 
> Regards,
> Jakab Haufe (sur5r)
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktXSW8ACgkQ1YAhDic+adbqXACeIJcx6GW6PhSXFO1YF72PafJG
> 7t8AoLNwnJYMZ4bssqMZt/nkTIPWs0LI
> =vuWN
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 10:54:04 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 10:54:04 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>


> my thought is that just like we have a filter to change control
> characters
> to escape sequences, it would be good to have a filter to escape non-
> ascii
> characters. this will mangle other character sets, but they are unlikly
> to
> go through cleanly anyway.

Just to be on the right path, you suggest escaping charactes with hex values
> 7f?

Rainer


From david at lang.hm  Fri Jan 22 11:07:02 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 22 Jan 2010 02:07:02 -0800 (PST)
Subject: [rsyslog] PostgreSQL: Problems with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>

On Fri, 22 Jan 2010, Rainer Gerhards wrote:

>> my thought is that just like we have a filter to change control
>> characters
>> to escape sequences, it would be good to have a filter to escape non-
>> ascii
>> characters. this will mangle other character sets, but they are unlikly
>> to
>> go through cleanly anyway.
>
> Just to be on the right path, you suggest escaping charactes with hex values
>> 7f?

correct. they can cause as much grief (or more) than control characters.

since control characters get escaped by default, rsyslog will already 
mangle UTF8 text sent to it if the final byte is in that range.

David Lang


From rgerhards at hq.adiscon.com  Fri Jan 22 11:09:52 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:09:52 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710372A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 11:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> >> my thought is that just like we have a filter to change control
> >> characters
> >> to escape sequences, it would be good to have a filter to escape
> non-
> >> ascii
> >> characters. this will mangle other character sets, but they are
> unlikly
> >> to
> >> go through cleanly anyway.
> >
> > Just to be on the right path, you suggest escaping charactes with hex
> values
> >> 7f?
> 
> correct. they can cause as much grief (or more) than control
> characters.
> 
> since control characters get escaped by default, rsyslog will already
> mangle UTF8 text sent to it if the final byte is in that range.

jup, just wanted to be sure. that can probably be best implemented as a
property replacer option (or at the parser level, but then it applies to
everything). Note that many European languages use these characters (and
without grief), much as Asian languages use sequences which would be
destroyed by the current escaping (which thus can be turned off). But I
definitely see the value. Given it looks easy to implement, I'll see if I can
integrate an option.

Rainer

> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 11:15:59 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:15:59 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems with
	character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>

Been on the road and will be over the weekend and part of next week (thus
sluggish responses ;)).

> > > For me, this sums up to one question:
> > >
> > > Can we make ompgsql UTF/UCS-clean and at the same time not choke on
> > > non-UTF8 strings? Everyone is trying to be UTF-8 clean these days,
> so it
> > > would be bad if ompgsql could not keep up.
> >
> > I think this is a special case because rsyslog is not the originator
> of
> > those messages. It "just" transports them. And because the syslog-
> Protocol
> > does not define something like encoding in any way the best thing to
> do is
> > just leave those strings "as-is" and make the database behind it do
> so as
> > well with SQL_ASCII.
> 
> I like the idea of seeing rsyslog as some kind of transport only. This
> is the
> best argument for switching to SQL_ASCII altogether so far.
> 
> Rainer, do you have any thoughts on this?

Let me elaborte a bit: the new IETF syslog standards *do* specify character
encoding and strongly recommend Unicode (UTF-8) to be used. Of course, this
does not solve the issue with original senders that use another, unspecified,
coding. But it helps.

Unfortunately, rsyslog's "old" code is far from being Unicode-aware. As a
side-activity, I am upgrading "old" code to "new" code, which then uses
rsyslog's string classes. While they do not yet support Unicode, it is much
easier to make them support it once all string handling is done consistently.

However, even then I need to have a build time switch to turn this on/off,
because rsyslog in Unicode mode will take not only considerably more space
(especially with larger in-memory queues), it will also considerably affect
its performance (in terms of bytes, the memory transfer rate is effectively
cut in half, as most data in syslog is character-based - also think about the
effects on cache performance).

So moving the whole system to Unicode, while desirable, is far from being a
trivial task. Having seen extremely low demand for that, I have so far opted
to do this at a very low priorty (even though that means I violate RFC5424).
	
> 
> > I thing everythign else will be error prone in some way. The
> Documentation
> > of rsyslog should bring a big fat NOTE that the database must be
> SQL_ASCII
> > as other wise thesrings might not be accepted.
> 
> Yes, and the createDB.sql for ompgsql should be changed as well.
> 

The doc needs to be written so that I can add this warning ;) Is someone with
actual Postgres knowledge up for this task. Plain text is OK, I can then
copy&paste that into a module doc template.

As for createDB.sql: let me know what I need to change, and I'll apply that
change.

Rainer

> Regards,
> Jakob Haufe (sur5r)
> 
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktYxoIACgkQ1YAhDic+adZvugCffdUcjqR/EiQIGojSgEh8A8lU
> m2EAn1AZ1ebx4l+GCFqQLSvg6FqBZFvG
> =1POP
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Fri Jan 22 11:50:20 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 11:50:20 +0100
Subject: [rsyslog] How to add new configuration option
References: <4B4DAB76.7070201@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BA@GRFEXC.intern.adiscon.com>	<4B4DCF31.6090105@fi.muni.cz><9B6E2A8877C38245BFB15CC491A11DA71036BD@GRFEXC.intern.adiscon.com>
	<4B582AA7.3040906@fi.muni.cz>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103735@GRFEXC.intern.adiscon.com>

mhhh... doesn't look too bad. Maybe it's a problem with the calling sequence.
When do you call your new function? If should be called after the
nsdConstruct but before the nsdConsructFinalize (actual function names may be
slightly different).

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Tomas Kubina
> Sent: Thursday, January 21, 2010 11:21 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] How to add new configuration option
> 
> Rainer Gerhards wrote:
> > Thanks for the code. Unfortunately, adding the config switch to it is
> not
> > quite easy in that case (good I asked for the actual code). I'd say
> that you
> > best do it similar to the other config directives, like the
> authentication
> > mode. They actual directives are in the upper level code
> (imtcp/omfwd).
> > There, they are shuffled over to the instance data, which goes along
> with
> > each of the configured listeners/sender. Then, when a new network
> stream is
> > created, the params are passed down to the generic stream interface
> and there
> > passed down to the selected stream driver, which finally stores and
> acts on
> > them. It's clumpsy and quite some work, but that is what is needed
> for the
> > old config system. You probably need to add around 50 to 100 lines of
> code
> > altogether to the various files. It's not complex, but easy to forget
> > something. Best start by a directive (like $..AuthMode), see how it
> is
> > handled (and passed down) in imtcp and work your way down the stack
> ;)
> >
> > Rainer
> >
> >
> Hi Rainer,
> 
> I have added some code that I have thought was necessary, but
> I am stuck now. In nsd_gtls.c is added function:
> 
> static rsRetVal
> SetAddClientCN(nsd_t *pNsd, int mode)
> {
>     DEFiRet;
>     nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
> 
>     ISOBJ_TYPE_assert((pThis), nsd_gtls);
>     if(mode != 0 && mode != 1) {
>         errmsg.LogError(0, RS_RET_INVALID_DRVR_MODE, "error: driver
> mode
> %d not supported by "
>                 "gtls netstream driver", mode);
>         ABORT_FINALIZE(RS_RET_INVALID_DRVR_MODE);
>     }
> 
>     pThis->iAddClientCN = mode;
>         dbgprintf("GTLS:%d\n", pThis->iAddClientCN);
> finalize_it:
>     RETiRet;
> }
> 
> The "dbgprintf" shows correct value in pThis, but if I check
> pThis->iAddClientCN
> later in function:
> 
> static rsRetVal
> Rcv(nsd_t *pNsd, uchar *pBuf, ssize_t *pLenBuf)
> {
>     DEFiRet;
>     ssize_t iBytesCopy; /* how many bytes are to be copied to the
> client
> buffer? */
>     nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd;
>     ISOBJ_TYPE_assert(pThis, nsd_gtls);
> 
>         cstr_t *pstrCN = NULL;
>         const gnutls_datum *cert_list;
>     unsigned int cert_list_size = 0;
>         gnutls_x509_crt cert;
>         int len = 0;
>         char *buf_temp;
> 
>     if(pThis->bAbortConn)
>         ABORT_FINALIZE(RS_RET_CONNECTION_ABORTREQ);
> 
>     if(pThis->iMode == 0) {
>         CHKiRet(nsd_ptcp.Rcv(pThis->pTcp, pBuf, pLenBuf));
>         FINALIZE;
>     }
> 
>     /* --- in TLS mode now --- */
> 
>     /* Buffer logic applies only if we are in TLS mode. Here we
>      * assume that we will switch from plain to TLS, but never back.
> This
>      * assumption may be unsafe, but it is the model for the time being
> and I
>      * do not see any valid reason why we should switch back to plain
> TCP after
>      * we were in TLS mode. However, in that case we may lose something
> that
>      * is already in the receive buffer ... risk accepted. --
> rgerhards,
> 2008-06-23
>      */
> 
>     if(pThis->pszRcvBuf == NULL) {
>         /* we have no buffer, so we need to malloc one */
>         CHKmalloc(pThis->pszRcvBuf = MALLOC(NSD_GTLS_MAX_RCVBUF));
>         pThis->lenRcvBuf = -1;
>     }
> 
>     /* now check if we have something in our buffer. If so, we satisfy
>      * the request from buffer contents.
>      */
>     if(pThis->lenRcvBuf == -1) { /* no data present, must read */
>         CHKiRet(gtlsRecordRecv(pThis));
>     }
> 
>     if(pThis->lenRcvBuf == 0) { /* EOS */
>         *pLenBuf = 0;
>         /* in this case, we also need to free the receive buffer, if we
>          * allocated one. -- rgerhards, 2008-12-03
>          */
>         if(pThis->pszRcvBuf != NULL) {
>             free(pThis->pszRcvBuf);
>             pThis->pszRcvBuf = NULL;
>         }
>         ABORT_FINALIZE(RS_RET_CLOSED);
>     }
> 
>     /* if we reach this point, data is present in the buffer and must
> be
> copied */
>     iBytesCopy = pThis->lenRcvBuf - pThis->ptrRcvBuf;
>     if(iBytesCopy > *pLenBuf) {
>         iBytesCopy = *pLenBuf;
>     } else {
>         pThis->lenRcvBuf = -1; /* buffer will be emptied below */
>     }
> 
>         dbgprintf("!!!!!!!!!!!%d!!!!!!!!!!!!!!\n\n", pThis-
> >iAddClientCN);
>         if (pThis->iAddClientCN)
>         {
>           if (pThis->clientCNValid != 1)
>           {
>             cert_list = gnutls_certificate_get_peers(pThis->sess,
> &cert_list_size);
> 
>         if(cert_list_size > 0)
>             {
>           // we only print information about the first certificate
>           gnutls_x509_crt_init(&cert);
>           gnutls_x509_crt_import(cert, &cert_list[0],
> GNUTLS_X509_FMT_DER);
> 
>               CHKiRet(gtlsGetCN(pThis, &cert, &pstrCN));
> 
>               len = snprintf(NULL, 0, "CN:%s ",
> (char*)cstrGetSzStr(pstrCN));
>               if ( !(pThis->clientCN = malloc((len + 1)*sizeof(char)))
> )
>                 return -1;
> 
>               snprintf(pThis->clientCN, len + 1, "CN:%s ",
> (char*)cstrGetSzStr(pstrCN));
>               pThis->clientCN[len] = '\0';
>               pThis->clientCNLen = len + 1;
> 
>               pThis->clientCNValid = 1;
>             }
>           }
> 
>           iBytesCopy = iBytesCopy + pThis->clientCNLen - 1 < *pLenBuf ?
> iBytesCopy + pThis->clientCNLen - 1 : *pLenBuf;
> 
>           buf_temp = (char*)malloc(iBytesCopy);
> 
>           if (buf_temp)
>           {
>             memset(buf_temp, 0, iBytesCopy);
>             strncpy(buf_temp, pThis->clientCN, iBytesCopy);
>             buf_temp[strlen(buf_temp)] ='\0';
>             strncat(buf_temp, pThis->pszRcvBuf, iBytesCopy -
> strlen(buf_temp));
>             buf_temp[strlen(buf_temp)] ='\0';
>           }
> 
>           memset(pBuf, 0, *pLenBuf);
>       memcpy(pBuf, buf_temp + pThis->ptrRcvBuf, iBytesCopy);
> 
>           if (buf_temp)
>             free(buf_temp);
>         }
>         else
>         {
>           memcpy(pBuf, pThis->pszRcvBuf + pThis->ptrRcvBuf,
> iBytesCopy);
>     }
> 
>         pThis->ptrRcvBuf += iBytesCopy;
>     *pLenBuf = iBytesCopy;
> 
> finalize_it:
>     dbgprintf("gtlsRcv return. nsd %p, iRet %d, lenRcvBuf %d, ptrRcvBuf
> %d\n", pThis, iRet, pThis->lenRcvBuf, pThis->ptrRcvBuf);
>     RETiRet;
> }
> 
> The value is zero. Can you help me what I have to check in the sources
> code?
> 
> Thanks.
> 
> Regards,
> 
> Tomas
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From marc.schiffbauer at mightycare.de  Fri Jan 22 15:20:01 2010
From: marc.schiffbauer at mightycare.de (Marc Schiffbauer)
Date: Fri, 22 Jan 2010 15:20:01 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
	<9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
Message-ID: <201001221520.02432.marc.schiffbauer@mightycare.de>

Am Freitag, 22. Januar 2010 11:15:59 schrieb Rainer Gerhards:
> 
> The doc needs to be written so that I can add this warning ;) Is someone
> with actual Postgres knowledge up for this task. Plain text is OK, I can
> then copy&paste that into a module doc template.
> 
> As for createDB.sql: let me know what I need to change, and I'll apply that
> change.
> 

I can write it. I will send it to you/this thread next week!

Have a nice weekend
-Marc


From rgerhards at hq.adiscon.com  Fri Jan 22 17:09:21 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 22 Jan 2010 17:09:21 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problemswith
	character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<201001221520.02432.marc.schiffbauer@mightycare.de>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710373A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Marc Schiffbauer
> Sent: Friday, January 22, 2010 3:20 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problemswith character encoding
> 
> Am Freitag, 22. Januar 2010 11:15:59 schrieb Rainer Gerhards:
> >
> > The doc needs to be written so that I can add this warning ;) Is
> someone
> > with actual Postgres knowledge up for this task. Plain text is OK, I
> can
> > then copy&paste that into a module doc template.
> >
> > As for createDB.sql: let me know what I need to change, and I'll
> apply that
> > change.
> >
> 
> I can write it. I will send it to you/this thread next week!

excellent! looking forward to it :)

Rainer


From david at lang.hm  Fri Jan 22 19:19:25 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 22 Jan 2010 10:19:25 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
 with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de>
	<20100121222626.083c7a49@samsa>
	<9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>

On Fri, 22 Jan 2010, Rainer Gerhards wrote:

> However, even then I need to have a build time switch to turn this on/off,
> because rsyslog in Unicode mode will take not only considerably more space
> (especially with larger in-memory queues), it will also considerably affect
> its performance (in terms of bytes, the memory transfer rate is effectively
> cut in half, as most data in syslog is character-based - also think about the
> effects on cache performance).

if the code uses UTF-8 throughout this doesn't make sense. assuming the 
input is plain ascii, UTF-8 strings and ASCII strings should be the same 
size (there is some additional cpu cycles involved to figure out the 
length in characters for any output routines that grab substrings, but 
that should be all)

the only way things would take double the space (and therefor halve the 
memory transfer rate) is if it converts everything to UTF-16 strings 
internally. This is a bad idea to start with as UTF-16 does not handle all 
characters (which is why there is UTF-32 as well), but also because UTF-16 
is significantly more expensive to store/copy/etc than UTF-8 for the 
common case where most of the characters are ASCII.

It may be that you have picked the wrong string library to use. prior to 
UTF-8 being defined 'unicode' and UTF-16 were basicly synonomous and a 
_lot_ of string libraries have been written with this assumption 
(converting everything to UTF-16 on input and to whatever on output). If 
you can find one that can handle the strings as UTF-8 internally it should 
be able to just about eliminate the overhead.

David Lang


From sur5r at sur5r.net  Sun Jan 24 20:14:41 2010
From: sur5r at sur5r.net (Jakob Haufe)
Date: Sun, 24 Jan 2010 20:14:41 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de>
	<20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de>
	<9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>
Message-ID: <20100124201441.7990c850@samsa>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 22 Jan 2010 10:51:41 +0100
"Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:

> V5 has the capability to discard messages that cause an action failure.
> However, this is mostly untested yet, AND the action must support it by
> providing proper status information - it must differentiate between
> system-induced errors (which can be retried) and message-induced errors
> (which need the discard). ompgsql currently does not provide that status
> information.

If you can point me at some example code or docs on how to do this, I would
like to try and add this functionality to ompgsql. Does ommysql already
implement that?

Regards,
Jakob Haufe (sur5r)

- -- 
ceterum censeo microsoftem esse delendam.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAktcnCEACgkQ1YAhDic+ada9WACeMkawcNTL/lt5E70mWeVjd38G
ARoAn1OAkEqm7NXRMwwVzUDC3B/2TeCB
=eDPw
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Mon Jan 25 08:34:53 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 08:34:53 +0100
Subject: [rsyslog] PostgreSQL: Problems with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><9B6E2A8877C38245BFB15CC491A11DA7103728@GRFEXC.intern.adiscon.com>
	<20100124201441.7990c850@samsa>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710373F@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jakob Haufe
> Sent: Sunday, January 24, 2010 8:15 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On Fri, 22 Jan 2010 10:51:41 +0100
> "Rainer Gerhards" <rgerhards at hq.adiscon.com> wrote:
> 
> > V5 has the capability to discard messages that cause an action
> failure.
> > However, this is mostly untested yet, AND the action must support it
> by
> > providing proper status information - it must differentiate between
> > system-induced errors (which can be retried) and message-induced
> errors
> > (which need the discard). ompgsql currently does not provide that
> status
> > information.
> 
> If you can point me at some example code or docs on how to do this, I
> would
> like to try and add this functionality to ompgsql. Does ommysql already
> implement that?


It's pretty new functionality and there is not yet a good example plugin that
uses it (it makes most sense for database plugins, where I have limited
knowledge). It would be useful to read this first (unfortunately not an easy
read:

http://download.rsyslog.com/design.pdf

Actually implementing it is rather easy. The core point is that for
system-induced errors (those that can be retried) the plugin must return
RS_RET_SUSPENDED and for message-induced errors it must return an "real"
error state (like RS_RET_ERR, but it would be better, and I'd be glad to
include, more precise error codes). The core engine than knows what to do.
Well, the core may have undiscovered bugs right now, as this functionality
was never before used in practice. It is very criticial to think about which
error class a failure belongs to. Messages with message-induced errors are
simply thrown away, so one needs to think twice before assigning this class -
but on the contrary if such a message is flagged as system-induced, it will
block the system, just as you can currently see...

HTH
Rainer

> 
> Regards,
> Jakob Haufe (sur5r)
> 
> - --
> ceterum censeo microsoftem esse delendam.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
> 
> iEYEARECAAYFAktcnCEACgkQ1YAhDic+ada9WACeMkawcNTL/lt5E70mWeVjd38G
> ARoAn1OAkEqm7NXRMwwVzUDC3B/2TeCB
> =eDPw
> -----END PGP SIGNATURE-----
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Mon Jan 25 09:12:08 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 09:12:08 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL: Problems
	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 7:19 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> > However, even then I need to have a build time switch to turn this
> on/off,
> > because rsyslog in Unicode mode will take not only considerably more
> space
> > (especially with larger in-memory queues), it will also considerably
> affect
> > its performance (in terms of bytes, the memory transfer rate is
> effectively
> > cut in half, as most data in syslog is character-based - also think
> about the
> > effects on cache performance).

David,

we need to make a distinction between UTF, a transformation (and transfer)
format and UCS, the actual native encoding format here. I think you mix these
two things up. Unicode has two (primary) flavors, which are usually encoded
in UCS-16 and UCS-32 (or ws it named UCS-2 and UCS-4 - guess so), being 2 and
4 bytes respectively. UCS-16 is what is implemented for example in Windows.
It covers many of this worlds scripts, but has proven to not cover all, which
caused additional code tables and UCS-32 presentation (at least as far as I
know, I am not an Unicode expert ;)).

UTF-8 is an encoding of Unicode code tables. You can think of it as
traditional multi-byte character set which means each character takes up a
varying number of bytes. Usually, UTF representations are converted into UCS
and then UCS is used to do the processing. While UCS requires more bytes, UTF
requires parsing of the message *each time* it is processed (e.g. to check
for a string match, count character sizes, obtain a substring). So using UTF
may use up fewer bytes, but can very considerably increase processing time
need and program complexity. For US-ASCII, of course, this is no problem. But
for other encodings, the performance hit can be very sever, much more than
the hit by double memory consumption (UCS-2 is still being considered as
"sufficient" for almost all cases, even in the future).

So I don't think it would serve the non-US-ASCII world well to process the
transformation formats. I guess that's a good option if you have a US-ASCII
based system that only very occasionally needs to process a foreign language
string (and even then, you need to parse the message *each* time you access
it, specifically when obtaining substrings...).

My conclusion is that rsyslog needs to do a UTF to UCS conversion on entry to
the system and then uses UCS internally (and converts back when messages are
output). Many software systems do so, and, as I said, IMHO do so for good
reasons.

Rainer

> 
> if the code uses UTF-8 throughout this doesn't make sense. assuming the
> input is plain ascii, UTF-8 strings and ASCII strings should be the
> same
> size (there is some additional cpu cycles involved to figure out the
> length in characters for any output routines that grab substrings, but
> that should be all)
> 
> the only way things would take double the space (and therefor halve the
> memory transfer rate) is if it converts everything to UTF-16 strings
> internally. This is a bad idea to start with as UTF-16 does not handle
> all
> characters (which is why there is UTF-32 as well), but also because
> UTF-16
> is significantly more expensive to store/copy/etc than UTF-8 for the
> common case where most of the characters are ASCII.
> 
> It may be that you have picked the wrong string library to use. prior
> to
> UTF-8 being defined 'unicode' and UTF-16 were basicly synonomous and a
> _lot_ of string libraries have been written with this assumption
> (converting everything to UTF-16 on input and to whatever on output).
> If
> you can find one that can handle the strings as UTF-8 internally it
> should
> be able to just about eliminate the overhead.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From david at lang.hm  Mon Jan 25 09:42:32 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 25 Jan 2010 00:42:32 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
 Problems	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
Message-ID: <alpine.DEB.2.00.1001250020090.3093@asgard.lang.hm>

On Mon, 25 Jan 2010, Rainer Gerhards wrote:

> David,
>
> we need to make a distinction between UTF, a transformation (and transfer)
> format and UCS, the actual native encoding format here. I think you mix these
> two things up. Unicode has two (primary) flavors, which are usually encoded
> in UCS-16 and UCS-32 (or ws it named UCS-2 and UCS-4 - guess so), being 2 and
> 4 bytes respectively. UCS-16 is what is implemented for example in Windows.
> It covers many of this worlds scripts, but has proven to not cover all, which
> caused additional code tables and UCS-32 presentation (at least as far as I
> know, I am not an Unicode expert ;)).
>
> UTF-8 is an encoding of Unicode code tables. You can think of it as
> traditional multi-byte character set which means each character takes up a
> varying number of bytes. Usually, UTF representations are converted into UCS
> and then UCS is used to do the processing. While UCS requires more bytes, UTF
> requires parsing of the message *each time* it is processed (e.g. to check
> for a string match, count character sizes, obtain a substring). So using UTF
> may use up fewer bytes, but can very considerably increase processing time
> need and program complexity. For US-ASCII, of course, this is no problem. But
> for other encodings, the performance hit can be very sever, much more than
> the hit by double memory consumption (UCS-2 is still being considered as
> "sufficient" for almost all cases, even in the future).

thanks for the clarification on terms. I had the basic understanding, but 
not the exact terminology.

> So I don't think it would serve the non-US-ASCII world well to process the
> transformation formats. I guess that's a good option if you have a US-ASCII
> based system that only very occasionally needs to process a foreign language
> string (and even then, you need to parse the message *each* time you access
> it, specifically when obtaining substrings...).
>
> My conclusion is that rsyslog needs to do a UTF to UCS conversion on entry to
> the system and then uses UCS internally (and converts back when messages are
> output). Many software systems do so, and, as I said, IMHO do so for good
> reasons.

the question is how many different places/times are we parsing the data as 
strings, vs how many places are we just moving the data around as 
essentially opaque blobs.

when we receive and parse the message we have to deal with the data as 
strings of characters, but this is generally done in one pass through the 
input data, so it would be about the same to process the data as-is as to 
convert it to UCS-2 (let alone then processing it as UCS-2). This pass can 
calculate the number of characters in the string (i.e. 'length') and store 
it

then these parsed chunks of data get copied around (in complex 
configurations with many queues, they get copied around a LOT).

At some point (or points) comparisons are made, but in most cases these 
comparisons can be done byte-by-byte, you don't actually have to parse the 
data (for regex matches you do, and for contains you would have to check 
the byte prior to the start of the match to make sure that that first 
matching byte isn't the tail end of a prior character, but I think that's 
it)

and then eventually we create the output string. At that point we are 
assembling the string from the various substrings that we have stored 
(which still can be treated as a series of bytes). It's only when the 
property replacer is invoked with either character positions or options 
that the data needs to be treated as a UTF-8 string instead of a series of 
bytes again. Yes there are a lot of things that it can do, but how much 
are they used in real life (other than setting a max length, which could 
be special cased to not be checked if the number of bytes is less than 
the length you are checking against)?

Remember that this is not general-purpose input and output that we are 
dealing with, it's logs. And like it or not, most logs really are in 
ASCII, simply because for so many years there was no option.

Also consider that the input and output stages can be split into multiple 
worker threads, while the queue manipulation (and copying) is done inside 
locks.

It may be best to leave the data as UTF-8 unless the property replacer has 
been given options, and then let the property replacer convert the data, 
work on it, and convert it back (if there is more than one option being 
invoked)

David Lang


From zhengfeng at cn.fujitsu.com  Mon Jan 25 11:36:47 2010
From: zhengfeng at cn.fujitsu.com (zhengfeng)
Date: Mon, 25 Jan 2010 18:36:47 +0800
Subject: [rsyslog] help: what induced syslogd test results are so fluctuated?
Message-ID: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>

Hi~,all

For test the performance of syslogd in RHEL5.4GA, I had written one small program by calling the interfaces syslog()...

But the results is not very steady.
Please look the results below:

The 1st time, some days ago, I test 10 times, every time sending logs to syslogd 30 secs, after then reboot.

Results:
      1,110  1,101  1,103  1,092  1,088  1,101  1,098  1,096  1,087  1,087  2.12% 



But Today , I use the same codes and method to test , the results are:

      1,295  1,292  1,297  1,291  1,288  1,287  1,284  1,279  1,275  1,270  2.13% 


(1290-1100)/1100 > 10%

Why the resluts upwards are so different? What induced? And how can i avoid that?

Thanks a lot.:-D  

From david at lang.hm  Mon Jan 25 14:10:44 2010
From: david at lang.hm (david at lang.hm)
Date: Mon, 25 Jan 2010 05:10:44 -0800 (PST)
Subject: [rsyslog] help: what induced syslogd test results are so
 fluctuated?
In-Reply-To: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>
References: <004001ca9daa$4c0a1210$8d8da70a@fnst9f95f00c19>
Message-ID: <alpine.DEB.2.00.1001250500200.3093@asgard.lang.hm>

On Mon, 25 Jan 2010, zhengfeng wrote:

> Hi~,all
>
> For test the performance of syslogd in RHEL5.4GA, I had written one small program by calling the interfaces syslog()...
>
> But the results is not very steady.
> Please look the results below:
>
> The 1st time, some days ago, I test 10 times, every time sending logs to syslogd 30 secs, after then reboot.
>
> Results:
>      1,110  1,101  1,103  1,092  1,088  1,101  1,098  1,096  1,087  1,087  2.12%
>
>
>
> But Today , I use the same codes and method to test , the results are:
>
>      1,295  1,292  1,297  1,291  1,288  1,287  1,284  1,279  1,275  1,270  2.13%
>
>
> (1290-1100)/1100 > 10%
>
> Why the resluts upwards are so different? What induced? And how can i avoid that?

there are a lot of things that could be causing this. However you didn't 
give us enough information to figure it out.

what else is running on the system?

what are the specs of the system? (is it a single core single processor, 
or do you have multiple processors)

what filesystem are you using?

when you say that you write logs for 30 seconds and reboot, are you 
allowing rsyslog to flush out pending writes, or are you loosing all logs 
that haven't been written yet (this will also involve what version of 
rsyslog are you testing and how do you have it configured)

how much cpu is rsyslog using during the time that it is running the test? 
(total time, and if you have multiple cpu cores on the system, the peak 
cpu of individual threads)

how large is the queue that rsyslog is allowed to use?

if the numbers you are reporting are the total logs written, they seem 
very low. on a current rsyslog with reasonable hardware I would expect the 
numbers to be tens of thousands of log messages per second, it may be that 
the bottleneck is the process writing to syslog() rather than rsyslog 
itself.

30 seconds is a very short time for a test, depending on your filesystem 
it may not have written anything out to the disk by the time you finish 
the test.


David Lang


From rgerhards at hq.adiscon.com  Mon Jan 25 14:38:30 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 25 Jan 2010 14:38:30 +0100
Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog 5.3.6
	(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>

Hi Michael,

finally, good news: I finally managed to reproduce the problem under 32-bit
Debian sid. It looks like only v5 is affected, and not the quite similar
v4-beta. I will now try to pinpoint the problem (hoping that the repro is
stable).

Will post more news when I have it.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> Sent: Sunday, January 17, 2010 12:52 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> 
> 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>> -----Original Message-----
> >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> >>> Sent: Friday, January 15, 2010 11:57 PM
> >>> To: rsyslog-users
> >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >>>
> >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> >>> > Michael,
> >>> >
> >>> > Fix now in git, links at the bug tracker:
> >>> >
> >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> >>> >
> >>> > Please let me know if it works for you (the patch is a bit
> trickier
> >>> than it
> >>> > looks, so confirmations would be good).
> >>>
> >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of 5.3.6.
> >>> But now I'm getting a crash when rsyslog encounters the xconsole
> pipe
> >>> config.
> >>
> >> I am a bit puzzled, but will try to reproduce that on my Debian box.
> I assume
> >> stock Debian config?
> >
> > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and then
> > got the crash. I use the default rsyslog.conf from the official
> debian
> > package.
> 
> As an additonal hint: If I start xconsole (a process reading from
> /dev/xconsole) before I start rsyslogd, then the crash does not occur.
> 
> 
> 
> --
> Why is it that all of the instruments seeking intelligent life in the
> universe are pointed away from Earth?
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Tue Jan 26 05:18:04 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Tue, 26 Jan 2010 04:18:04 +0000
Subject: [rsyslog] [patch]: fix from-host dns name reporting,
 add microseconds to MySQL
In-Reply-To: <mailman.112.1264479218.2883.rsyslog@lists.adiscon.com>
References: <mailman.112.1264479218.2883.rsyslog@lists.adiscon.com>
Message-ID: <4B5E6CFC.7070303@p6m7g8.com>

re-send from subscribed address

> ------------------------------------------------------------------------
> 
> Subject:
> [patch]: fix from-host dns name reporting, add microseconds to MySQL
> From:
> "Philip M. Gollucci" <pgollucci at ridecharge.com>
> Date:
> Mon, 25 Jan 2010 20:04:10 -0800
> To:
> rsyslog-users <rsyslog at lists.adiscon.com>
> 
> To:
> rsyslog-users <rsyslog at lists.adiscon.com>
> CC:
> "cristianorolim at hotmail.com" <cristianorolim at hotmail.com>
> 
> 
> Hi,
> 
> I have the following local patches running on a patched 5.3.6 on 50+ 
> FreeBSD machines at $work.
> 
> 1) I wanted the FQDN for $from-host
>     yes, I have this var set $PreserveFQDN to on
> 
> 2) I *need* microseconds int time:::*
> 3) Optionally add an #ifdef for the _PATH_MODDIR
>     to get the right default for fbsd
> 
> Maybe someone can explain to me why getting the host name is so complex, 
> it shouldn't be.
> 
> You can fetch them here --
> ASF mirror
> 1)
> http://people.freebsd.org/~pgollucci/patch-runtime__datetime.c
> http://people.freebsd.org/~pgollucci/patch-runtime__msg.c
> 2,3)
> http://people.freebsd.org/~pgollucci/patch-tools__syslogd.c
> 
> FreeBSD mirror
> 1)
> http://people.apache.org/~pgollucci/patch-runtime__datetime.c
> 2,3)
> http://people.apache.org/~pgollucci/patch-runtime__msg.c
> http://people.apache.org/~pgollucci/patch-tools__syslogd.c
> 
> 
> 
> 


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From rgerhards at hq.adiscon.com  Tue Jan 26 16:44:57 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 16:44:57 +0100
Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
	5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>

An update: this bug is *not* related to debian, but requires certain compiler
settings. I now also get it on Fedora. Also, it has nothing to do with the
named pipe. In fact, there seems to be a problem with the way direct queues
are handled. I don't have full details yet, but finally I begin to understand
the issue. It is a v5-only bug, introduced by the new queue engine. Direct
queues (at least action queues) can cause a segfault, at least if something
goes wrong in the action. Will post more details and/or a fix when I have
better info.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, January 25, 2010 2:39 PM
> To: rsyslog-users
> Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
> 5.3.6(v5-beta) released
> 
> Hi Michael,
> 
> finally, good news: I finally managed to reproduce the problem under
> 32-bit
> Debian sid. It looks like only v5 is affected, and not the quite
> similar
> v4-beta. I will now try to pinpoint the problem (hoping that the repro
> is
> stable).
> 
> Will post more news when I have it.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > Sent: Sunday, January 17, 2010 12:52 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> >
> > 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > >>> -----Original Message-----
> > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > >>> Sent: Friday, January 15, 2010 11:57 PM
> > >>> To: rsyslog-users
> > >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > >>>
> > >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > >>> > Michael,
> > >>> >
> > >>> > Fix now in git, links at the bug tracker:
> > >>> >
> > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> > >>> >
> > >>> > Please let me know if it works for you (the patch is a bit
> > trickier
> > >>> than it
> > >>> > looks, so confirmations would be good).
> > >>>
> > >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of
> 5.3.6.
> > >>> But now I'm getting a crash when rsyslog encounters the xconsole
> > pipe
> > >>> config.
> > >>
> > >> I am a bit puzzled, but will try to reproduce that on my Debian
> box.
> > I assume
> > >> stock Debian config?
> > >
> > > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and
> then
> > > got the crash. I use the default rsyslog.conf from the official
> > debian
> > > package.
> >
> > As an additonal hint: If I start xconsole (a process reading from
> > /dev/xconsole) before I start rsyslogd, then the crash does not
> occur.
> >
> >
> >
> > --
> > Why is it that all of the instruments seeking intelligent life in the
> > universe are pointed away from Earth?
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Tue Jan 26 17:48:19 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 17:48:19 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid - was:RE:
	rsyslog5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001142342y5ff598c1va75ae11eba6fca72@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D2@GRFEXC.intern.adiscon.com><aa76cdac1001150545l201b8a8bycc0d32b0630da256@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036D8@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>

OK, once the problematic spot is found, a fix is not far away...

The (very small) patch is self-explanatory, please see:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bab3ee566c883ac88df369ec
32df0c9100f97343

I've run it through a couple of tests now, and both theory and practice seem
to agree that this was the bug. 

Michael,
I'd appreciate if you could check if this solves the issue for you as well.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Tuesday, January 26, 2010 4:45 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] rsyslog v5 crash on Debian sid - was:RE:
> rsyslog5.3.6(v5-beta) released
> 
> An update: this bug is *not* related to debian, but requires certain
> compiler
> settings. I now also get it on Fedora. Also, it has nothing to do with
> the
> named pipe. In fact, there seems to be a problem with the way direct
> queues
> are handled. I don't have full details yet, but finally I begin to
> understand
> the issue. It is a v5-only bug, introduced by the new queue engine.
> Direct
> queues (at least action queues) can cause a segfault, at least if
> something
> goes wrong in the action. Will post more details and/or a fix when I
> have
> better info.
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Monday, January 25, 2010 2:39 PM
> > To: rsyslog-users
> > Subject: [rsyslog] rsyslog v5 crash on Debian sid - was:RE: rsyslog
> > 5.3.6(v5-beta) released
> >
> > Hi Michael,
> >
> > finally, good news: I finally managed to reproduce the problem under
> > 32-bit
> > Debian sid. It looks like only v5 is affected, and not the quite
> > similar
> > v4-beta. I will now try to pinpoint the problem (hoping that the
> repro
> > is
> > stable).
> >
> > Will post more news when I have it.
> >
> > Rainer
> >
> > > -----Original Message-----
> > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > > Sent: Sunday, January 17, 2010 12:52 PM
> > > To: rsyslog-users
> > > Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > >
> > > 2010/1/17 Michael Biebl <mbiebl at gmail.com>:
> > > > 2010/1/17 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > >>> -----Original Message-----
> > > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > > >>> bounces at lists.adiscon.com] On Behalf Of Michael Biebl
> > > >>> Sent: Friday, January 15, 2010 11:57 PM
> > > >>> To: rsyslog-users
> > > >>> Subject: Re: [rsyslog] rsyslog 5.3.6 (v5-beta) released
> > > >>>
> > > >>> 2010/1/15 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> > > >>> > Michael,
> > > >>> >
> > > >>> > Fix now in git, links at the bug tracker:
> > > >>> >
> > > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=169
> > > >>> >
> > > >>> > Please let me know if it works for you (the patch is a bit
> > > trickier
> > > >>> than it
> > > >>> > looks, so confirmations would be good).
> > > >>>
> > > >>> I applied 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a on top of
> > 5.3.6.
> > > >>> But now I'm getting a crash when rsyslog encounters the
> xconsole
> > > pipe
> > > >>> config.
> > > >>
> > > >> I am a bit puzzled, but will try to reproduce that on my Debian
> > box.
> > > I assume
> > > >> stock Debian config?
> > > >
> > > > Yes. As said, I just downloaded the 5.3.6 tarball applied the
> > > > 5b4e06fc28ef217e9ca26611e11afd974bdd1a4a patch on top of it and
> > then
> > > > got the crash. I use the default rsyslog.conf from the official
> > > debian
> > > > package.
> > >
> > > As an additonal hint: If I start xconsole (a process reading from
> > > /dev/xconsole) before I start rsyslogd, then the crash does not
> > occur.
> > >
> > >
> > >
> > > --
> > > Why is it that all of the instruments seeking intelligent life in
> the
> > > universe are pointed away from Earth?
> > > _______________________________________________
> > > rsyslog mailing list
> > > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From mbiebl at gmail.com  Tue Jan 26 18:57:10 2010
From: mbiebl at gmail.com (Michael Biebl)
Date: Tue, 26 Jan 2010 18:57:10 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid - was:RE:
	rsyslog5.3.6(v5-beta) released
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com>
	<aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com>
	<aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com>
	<aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com>
	<aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com>
	<9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
Message-ID: <aa76cdac1001260957r6d7cae7gc2ad4f21e3955fc2@mail.gmail.com>

010/1/26 Rainer Gerhards <rgerhards at hq.adiscon.com>:
> OK, once the problematic spot is found, a fix is not far away...
>
> The (very small) patch is self-explanatory, please see:
>
> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=bab3ee566c883ac88df369ec
> 32df0c9100f97343
>
> I've run it through a couple of tests now, and both theory and practice seem
> to agree that this was the bug.
>
> Michael,
> I'd appreciate if you could check if this solves the issue for you as well.

Looks like you nailed the bug.

I can no longer reproduce the crash with the above patch applied.

Cheers,
Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?


From rgerhards at hq.adiscon.com  Tue Jan 26 18:59:07 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 26 Jan 2010 18:59:07 +0100
Subject: [rsyslog] PATCH - rsyslog v5 crash on Debian sid -
	was:RE:rsyslog5.3.6(v5-beta) released
References: <9B6E2A8877C38245BFB15CC491A11DA71036BB@GRFEXC.intern.adiscon.com><aa76cdac1001150702qa21258bk32e267c490d50400@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036DD@GRFEXC.intern.adiscon.com><aa76cdac1001151457h46f6a465s5ca9b621857a170d@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA71036E5@GRFEXC.intern.adiscon.com><aa76cdac1001170348k51afaa1bp23c4927051dcea93@mail.gmail.com><aa76cdac1001170351s549fab67p7ef46e2667a59602@mail.gmail.com><9B6E2A8877C38245BFB15CC491A11DA710374B@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7103765@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7103767@GRFEXC.intern.adiscon.com>
	<aa76cdac1001260957r6d7cae7gc2ad4f21e3955fc2@mail.gmail.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103769@GRFEXC.intern.adiscon.com>

> > Michael,
> > I'd appreciate if you could check if this solves the issue for you as
> well.
> 
> Looks like you nailed the bug.
> 
> I can no longer reproduce the crash with the above patch applied.

Excellent. I am going through some minor things which may be useful to fix,
but that probably means we'll have a re-release soon :)

Rainer


From rgerhards at hq.adiscon.com  Wed Jan 27 07:28:02 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 07:28:02 +0100
Subject: [rsyslog] Tools to detect stack Adressing Problems?
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710376D@GRFEXC.intern.adiscon.com>

Hi all,

since I have begun to use the valgrind memory debugger routinely in
development (some two years ago), the quality of the source has much
increased. Unfortunately, however, valgrind is not able to detect problems
related to misaddressing variables on the stack. The 5.3.6 bug I was hunting
for almost a week is a good example of this. Valgrind also provides only
limited support for global data, as far as I know (and see from testing
results).

This becomes an even more important restriction as I moved a lot of former
heap memory use to the stack for performance reasons. I remember at least one
more major bug hunting effort that was hard to find because it affected only
stack space.

So I am currently looking for tools that could complement valgrind by
providing good stack checking capabilities. As one tool, mudflap was
suggested to me. It sounds interesting, but gives me a very hard time [very
hard to read debug output (no symbolic names for dlloade'ed modules, (false?)
reports for areas where I can not see anything wrong as well as frequent
(threading-related?) crashes when running under instrumentation). Maybe I am
just misinterpreting the output...

In short: I would highly appreciate suggestions for tools that can help with
debugging stack memory access (global data would be a plus) - and/or
instructions on how to interpret mudflap, if that is considered to be *the*
tool for that use case.

Thanks,
Rainer


From janfrode at tanso.net  Wed Jan 27 13:42:38 2010
From: janfrode at tanso.net (Jan-Frode Myklebust)
Date: Wed, 27 Jan 2010 13:42:38 +0100
Subject: [rsyslog] filtering postfix/smtpd
Message-ID: <20100127124238.GA25239@janfrode.ibm.com>

I'm drowning in logs from postfix/smtpd, and need to filter these
messages out to a separate file. The maillog looks something like:

Jan 27 13:34:02 asav5.example.net postfix/lmtp[31977]::  53843908E2: to=<me at example.com>, relay=127.0.0.1[127.0.0.1]:10020, delay=0.54, delays=0.03/0.33/0.01/0.49, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 249FB906AD)
Jan 27 13:34:02 asav3.example.net postfix/smtpd[12077]::  connect from 26.81-111-54.customer.example.net[21.111.54.26]
Jan 27 13:34:02 asav5.example.net postfix/qmgr[32165]::  53843908E2: removed
Jan 27 13:34:02 asav3.mro.example.net postfix/smtpd[12077]::  disconnect from 26.81-111-54.customer.example.net[21.111.54.26]

So I want to separate out the lines from "postfix/smtpd" to
its own file, and not touch the postfix/lmtp or postfix/qmgr
or whatever-lines.

>From the documentation it seems to me that I should be able
to use:

	:programname,   isequal,        "postfix/smtpd" -?HourlyMaillogNonSplunked;MaillogTemplate
	:programname,   isequal,        "postfix/smtpd"        ~

But these doesn't match anything. If I use simply "postfix",
it matched all "postfix/*" messages:

	:programname,   isequal,        "postfix" -?HourlyMaillogNonSplunked;MaillogTemplate
	:programname,   isequal,        "postfix"        ~

So, any idea for how I can match just "postfix/smtpd" ?


  -jf



From rgerhards at hq.adiscon.com  Wed Jan 27 14:40:37 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 14:40:37 +0100
Subject: [rsyslog] filtering postfix/smtpd
References: <20100127124238.GA25239@janfrode.ibm.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103776@GRFEXC.intern.adiscon.com>

Hi,

could you run it in debug mode and post the relevant part of a log message
being processed? I guess that %programname% gets some weird value...

Rainer 

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Jan-Frode Myklebust
> Sent: Wednesday, January 27, 2010 1:43 PM
> To: rsyslog at lists.adiscon.com
> Subject: [rsyslog] filtering postfix/smtpd
> 
> I'm drowning in logs from postfix/smtpd, and need to filter these
> messages out to a separate file. The maillog looks something like:
> 
> Jan 27 13:34:02 asav5.example.net postfix/lmtp[31977]::  53843908E2:
> to=<me at example.com>, relay=127.0.0.1[127.0.0.1]:10020, delay=0.54,
> delays=0.03/0.33/0.01/0.49, dsn=2.0.0, status=sent (250 2.0.0 Ok:
> queued as 249FB906AD)
> Jan 27 13:34:02 asav3.example.net postfix/smtpd[12077]::  connect from
> 26.81-111-54.customer.example.net[21.111.54.26]
> Jan 27 13:34:02 asav5.example.net postfix/qmgr[32165]::  53843908E2:
> removed
> Jan 27 13:34:02 asav3.mro.example.net postfix/smtpd[12077]::
> disconnect from 26.81-111-54.customer.example.net[21.111.54.26]
> 
> So I want to separate out the lines from "postfix/smtpd" to
> its own file, and not touch the postfix/lmtp or postfix/qmgr
> or whatever-lines.
> 
> >From the documentation it seems to me that I should be able
> to use:
> 
> 	:programname,   isequal,        "postfix/smtpd" -
> ?HourlyMaillogNonSplunked;MaillogTemplate
> 	:programname,   isequal,        "postfix/smtpd"        ~
> 
> But these doesn't match anything. If I use simply "postfix",
> it matched all "postfix/*" messages:
> 
> 	:programname,   isequal,        "postfix" -
> ?HourlyMaillogNonSplunked;MaillogTemplate
> 	:programname,   isequal,        "postfix"        ~
> 
> So, any idea for how I can match just "postfix/smtpd" ?
> 
> 
>   -jf
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 27 15:27:51 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 15:27:51 +0100
Subject: [rsyslog] 8Bit character escaping - was: RE: PostgreSQL: Problems
	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><alpine.DEB.2.00.1001201041520.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103729@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001220205050.3093@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103778@GRFEXC.intern.adiscon.com>

David,

I have now added the functionality to escape 8-bit characters. Patch is here:

http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=85045270f69e4dcb25c409c9
661e96e3172d7f30

I hope it is useful.

I plan to release a new v5 devel soon, probably tomorrow or friday.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 22, 2010 11:07 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] PostgreSQL: Problems with character encoding
> 
> On Fri, 22 Jan 2010, Rainer Gerhards wrote:
> 
> >> my thought is that just like we have a filter to change control
> >> characters
> >> to escape sequences, it would be good to have a filter to escape
> non-
> >> ascii
> >> characters. this will mangle other character sets, but they are
> unlikly
> >> to
> >> go through cleanly anyway.
> >
> > Just to be on the right path, you suggest escaping charactes with hex
> values
> >> 7f?
> 
> correct. they can cause as much grief (or more) than control
> characters.
> 
> since control characters get escaped by default, rsyslog will already
> mangle UTF8 text sent to it if the final byte is in that range.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Wed Jan 27 20:16:36 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Wed, 27 Jan 2010 19:16:36 +0000
Subject: [rsyslog] config file help
Message-ID: <4B609114.9090103@p6m7g8.com>

rsyslog.conf:

...
if $facility == '1' && $priority == '7' then ~
*.* :ommysql:localhost,logs,logs,logs;db_std ## not actual l/p
EOF

select facility,priority, count(1) as c
from syslogs
where facility = 1
and priority = 7
group by facility,priority;
+----------+----------+------+
| facility | priority | c    |
+----------+----------+------+
|        1 |        7 | 1637 |
+----------+----------+------+
1 row in set (0.00 sec)

am I missing something ? I just want to throw it away.



-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From joe at joetify.com  Wed Jan 27 20:16:57 2010
From: joe at joetify.com (Joe Williams)
Date: Wed, 27 Jan 2010 11:16:57 -0800
Subject: [rsyslog] tripling of log lines
Message-ID: <4B609129.6040301@joetify.com>


I have an odd issue where with a specific config I see triple of each 
line in the log but using another config that should effectively be 
doing the same thing it does not.

Doing something like the following produces three identical lines in the 
log.

$template DbFormat,"%timegenerated% %HOSTNAME% %msg:::drop-last-lf%\n"
$template DbNotice,"<%= @log_dir 
%>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
local2.notice      -?DbNotice;DbFormat

Example:

Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0


The following produces the expected one line in the log without duplication.

$template DbFormat,"%timegenerated% %fromhost% %msg:::drop-last-lf%\n"
$template DbNotice,"<%= @log_dir 
%>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"

if \
   ( $syslogfacility-text == 'local2' ) \
     and \
   ( $syslogseverity-text == 'notice' ) \
then -?DbNotice;DbFormat


For brevity in both examples I just showed an example for one severity 
level, we have individual log templates and filters for all of them.

Any ideas what could be going on here? To me these should be equivalent.

Thanks.

-Joe

-- 
Name: Joseph A. Williams
Email: joe at joetify.com
Blog: http://www.joeandmotorboat.com/



From rgerhards at hq.adiscon.com  Wed Jan 27 21:34:48 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 21:34:48 +0100
Subject: [rsyslog] tripling of log lines
References: <4B609129.6040301@joetify.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103783@GRFEXC.intern.adiscon.com>

Maybe you don't discard the message after writing it?

Please see:

http://cookbook.rsyslog.com/node7.html

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Joe Williams
> Sent: Wednesday, January 27, 2010 8:17 PM
> To: rsyslog-users
> Subject: [rsyslog] tripling of log lines
> 
> 
> I have an odd issue where with a specific config I see triple of each
> line in the log but using another config that should effectively be
> doing the same thing it does not.
> 
> Doing something like the following produces three identical lines in
> the
> log.
> 
> $template DbFormat,"%timegenerated% %HOSTNAME% %msg:::drop-last-lf%\n"
> $template DbNotice,"<%= @log_dir
> %>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
> local2.notice      -?DbNotice;DbFormat
> 
> Example:
> 
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> Jan 27 19:01:22 HOSTNAME [<0.28726.158>] IP undefined GET / 200 0
> 
> 
> The following produces the expected one line in the log without
> duplication.
> 
> $template DbFormat,"%timegenerated% %fromhost% %msg:::drop-last-lf%\n"
> $template DbNotice,"<%= @log_dir
> %>/%$YEAR%/%$MONTH%/%$DAY%/db/%programname%/%syslogseverity-text%"
> 
> if \
>    ( $syslogfacility-text == 'local2' ) \
>      and \
>    ( $syslogseverity-text == 'notice' ) \
> then -?DbNotice;DbFormat
> 
> 
> For brevity in both examples I just showed an example for one severity
> level, we have individual log templates and filters for all of them.
> 
> Any ideas what could be going on here? To me these should be
> equivalent.
> 
> Thanks.
> 
> -Joe
> 
> --
> Name: Joseph A. Williams
> Email: joe at joetify.com
> Blog: http://www.joeandmotorboat.com/
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Wed Jan 27 22:26:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 27 Jan 2010 22:26:24 +0100
Subject: [rsyslog] config file help
References: <4B609114.9090103@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Wednesday, January 27, 2010 8:17 PM
> To: rsyslog-users
> Subject: [rsyslog] config file help
> 
> rsyslog.conf:
> 
> ...
> if $facility == '1' && $priority == '7' then ~

I don't have the code at hand right now, but I guess the codes must be
numeric:

if $facility == 1 && $priority == 7 then ~

The scripting engine may not spit out a meaningful error message - it is in
its infancy with no time til today to complete it...

Rainer

> *.* :ommysql:localhost,logs,logs,logs;db_std ## not actual l/p
> EOF
> 
> select facility,priority, count(1) as c
> from syslogs
> where facility = 1
> and priority = 7
> group by facility,priority;
> +----------+----------+------+
> | facility | priority | c    |
> +----------+----------+------+
> |        1 |        7 | 1637 |
> +----------+----------+------+
> 1 row in set (0.00 sec)
> 
> am I missing something ? I just want to throw it away.
> 
> 
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Consultant,                       P6M7G8 Inc.
> Sr. System Admin,                 Ridecharge Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From pgollucci at p6m7g8.com  Wed Jan 27 22:59:00 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Wed, 27 Jan 2010 21:59:00 +0000
Subject: [rsyslog] config file help
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
Message-ID: <4B60B724.8060506@p6m7g8.com>

Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
>> Sent: Wednesday, January 27, 2010 8:17 PM
>> To: rsyslog-users
>> Subject: [rsyslog] config file help
>>
>> rsyslog.conf:
>>
>> ...
>> if $facility == '1' && $priority == '7' then ~
> 
> I don't have the code at hand right now, but I guess the codes must be
> numeric:
> 
> if $facility == 1 && $priority == 7 then ~

Ha, you think I didn't try that too.  No dice either way.

Forget meaningful, it spits out nothing [with debugging and/or ktracing]
Just merely goes along and 'works' too well.


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From pgollucci at p6m7g8.com  Thu Jan 28 02:35:45 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Thu, 28 Jan 2010 01:35:45 +0000
Subject: [rsyslog] config file help
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
Message-ID: <4B60E9F1.6000800@p6m7g8.com>

Rainer Gerhards wrote:
> if $facility == 1 && $priority == 7 then ~
looking up the text values in includes/syslog.h does  work

user.debug ~

but

1.7 ~

does not.

-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From jbondc at openmv.com  Thu Jan 28 03:32:13 2010
From: jbondc at openmv.com (Jonathan Bond-Caron)
Date: Wed, 27 Jan 2010 21:32:13 -0500
Subject: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
	Problems	with character encoding
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
Message-ID: <000f01ca9fc2$1a574840$4f05d8c0$@com>

On Mon Jan 25 03:12 AM, Rainer Gerhards wrote:
> So I don't think it would serve the non-US-ASCII world well to process 
> the transformation formats. I guess that's a good option if you have a 
> US-ASCII based system that only very occasionally needs to process a 
> foreign language string (and even then, you need to parse the message
> *each* time you access it, specifically when obtaining substrings...).
> 
> My conclusion is that rsyslog needs to do a UTF to UCS conversion on 
> entry to the system and then uses UCS internally (and converts back 
> when messages are output). Many software systems do so, and, as I 
> said, IMHO do so for good reasons.
> 

What about adding a property option ~ 'normalize-utf8' where invalid utf8
bytes would be escaped?

$template dbFormat,"insert into text_logs (utf8_message) values
('%msg:::normalize-utf8%')",stdsql

I can probably dig through postgresql to find the code to detect invalid
utf8 bytes.

I'm not sure if I understood but are you suggesting that all input to
rsyslog is converted to UCS internally?
That seems like a huge performance penalty to pay when most people (?) log
US-ascii or UTF-8 data.




From david at lang.hm  Thu Jan 28 06:32:07 2010
From: david at lang.hm (david at lang.hm)
Date: Wed, 27 Jan 2010 21:32:07 -0800 (PST)
Subject: [rsyslog] Unicode & rsyslog - was: RE:
 PostgreSQL:	Problems	with character encoding
In-Reply-To: <000f01ca9fc2$1a574840$4f05d8c0$@com>
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com>
	<alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com>
	<000f01ca9fc2$1a574840$4f05d8c0$@com>
Message-ID: <alpine.DEB.2.00.1001272129130.28980@asgard.lang.hm>

On Wed, 27 Jan 2010, Jonathan Bond-Caron wrote:

> On Mon Jan 25 03:12 AM, Rainer Gerhards wrote:
>> So I don't think it would serve the non-US-ASCII world well to process
>> the transformation formats. I guess that's a good option if you have a
>> US-ASCII based system that only very occasionally needs to process a
>> foreign language string (and even then, you need to parse the message
>> *each* time you access it, specifically when obtaining substrings...).
>>
>> My conclusion is that rsyslog needs to do a UTF to UCS conversion on
>> entry to the system and then uses UCS internally (and converts back
>> when messages are output). Many software systems do so, and, as I
>> said, IMHO do so for good reasons.
>>
>
> What about adding a property option ~ 'normalize-utf8' where invalid utf8
> bytes would be escaped?
>
> $template dbFormat,"insert into text_logs (utf8_message) values
> ('%msg:::normalize-utf8%')",stdsql
>
> I can probably dig through postgresql to find the code to detect invalid
> utf8 bytes.

Rainer just added a property option to escape characters > 127. you could 
probably take that patch and basicly clone it to make a version that only 
escapes things if they aren't valid UTF8 instead.

> I'm not sure if I understood but are you suggesting that all input to
> rsyslog is converted to UCS internally?
> That seems like a huge performance penalty to pay when most people (?) log
> US-ascii or UTF-8 data.

right now rsyslog doesn't do any unicode stuff, it treats everything as a 
string of bytes (with some code to escape specific characters). He is 
saying that the path he has been planning to take would convert everything 
to UCS internally. you saw my argument against that.

David Lang


From rgerhards at hq.adiscon.com  Thu Jan 28 08:52:00 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 08:52:00 +0100
Subject: [rsyslog] config file help
References: <4B609114.9090103@p6m7g8.com><9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60E9F1.6000800@p6m7g8.com>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7103788@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> Sent: Thursday, January 28, 2010 2:36 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] config file help
> 
> Rainer Gerhards wrote:
> > if $facility == 1 && $priority == 7 then ~
> looking up the text values in includes/syslog.h does  work
> 
> user.debug ~
> 
> but
> 
> 1.7 ~

These kind of filters are a different beast (and the traditional ones).
Rsyslog has three types of filters:

- the traditional ones
- property based
- script bases

Functionality increases on the way down, but also performance decreases.
Filters evolved, so each class has the syntax that best fits it. Note that
the if statement above and the traditional filter user.debug is *very*
different when looking from the executed code. User.debug is *much* faster
than starting up the script logic for the same thing. Should have mentioned
that yesterday...

Rainer

> 
> does not.
> 
> --
> -----------------------------------------------------------------------
> -
> 1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
> Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
> VP Apache Infrastructure; Member, Apache Software Foundation
> Committer,                        FreeBSD Foundation
> Consultant,                       P6M7G8 Inc.
> Sr. System Admin,                 Ridecharge Inc.
> 
> Work like you don't need the money,
> love like you'll never get hurt,
> and dance like nobody's watching.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Jan 28 09:04:40 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 09:04:40 +0100
Subject: [rsyslog] Unicode & rsyslog - was: RE:
	PostgreSQL:	Problems	with character encoding
References: <201001061615.00121.marc.schiffbauer@mightycare.de><20100120192031.143c119a@mp-atlantis3.ziti.uni-heidelberg.de><201001210149.48041.marc.schiffbauer@mightycare.de><20100121222626.083c7a49@samsa><9B6E2A8877C38245BFB15CC491A11DA710372B@GRFEXC.intern.adiscon.com><alpine.DEB.2.00.1001221012180.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA7103740@GRFEXC.intern.adiscon.com><000f01ca9fc2$1a574840$4f05d8c0$@com>
	<alpine.DEB.2.00.1001272129130.28980@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710378A@GRFEXC.intern.adiscon.com>

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Thursday, January 28, 2010 6:32 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] Unicode & rsyslog - was: RE: PostgreSQL:
> Problems with character encoding
> 
> On Wed, 27 Jan 2010, Jonathan Bond-Caron wrote:
> > I'm not sure if I understood but are you suggesting that all input to
> > rsyslog is converted to UCS internally?
> > That seems like a huge performance penalty to pay when most people
> (?) log
> > US-ascii or UTF-8 data.
> 
> right now rsyslog doesn't do any unicode stuff, it treats everything as
> a
> string of bytes (with some code to escape specific characters). He is
> saying that the path he has been planning to take would convert
> everything
> to UCS internally. you saw my argument against that.

I didn't yet respond to the original message because David's argument is a
good one and I did not yet have time to think it over. Please note that there
are many subtle issues, especially when combining it with the demands of the
relevant RFCs (and if I implement it, I will definitely take a path that is
standards-compliant). David's argument and proposed solutions sounds good to
me, though I have some long-term concerns (eg. Can we really expect that
Japanese/Chinese systems always use US-ASCII for the core logging information
- I do not truly believe in that...).

However, I simply have no time to implement Unicode right now, so what I most
probably will do is copy over this valuable discussion and arguments into the
design doc, so that I have them ready at hand when I can turn into that
direction.

But in general, I now tend to agree to David's argument and think that it can
probably even speed up the process of a full Unicode implementation.

Rainer


From tbergfeld at hq.adiscon.com  Thu Jan 28 09:11:08 2010
From: tbergfeld at hq.adiscon.com (Tom Bergfeld)
Date: Thu, 28 Jan 2010 09:11:08 +0100
Subject: [rsyslog] rsyslog 5.3.7 (v5-beta) released
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710378B@GRFEXC.intern.adiscon.com>

Hi all,

We have released a new v5-beta, version 5.3.7. Most importantly, it contains
the fixes for the problem with named pipes that Michael Biebl discovered.
There are also some other fixes (see changelog for detail). No new
functionality is included.

Once again, this is scheduled to become the new v5-stable, if no further
issues exist. As such, we would appreciate if you could try out the version
and report back your experience (even if everything works).
See Changelog for more details.

ChangeLog:

http://www.rsyslog.com/Article437.phtml

Download:

http://www.rsyslog.com/Downloads-req-viewdownloaddetails-lid-192.phtml


As always, feedback is appreciated.

Best regards,
Tom Bergfeld
--
Support
=======

Improving rsyslog is costly, but you can help!  We are looking for
organizations that find rsyslog useful and wish to contribute back.  You can
contribute by reporting bugs, improve the software, or donate money or
equipment.

Commercial support contracts for rsyslog are available, and they help finance
continued maintenance.  Adiscon GmbH, a privately held German company, is
currently funding rsyslog development. We are always looking for interesting
development projects. For details on how to help, please see
http://www.rsyslog.com/doc-how2help.html .
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com


From rgerhards at hq.adiscon.com  Thu Jan 28 18:38:55 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 18:38:55 +0100
Subject: [rsyslog] config file help
In-Reply-To: <4B60B724.8060506@p6m7g8.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60B724.8060506@p6m7g8.com>
Message-ID: <1264700335.11821.2.camel@localhost>

On Wed, 2010-01-27 at 22:59 +0100, Philip M. Gollucci wrote:
> Rainer Gerhards wrote:
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Philip M. Gollucci
> >> Sent: Wednesday, January 27, 2010 8:17 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] config file help
> >>
> >> rsyslog.conf:
> >>
> >> ...
> >> if $facility == '1' && $priority == '7' then ~
> >
> > I don't have the code at hand right now, but I guess the codes must
> be
> > numeric:
> >
> > if $facility == 1 && $priority == 7 then ~
> 
> Ha, you think I didn't try that too.  No dice either way.
> 
> Forget meaningful, it spits out nothing [with debugging and/or
> ktracing]
> Just merely goes along and 'works' too well.

I just tried it out. I got the following error message:

===
2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
expression [try http://www.rsyslog.com/e/2051 ]
2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
$priority == 7 then ~"
2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
without actions will be discarded
2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
could not interpret master config file
'/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]
===

Not too precise (as expected), but far from not existent ;)

The syntax error is &&, you need to use "and". Also, the property names
were incorrect. So the correct line would have been:

if $syslogfacility == 1 and $syslogseverity == 7 then ~

While I have verified that this line works, you are far better of
(performance-wise) with the traditional priority filter that you now
use.

Rainer



From pgollucci at p6m7g8.com  Thu Jan 28 18:41:41 2010
From: pgollucci at p6m7g8.com (Philip M. Gollucci)
Date: Thu, 28 Jan 2010 17:41:41 +0000
Subject: [rsyslog] config file help
In-Reply-To: <1264700335.11821.2.camel@localhost>
References: <4B609114.9090103@p6m7g8.com>	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>	<4B60B724.8060506@p6m7g8.com>
	<1264700335.11821.2.camel@localhost>
Message-ID: <4B61CC55.1030106@p6m7g8.com>

> I just tried it out. I got the following error message:
> 
> ===
> 2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
> expression [try http://www.rsyslog.com/e/2051 ]
> 2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
> in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
> $priority == 7 then ~"
> 2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
> without actions will be discarded
> 2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
> could not interpret master config file
> '/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]

well thats useful at least.  I wonder why I don't see it.

> The syntax error is &&, you need to use "and". Also, the property names
> were incorrect. So the correct line would have been:
> 
> if $syslogfacility == 1 and $syslogseverity == 7 then ~
d'oh

> 
> While I have verified that this line works, you are far better of
> (performance-wise) with the traditional priority filter that you now
> use.
Yes!

Thx!


-- 
------------------------------------------------------------------------
1024D/DB9B8C1C B90B FBC3 A3A1 C71A 8E70  3F8C 75B8 8FFB DB9B 8C1C
Philip M. Gollucci (pgollucci at p6m7g8.com) c: 703.336.9354
VP Apache Infrastructure; Member, Apache Software Foundation
Committer,                        FreeBSD Foundation
Consultant,                       P6M7G8 Inc.
Sr. System Admin,                 Ridecharge Inc.

Work like you don't need the money,
love like you'll never get hurt,
and dance like nobody's watching.


From rgerhards at hq.adiscon.com  Thu Jan 28 18:44:24 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 28 Jan 2010 18:44:24 +0100
Subject: [rsyslog] config file help
In-Reply-To: <4B61CC55.1030106@p6m7g8.com>
References: <4B609114.9090103@p6m7g8.com>
	<9B6E2A8877C38245BFB15CC491A11DA7103785@GRFEXC.intern.adiscon.com>
	<4B60B724.8060506@p6m7g8.com> <1264700335.11821.2.camel@localhost>
	<4B61CC55.1030106@p6m7g8.com>
Message-ID: <1264700664.11821.4.camel@localhost>

On Thu, 2010-01-28 at 17:41 +0000, Philip M. Gollucci wrote:
> > I just tried it out. I got the following error message:
> > 
> > ===
> > 2010-01-28T18:33:41.635158+01:00 rgf12 rsyslogd-2051: syntax error in
> > expression [try http://www.rsyslog.com/e/2051 ]
> > 2010-01-28T18:33:41.635413+01:00 rgf12 rsyslogd: the last error occured
> > in /home/rger/proj/rsyslog/rg.conf, line 10:"if $facility == 1 &&
> > $priority == 7 then ~"
> > 2010-01-28T18:33:41.635477+01:00 rgf12 rsyslogd: warning: selector line
> > without actions will be discarded
> > 2010-01-28T18:33:41.636170+01:00 rgf12 rsyslogd-2124: CONFIG ERROR:
> > could not interpret master config file
> > '/home/rger/proj/rsyslog/rg.conf'. [try http://www.rsyslog.com/e/2124 ]
> 
> well thats useful at least.  I wonder why I don't see it.
> 

well, I guess that's the ole question on alternatives to using the
logging system itself to log error messages... The mailing list has a
couple of posts on this, one I thread I think in December or early this
month. I guess you did not capture syslog messages themselves.

Rainer

> > The syntax error is &&, you need to use "and". Also, the property names
> > were incorrect. So the correct line would have been:
> > 
> > if $syslogfacility == 1 and $syslogseverity == 7 then ~
> d'oh
> 
> > 
> > While I have verified that this line works, you are far better of
> > (performance-wise) with the traditional priority filter that you now
> > use.
> Yes!
> 
> Thx!
> 
> 




From david at lang.hm  Fri Jan 29 04:21:00 2010
From: david at lang.hm (david at lang.hm)
Date: Thu, 28 Jan 2010 19:21:00 -0800 (PST)
Subject: [rsyslog] no v5.3.7 announcement?
Message-ID: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>

I see it in git, I even see an announcement on freshmeat, but I didn't see 
an announcement that it was released here ;-)

for those who have missed it, 5.3.7 includes a couple fixes that were 
discussed here over the last couple of weeks.

David Lang


From rgerhards at hq.adiscon.com  Fri Jan 29 14:56:21 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jan 2010 14:56:21 +0100
Subject: [rsyslog] no v5.3.7 announcement?
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>

Mhhh... Tom sent it out yesterday, and I also see it in the archive:

http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html

Maybe we have some mail delivery problems...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 29, 2010 4:21 AM
> To: rsyslog-users
> Subject: [rsyslog] no v5.3.7 announcement?
> 
> I see it in git, I even see an announcement on freshmeat, but I didn't
> see
> an announcement that it was released here ;-)
> 
> for those who have missed it, 5.3.7 includes a couple fixes that were
> discussed here over the last couple of weeks.
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com


From ktm at rice.edu  Fri Jan 29 14:59:17 2010
From: ktm at rice.edu (Kenneth Marshall)
Date: Fri, 29 Jan 2010 07:59:17 -0600
Subject: [rsyslog] no v5.3.7 announcement?
In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
Message-ID: <20100129135917.GT1221@it.is.rice.edu>

I saw it here.

Ken
On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
> 
> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
> 
> Maybe we have some mail delivery problems...
> 
> Rainer
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> > Sent: Friday, January 29, 2010 4:21 AM
> > To: rsyslog-users
> > Subject: [rsyslog] no v5.3.7 announcement?
> > 
> > I see it in git, I even see an announcement on freshmeat, but I didn't
> > see
> > an announcement that it was released here ;-)
> > 
> > for those who have missed it, 5.3.7 includes a couple fixes that were
> > discussed here over the last couple of weeks.
> > 
> > David Lang
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> 


From david at lang.hm  Fri Jan 29 16:15:53 2010
From: david at lang.hm (david at lang.hm)
Date: Fri, 29 Jan 2010 07:15:53 -0800 (PST)
Subject: [rsyslog] no v5.3.7 announcement?
In-Reply-To: <20100129135917.GT1221@it.is.rice.edu>
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm>
	<9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com>
	<20100129135917.GT1221@it.is.rice.edu>
Message-ID: <alpine.DEB.2.00.1001290715390.28980@asgard.lang.hm>

in that case, sorry for the noise.

David Lang

On Fri, 29 Jan 2010, Kenneth Marshall wrote:

> I saw it here.
>
> Ken
> On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
>> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
>>
>> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
>>
>> Maybe we have some mail delivery problems...
>>
>> Rainer
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>> Sent: Friday, January 29, 2010 4:21 AM
>>> To: rsyslog-users
>>> Subject: [rsyslog] no v5.3.7 announcement?
>>>
>>> I see it in git, I even see an announcement on freshmeat, but I didn't
>>> see
>>> an announcement that it was released here ;-)
>>>
>>> for those who have missed it, 5.3.7 includes a couple fixes that were
>>> discussed here over the last couple of weeks.
>>>
>>> David Lang
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>


From rgerhards at hq.adiscon.com  Fri Jan 29 16:50:38 2010
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 29 Jan 2010 16:50:38 +0100
Subject: [rsyslog] no v5.3.7 announcement?
References: <alpine.DEB.2.00.1001281919440.7748@asgard.lang.hm><9B6E2A8877C38245BFB15CC491A11DA710379E@GRFEXC.intern.adiscon.com><20100129135917.GT1221@it.is.rice.edu>
	<alpine.DEB.2.00.1001290715390.28980@asgard.lang.hm>
Message-ID: <9B6E2A8877C38245BFB15CC491A11DA71037A3@GRFEXC.intern.adiscon.com>

no probalem at all - better twice than never. We had some problems with mail
delivery in december, and so I am always alerted if something in that
direction comes up...

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, January 29, 2010 4:16 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] no v5.3.7 announcement?
> 
> in that case, sorry for the noise.
> 
> David Lang
> 
> On Fri, 29 Jan 2010, Kenneth Marshall wrote:
> 
> > I saw it here.
> >
> > Ken
> > On Fri, Jan 29, 2010 at 02:56:21PM +0100, Rainer Gerhards wrote:
> >> Mhhh... Tom sent it out yesterday, and I also see it in the archive:
> >>
> >> http://lists.adiscon.net/pipermail/rsyslog/2010-January/003391.html
> >>
> >> Maybe we have some mail delivery problems...
> >>
> >> Rainer
> >>
> >>> -----Original Message-----
> >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >>> Sent: Friday, January 29, 2010 4:21 AM
> >>> To: rsyslog-users
> >>> Subject: [rsyslog] no v5.3.7 announcement?
> >>>
> >>> I see it in git, I even see an announcement on freshmeat, but I
> didn't
> >>> see
> >>> an announcement that it was released here ;-)
> >>>
> >>> for those who have missed it, 5.3.7 includes a couple fixes that
> were
> >>> discussed here over the last couple of weeks.
> >>>
> >>> David Lang
> >>> _______________________________________________
> >>> rsyslog mailing list
> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >>> http://www.rsyslog.com
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com
> >>
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> >
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com