From bodik at civ.zcu.cz Wed Jan 4 18:09:45 2012 From: bodik at civ.zcu.cz (bodik) Date: Wed, 04 Jan 2012 18:09:45 +0100 Subject: [rsyslog] omgssapi segfault In-Reply-To: <4EE8CDA6.4050400@redhat.com> References: <4EE6843E.7040905@civ.zcu.cz> <4EE78EBA.6050006@redhat.com> <4EE7C388.5060700@civ.zcu.cz> <4EE8CDA6.4050400@redhat.com> Message-ID: <4F0487D9.8060409@civ.zcu.cz> Dne 12/14/11 17:24, Tomas Heinrich napsal(a): > On 12/13/2011 10:28 PM, bodik wrote: >> thanks for reply I tried this one, at it stops client from segfault ... >> is that what you mean to do ? or we shloud discard "case eDestFORW:" >> completely ? > > My guess at the solution is attached. Please let me know if it works for you. hi, i dig a little bit deeper and thgough testing found that suggested patch is not working well while remote server is rejecting connections by RST (server is up, but rsyslog down ..) 17:47:28.878679 IP aaa.bbb.ccc.125.35114 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.878714 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35114: Flags [R.], seq 17:47:28.880490 IP aaa.bbb.ccc.125.35115 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.880524 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35115: Flags [R.], seq 17:47:28.882414 IP aaa.bbb.ccc.125.35116 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.882445 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35116: Flags [R.], seq 17:47:28.884585 IP aaa.bbb.ccc.125.35117 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.884623 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35117: Flags [R.], seq ... very many packets possible from many hosts ... i managed to follow the instruction you left us, about omfwd.c and i have refactored old plugin in favor of new omfwd.c. I'd like to ask is attached solution could be accepted to trunk. it's a little bit dirty because i did not strip all of UDP stuff, but we can work on it either way to support it or remove completely .. best regards bodik -------------- next part -------------- A non-text attachment was scrubbed... Name: omgssapi.c Type: text/x-csrc Size: 20790 bytes Desc: not available URL: From bodik at civ.zcu.cz Wed Jan 4 18:13:42 2012 From: bodik at civ.zcu.cz (bodik) Date: Wed, 04 Jan 2012 18:13:42 +0100 Subject: [rsyslog] omgssapi segfault In-Reply-To: <4F0487D9.8060409@civ.zcu.cz> References: <4EE6843E.7040905@civ.zcu.cz> <4EE78EBA.6050006@redhat.com> <4EE7C388.5060700@civ.zcu.cz> <4EE8CDA6.4050400@redhat.com> <4F0487D9.8060409@civ.zcu.cz> Message-ID: <4F0488C6.1000308@civ.zcu.cz> > I'd like to ask is attached solution could be accepted to trunk. it's a little sory, that was a wrong version .. here comes the right one bodik -------------- next part -------------- A non-text attachment was scrubbed... Name: omgssapi.c Type: text/x-csrc Size: 25806 bytes Desc: not available URL: From kaiwang.chen at gmail.com Thu Jan 5 15:29:40 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Thu, 5 Jan 2012 22:29:40 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: Message-ID: Hello, I check ChangeLog and found 5.8.4 fixed some bug related to race condition as discussed in http://bugzilla.adiscon.com/show_bug.cgi?id=275 with patch protecting modification in the getMSGID(): http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb356cc230685c5b7c90f8590b4 I guess the situation is almost identical to what in my case: corruption is always at the APPNAME field and there is, I believe, a typo in line 1955 from getAPPNAME() effectively ruins the mutex proctection: 1947 /* rgerhards, 2005-11-24 1948 */ 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) 1950 { 1951 uchar *pszRet; 1952 1953 assert(pM != NULL); 1954 if(bLockMutex == LOCK_MUTEX) 1955 MsgUnlock(pM); /* should be MsgLock(pM) */ 1956 prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); 1957 if(pM->pCSAPPNAME == NULL) 1958 pszRet = UCHAR_CONSTANT(""); 1959 else 1960 pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); 1961 if(bLockMutex == LOCK_MUTEX) 1962 MsgUnlock(pM); 1963 return (char*)pszRet; 1964 } Similar cases were spotted in getPROCID(), getStructuredData() and getProgramName(). Hopefully the following patch against 5.8.6 tarball should solve the problem, I will report back later. --- rsyslog-5.8.6/runtime/msg.c.orig 2011-10-21 17:53:02.000000000 +0800 +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 @@ -1609,7 +1609,7 @@ ISOBJ_TYPE_assert(pM, msg); if(bLockMutex == LOCK_MUTEX) - MsgUnlock(pM); + MsgLock(pM); preparePROCID(pM, MUTEX_ALREADY_LOCKED); if(pM->pCSPROCID == NULL) pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ { uchar *pszRet; - MsgUnlock(pM); + MsgLock(pM); if(pM->pCSStrucData == NULL) pszRet = UCHAR_CONSTANT("-"); else @@ -1894,7 +1894,7 @@ uchar *pszRet; if(bLockMutex == LOCK_MUTEX) - MsgUnlock(pM); + MsgLock(pM); prepareProgramName(pM, MUTEX_ALREADY_LOCKED); if(pM->pCSProgName == NULL) pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ assert(pM != NULL); if(bLockMutex == LOCK_MUTEX) - MsgUnlock(pM); + MsgLock(pM); prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); if(pM->pCSAPPNAME == NULL) pszRet = UCHAR_CONSTANT(""); By the way, many crashes led to the convergence, for example //// db1 core.26581 signal 11, Segmentation fault. (gdb) p *pThis $1 = { pBuf = 0x7f7f5c000cd0 "sshd\177\177", pszBuf = 0x7f7f5c000dd0 "sshd", iBufSize = 4, iStrLen = 4 } (gdb) where #0 0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at stringbuf.c:334 #1 0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 '\001') at msg.c:1960 #2 0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, pTpe=0x12ef350, propID=115 's', pPropLen=0x43fb8f00, pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 #3 0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 #4 0x0000000000438498 in prepareBatch (pAction=0x12f72d0, pBatch=0x12f7898, pbShutdownImmediate=) at ../action.c:738 #5 processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, pbShutdownImmediate=) at ../action.c:1163 #6 0x000000000043190d in ConsumerReg (pThis=0x12f73f0, pWti=0x12f7870) at queue.c:1705 #7 0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 #8 0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 #9 0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 #10 0x0000000000000000 in ?? () Thanks, Kaiwang 2011/12/31 Kaiwang Chen : > Even worse case > <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog 5.8.6, > log source = /proc/kmsg started. > ...several SYN flooding entries in the same second... > *** glibc detected *** /sbin/rsyslogd: free(): invalid next size > (fast): 0x00007fafb8000e00 *** > *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: > 0x00007fafb8000e20 *** > > ?And possibly hours later the host was not responsive to ssh > connection requests, and even TTY login attempts failed... > Interestingly the debug log in another file continues a little beyond: > > Debug line with all properties: > FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, > syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', > APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > > Debug line with all properties: > FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, > syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', > APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > > Fortunately the squid service on that host was not badly affected. So > I did graceful failover and reboot the host, everything became OK. > Although there was no direct timestamp associated, I guess the problem > occured soon after rsyslogd started. I also spotted several corrupted > messages on other hosts when rsyslogd started. They are same version: > > # rsyslogd -v > rsyslogd 5.8.6, compiled with: > ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > ? ? ? ?FEATURE_DEBUG (debug build, slow code): No > ? ? ? ?32bit Atomic operations supported: ? ? ?Yes > ? ? ? ?64bit Atomic operations supported: ? ? ?Yes > ? ? ? ?Runtime Instrumentation (slow code): ? ?No > > > Thanks, > Kaiwang > > 2011/12/31 Kaiwang Chen : >> Also spotted corrupted messages from imuxsock: >> >> Debug line with all properties: >> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', PRI: 47, >> syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> ratelimit.numratelimiters=213 ' >> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> ratelimit.numratelimiters=213 ' >> inputname: impstats rawmsg: 'imuxsock: submitted=429 >> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >> >> >> Thanks, >> Kaiwang >> >> 2011/12/27 Kaiwang Chen : >>> 2011/12/25 ?: >>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >>>> >>>>> Hi all, >>>>> >>>>> I found rsyslogd occasionally produced corrupted log entries like >>>>> >>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >>>>> device eth0 entered promiscuous mode >>>>> >>>>> I believe that problem appeared in earlier versions including 5.8.2. >>>>> Looks like corruption never occur before hostname field. Is it a >>>>> reported bug? Any clue? >>>> >>>> >>>> the question is if this problem is in the raw message being sent to you, or >>>> is it something added by the rsyslog processing. >>>> >>>> I would suggest adding something along the following line. >>>> >>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >>>> >>>> this will look for any messages with the hex 01 character in them and spit >>>> out all the info that you can use about the message in to the file >>>> /var/log/badmessages >>>> >>>> run this for a little while and look to see what the raw message that is >>>> being received over the wire looks like. If it's bad, then you need to look >>>> at the sender. If the raw message looks sane, but rsyslog isn't handling it >>>> right, then we can troubleshoot from there. >>> >>> Looks like it's related to tcp transmission. On the original server >>> that get input from unix socket, it is OK; while on the centeral log >>> server, the raw message is already corrupted. >>> >>> ===== on central log server >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>> UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>> packet(s) from UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >>> Connection from UDP: [172.25.0.230]:53547 >>> >>> Debug line with all properties: >>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[2#032#0251]', programname: 'snmpd', APP-NAME: >>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>> UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>> packet(s) from UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>> UDP: [172.25.0.230]:53547 >>> >>> Debug line with all properties: >>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> Connection from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> Connection from UDP: [172.25.0.230]:53547' >>> >>> >>> Thanks, >>> Kaiwang >>>> >>>> David Lang >>>> >>>> ?> rsyslogd 5.8.6, compiled with: >>>>> >>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >>>>> >>>>> /etc/rsyslog.conf >>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>> >>>>> $ModLoad imklog >>>>> $ModLoad imuxsock >>>>> $ModLoad impstats >>>>> >>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >>>>> $SystemLogUsePIDFromSystem on >>>>> >>>>> $PStatInterval 600 >>>>> $PStatSeverity 7 >>>>> >>>>> $WorkDirectory /var/spool/rsyslog >>>>> >>>>> $MainMsgQueueSaveOnShutdown on >>>>> $MainMsgQueueFileName mq >>>>> $MainMsgQueueMaxFileSize 5m >>>>> >>>>> $ActionQueueType LinkedList >>>>> $ActionQueueSaveOnShutdown on >>>>> $ActionQueueFileName dbq >>>>> $ActionQueueMaxFileSize 10m >>>>> $ActionResumeRetryCount -1 >>>>> *.* ? ? ? @@(o)10.2.3.4 >>>>> >>>>> >>>>> # Log all kernel messages to the console. >>>>> # Logging much else clutters up the screen. >>>>> #kern.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /dev/console >>>>> >>>>> # Log anything (except mail) of level info or higher. >>>>> # Don't log private authentication messages! >>>>> *.info;mail.none;authpriv.none;cron.none ? ? ? ? ? ? ? ?/var/log/messages >>>>> >>>>> # The authpriv file has restricted access. >>>>> authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure >>>>> >>>>> # Log all the mail messages in one place. >>>>> mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog >>>>> >>>>> >>>>> # Log cron stuff >>>>> cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron >>>>> >>>>> # Everybody gets emergency messages >>>>> *.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? * >>>>> >>>>> # Save news errors of level crit and higher in a special file. >>>>> uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler >>>>> >>>>> # Save boot messages also to boot.log >>>>> local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot.log >>>>> >>>>> >>>>> Thanks, >>>>> Kaiwang >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ From penght at cn.fujitsu.com Fri Jan 6 07:48:18 2012 From: penght at cn.fujitsu.com (Peng Haitao) Date: Fri, 06 Jan 2012 14:48:18 +0800 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID Message-ID: <4F069932.9070408@cn.fujitsu.com> [add list to the CC list] When HAVE_SETSID is not defined, rsyslogd will use ioctl() make itself to daemon, but this can not make rsyslogd process become the process group leader of a new process group. In RHEL6.1, the status is as follows: # uname -a Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux # /etc/init.d/rsyslog restart Shutting down system logger: [ OK ] Starting system logger: [ OK ] # ps axo pgrp,ppid,pid,comm | grep rsyslog 6290 1 6301 rsyslogd When we send SIGTERM signal to 6290, rsyslogd will die:( So I think we should call setpgid() before ioctl(). Signed-off-by: Peng Haitao --- tools/syslogd.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/tools/syslogd.c b/tools/syslogd.c index d1224f2..ac954b7 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -356,8 +356,15 @@ void untty(void) #else { int i; + pid_t pid; if(!Debug) { + pid = getpid(); + if (setpgid(pid, pid) < 0) { + perror("setpgid"); + exit(1); + } + i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); if (i >= 0) { # if !defined(__hpux) -- 1.7.1 -- Best Regards, Peng Haitao From kaiwang.chen at gmail.com Mon Jan 9 09:29:05 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Mon, 9 Jan 2012 16:29:05 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: Message-ID: To report that coredump and message corruption disappeared after the application of my patch; I guess it's fixed. Thanks, Kaiwang 2012/1/5 Kaiwang Chen : > Hello, > > I check ChangeLog and found 5.8.4 fixed some bug related to race > condition as discussed in > http://bugzilla.adiscon.com/show_bug.cgi?id=275 > > with patch protecting modification in the getMSGID(): > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb356cc230685c5b7c90f8590b4 > > > I guess the situation is almost identical to what in my case: > corruption is always at the APPNAME field and there is, I believe, a > typo in line 1955 from getAPPNAME() effectively ruins the mutex > proctection: > 1947 /* rgerhards, 2005-11-24 > 1948 ?*/ > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) > 1950 { > 1951 ? ? ? ? uchar *pszRet; > 1952 > 1953 ? ? ? ? assert(pM != NULL); > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); > 1959 ? ? ? ? else > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); > 1963 ? ? ? ? return (char*)pszRet; > 1964 } > > Similar cases were spotted in getPROCID(), getStructuredData() and > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball > should solve the problem, I will report back later. > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 +0800 > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 > @@ -1609,7 +1609,7 @@ > > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > - ? ? ? ? ? ? ? MsgUnlock(pM); > + ? ? ? ? ? ? ? MsgLock(pM); > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); > ? ? ? ?if(pM->pCSPROCID == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > @@ -1846,7 +1846,7 @@ > ?{ > ? ? ? ?uchar *pszRet; > > - ? ? ? MsgUnlock(pM); > + ? ? ? MsgLock(pM); > ? ? ? ?if(pM->pCSStrucData == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); > ? ? ? ?else > @@ -1894,7 +1894,7 @@ > ? ? ? ?uchar *pszRet; > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > - ? ? ? ? ? ? ? MsgUnlock(pM); > + ? ? ? ? ? ? ? MsgLock(pM); > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); > ? ? ? ?if(pM->pCSProgName == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > @@ -1952,7 +1952,7 @@ > > ? ? ? ?assert(pM != NULL); > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > - ? ? ? ? ? ? ? MsgUnlock(pM); > + ? ? ? ? ? ? ? MsgLock(pM); > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > ? ? ? ?if(pM->pCSAPPNAME == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > > > By the way, many crashes led to the convergence, for example > //// db1 core.26581 signal 11, Segmentation fault. > (gdb) p *pThis > $1 = { > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", > ?pszBuf = 0x7f7f5c000dd0 "sshd", > ?iBufSize = 4, > ?iStrLen = 4 > } > (gdb) where > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at stringbuf.c:334 > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 > '\001') at msg.c:1960 > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, pTpe=0x12ef350, > propID=115 's', pPropLen=0x43fb8f00, > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, > pBatch=0x12f7898, pbShutdownImmediate=) > ? ?at ../action.c:738 > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, > pbShutdownImmediate=) at ../action.c:1163 > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, > pWti=0x12f7870) at queue.c:1705 > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 > #10 0x0000000000000000 in ?? () > > > > Thanks, > Kaiwang > > 2011/12/31 Kaiwang Chen : >> Even worse case >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog 5.8.6, >> log source = /proc/kmsg started. >> ...several SYN flooding entries in the same second... >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size >> (fast): 0x00007fafb8000e00 *** >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: >> 0x00007fafb8000e20 *** >> >> ?And possibly hours later the host was not responsive to ssh >> connection requests, and even TTY login attempts failed... >> Interestingly the debug log in another file continues a little beyond: >> >> Debug line with all properties: >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, >> syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> >> Debug line with all properties: >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, >> syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> >> Fortunately the squid service on that host was not badly affected. So >> I did graceful failover and reboot the host, everything became OK. >> Although there was no direct timestamp associated, I guess the problem >> occured soon after rsyslogd started. I also spotted several corrupted >> messages on other hosts when rsyslogd started. They are same version: >> >> # rsyslogd -v >> rsyslogd 5.8.6, compiled with: >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> Thanks, >> Kaiwang >> >> 2011/12/31 Kaiwang Chen : >>> Also spotted corrupted messages from imuxsock: >>> >>> Debug line with all properties: >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', PRI: 47, >>> syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> ratelimit.numratelimiters=213 ' >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> ratelimit.numratelimiters=213 ' >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >>> >>> >>> Thanks, >>> Kaiwang >>> >>> 2011/12/27 Kaiwang Chen : >>>> 2011/12/25 ?: >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I found rsyslogd occasionally produced corrupted log entries like >>>>>> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >>>>>> device eth0 entered promiscuous mode >>>>>> >>>>>> I believe that problem appeared in earlier versions including 5.8.2. >>>>>> Looks like corruption never occur before hostname field. Is it a >>>>>> reported bug? Any clue? >>>>> >>>>> >>>>> the question is if this problem is in the raw message being sent to you, or >>>>> is it something added by the rsyslog processing. >>>>> >>>>> I would suggest adding something along the following line. >>>>> >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >>>>> >>>>> this will look for any messages with the hex 01 character in them and spit >>>>> out all the info that you can use about the message in to the file >>>>> /var/log/badmessages >>>>> >>>>> run this for a little while and look to see what the raw message that is >>>>> being received over the wire looks like. If it's bad, then you need to look >>>>> at the sender. If the raw message looks sane, but rsyslog isn't handling it >>>>> right, then we can troubleshoot from there. >>>> >>>> Looks like it's related to tcp transmission. On the original server >>>> that get input from unix socket, it is OK; while on the centeral log >>>> server, the raw message is already corrupted. >>>> >>>> ===== on central log server >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>>> UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>>> packet(s) from UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >>>> Connection from UDP: [172.25.0.230]:53547 >>>> >>>> Debug line with all properties: >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[2#032#0251]', programname: 'snmpd', APP-NAME: >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >>>> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>>> UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>>> packet(s) from UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>>> UDP: [172.25.0.230]:53547 >>>> >>>> Debug line with all properties: >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>>> Connection from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>>> Connection from UDP: [172.25.0.230]:53547' >>>> >>>> >>>> Thanks, >>>> Kaiwang >>>>> >>>>> David Lang >>>>> >>>>> ?> rsyslogd 5.8.6, compiled with: >>>>>> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >>>>>> >>>>>> /etc/rsyslog.conf >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>>> >>>>>> $ModLoad imklog >>>>>> $ModLoad imuxsock >>>>>> $ModLoad impstats >>>>>> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >>>>>> $SystemLogUsePIDFromSystem on >>>>>> >>>>>> $PStatInterval 600 >>>>>> $PStatSeverity 7 >>>>>> >>>>>> $WorkDirectory /var/spool/rsyslog >>>>>> >>>>>> $MainMsgQueueSaveOnShutdown on >>>>>> $MainMsgQueueFileName mq >>>>>> $MainMsgQueueMaxFileSize 5m >>>>>> >>>>>> $ActionQueueType LinkedList >>>>>> $ActionQueueSaveOnShutdown on >>>>>> $ActionQueueFileName dbq >>>>>> $ActionQueueMaxFileSize 10m >>>>>> $ActionResumeRetryCount -1 >>>>>> *.* ? ? ? @@(o)10.2.3.4 >>>>>> >>>>>> >>>>>> # Log all kernel messages to the console. >>>>>> # Logging much else clutters up the screen. >>>>>> #kern.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /dev/console >>>>>> >>>>>> # Log anything (except mail) of level info or higher. >>>>>> # Don't log private authentication messages! >>>>>> *.info;mail.none;authpriv.none;cron.none ? ? ? ? ? ? ? ?/var/log/messages >>>>>> >>>>>> # The authpriv file has restricted access. >>>>>> authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure >>>>>> >>>>>> # Log all the mail messages in one place. >>>>>> mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog >>>>>> >>>>>> >>>>>> # Log cron stuff >>>>>> cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron >>>>>> >>>>>> # Everybody gets emergency messages >>>>>> *.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? * >>>>>> >>>>>> # Save news errors of level crit and higher in a special file. >>>>>> uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler >>>>>> >>>>>> # Save boot messages also to boot.log >>>>>> local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot.log >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Kaiwang >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ From francois at fmarier.org Mon Jan 9 10:35:19 2012 From: francois at fmarier.org (Francois Marier) Date: Mon, 9 Jan 2012 22:35:19 +1300 Subject: [rsyslog] Use of the "discard" action Message-ID: <20120109093519.GE6656@isafjordur.dyndns.org> Hi, I must be missing something because I'm trying to have messages received (via UDP) from 192.168.1.1: 1- go to a separate file, and 2- not show up in /var/log/syslog by loosely following the instructions in the cookbook [1] and the config file documentation [2]: $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log & ~ Unfortunately, only #1 works. I've got the log entries from 192.168.1.1 both in gargoyle-router.log and in /var/log/syslog just like when I use this config. I've also tried to get #2 working by itself (not logging to the gargoyle-router.log file and not logging in /var/log/syslog) using this config: $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then ~ but it also didn't work. I suspect that there is something I don't understand about the discard command because I can't figure out why it's not discarding the message. Any ideas on how I should go about debugging this further? (I'm running rsyslog 4.2.0-2ubuntu8.1 on Ubuntu Lucid and I have attached my full concatenated rsyslog.conf to this email.) Cheers, Francois [1] http://rsyslog.com/writing-specific-messages-to-a-file-and-discarding-them/ [2] http://rsyslog.com/doc/rsyslog_conf_actions.html -- Francois Marier identi.ca/fmarier http://fmarier.org twitter.com/fmarier -------------- next part -------------- # /etc/rsyslog.conf Configuration file for rsyslog. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html # # Default logging rules can be found in /etc/rsyslog.d/50-default.conf ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability $KLogPath /proc/kmsg # provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log & ~ # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use traditional timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # Filter duplicated messages $RepeatedMsgReduction on # # Set the default permissions for all log files. # $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 $PrivDropToUser syslog $PrivDropToGroup syslog # Default rules for rsyslog. # # For more information see rsyslog.conf(5) and /etc/rsyslog.conf # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # daemon.*;mail.*;\ news.err;\ *.=debug;*.=info;\ *.=notice;*.=warn |/dev/xconsole # Create an additional socket in postfix's chroot in order not to break # mail logging when rsyslog is restarted. If the directory is missing, # rsyslog will silently skip creating the socket. $AddUnixListenSocket /var/spool/postfix/dev/log From bodik at civ.zcu.cz Mon Jan 9 10:37:58 2012 From: bodik at civ.zcu.cz (bodik) Date: Mon, 09 Jan 2012 10:37:58 +0100 Subject: [rsyslog] Use of the "discard" action In-Reply-To: <20120109093519.GE6656@isafjordur.dyndns.org> References: <20120109093519.GE6656@isafjordur.dyndns.org> Message-ID: <4F0AB576.5030605@civ.zcu.cz> hi, lately i ends up with this configuration, remote configuration must come first ... http://home.zcu.cz/~bodik/rsyslog/rsyslog.srvr.conf b Dne 01/09/12 10:35, Francois Marier napsal(a): > Hi, > > I must be missing something because I'm trying to have messages received > (via UDP) from 192.168.1.1: > From rgerhards at hq.adiscon.com Mon Jan 9 14:34:57 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 9 Jan 2012 14:34:57 +0100 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Thanks for reporting and tracking down this bug. I was on vacation and have just returned. I applied your patch, it is now merged: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e338a6e5 fa14c0394bde6f1d I'd appreciate if you could double-check that the git version is correct (you never know...). I will release this version during the next days. Once again for your help! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen > Sent: Monday, January 09, 2012 9:29 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption > > To report that coredump and message corruption disappeared after the > application of my patch; I guess it's fixed. > > Thanks, > Kaiwang > > 2012/1/5 Kaiwang Chen : > > Hello, > > > > I check ChangeLog and found 5.8.4 fixed some bug related to race > > condition as discussed in > > http://bugzilla.adiscon.com/show_bug.cgi?id=275 > > > > with patch protecting modification in the getMSGID(): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 > > 6cc230685c5b7c90f8590b4 > > > > > > I guess the situation is almost identical to what in my case: > > corruption is always at the APPNAME field and there is, I believe, a > > typo in line 1955 from getAPPNAME() effectively ruins the mutex > > proctection: > > 1947 /* rgerhards, 2005-11-24 > > 1948 ?*/ > > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) > > 1950 { > > 1951 ? ? ? ? uchar *pszRet; > > 1952 > > 1953 ? ? ? ? assert(pM != NULL); > > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ > > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) > > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); > > 1959 ? ? ? ? else > > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); > > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); > > 1963 ? ? ? ? return (char*)pszRet; > > 1964 } > > > > Similar cases were spotted in getPROCID(), getStructuredData() and > > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball > > should solve the problem, I will report back later. > > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 > > +0800 > > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 > > @@ -1609,7 +1609,7 @@ > > > > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > > - ? ? ? ? ? ? ? MsgUnlock(pM); > > + ? ? ? ? ? ? ? MsgLock(pM); > > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); > > ? ? ? ?if(pM->pCSPROCID == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ > > ?{ > > ? ? ? ?uchar *pszRet; > > > > - ? ? ? MsgUnlock(pM); > > + ? ? ? MsgLock(pM); > > ? ? ? ?if(pM->pCSStrucData == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); > > ? ? ? ?else > > @@ -1894,7 +1894,7 @@ > > ? ? ? ?uchar *pszRet; > > > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > > - ? ? ? ? ? ? ? MsgUnlock(pM); > > + ? ? ? ? ? ? ? MsgLock(pM); > > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); > > ? ? ? ?if(pM->pCSProgName == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ > > > > ? ? ? ?assert(pM != NULL); > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > > - ? ? ? ? ? ? ? MsgUnlock(pM); > > + ? ? ? ? ? ? ? MsgLock(pM); > > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > > ? ? ? ?if(pM->pCSAPPNAME == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > > > > > > By the way, many crashes led to the convergence, for example //// db1 > > core.26581 signal 11, Segmentation fault. > > (gdb) p *pThis > > $1 = { > > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", > > ?pszBuf = 0x7f7f5c000dd0 "sshd", > > ?iBufSize = 4, > > ?iStrLen = 4 > > } > > (gdb) where > > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at > > stringbuf.c:334 > > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 > > '\001') at msg.c:1960 > > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, > pTpe=0x12ef350, > > propID=115 's', pPropLen=0x43fb8f00, > > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 > > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, > > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 > > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, > > pBatch=0x12f7898, pbShutdownImmediate=) > > ? ?at ../action.c:738 > > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, > > pbShutdownImmediate=) at ../action.c:1163 > > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, > > pWti=0x12f7870) at queue.c:1705 > > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 > > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 > > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 > > #10 0x0000000000000000 in ?? () > > > > > > > > Thanks, > > Kaiwang > > > > 2011/12/31 Kaiwang Chen : > >> Even worse case > >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog > >> 5.8.6, log source = /proc/kmsg started. > >> ...several SYN flooding entries in the same second... > >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size > >> (fast): 0x00007fafb8000e00 *** > >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: > >> 0x00007fafb8000e20 *** > >> > >> ?And possibly hours later the host was not responsive to ssh > >> connection requests, and even TTY login attempts failed... > >> Interestingly the debug log in another file continues a little beyond: > >> > >> Debug line with all properties: > >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', > >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >> 'rsyslogd-pstats', > >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 > maxqsize=0 ' > >> > >> Debug line with all properties: > >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', > >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >> 'rsyslogd-pstats', > >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 > maxqsize=5 ' > >> > >> Fortunately the squid service on that host was not badly affected. So > >> I did graceful failover and reboot the host, everything became OK. > >> Although there was no direct timestamp associated, I guess the > >> problem occured soon after rsyslogd started. I also spotted several > >> corrupted messages on other hosts when rsyslogd started. They are same > version: > >> > >> # rsyslogd -v > >> rsyslogd 5.8.6, compiled with: > >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No > >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes > >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes > >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No > >> > >> > >> Thanks, > >> Kaiwang > >> > >> 2011/12/31 Kaiwang Chen : > >>> Also spotted corrupted messages from imuxsock: > >>> > >>> Debug line with all properties: > >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', > >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 > >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', > >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', > >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> ratelimit.numratelimiters=213 ' > >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> ratelimit.numratelimiters=213 ' > >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 > >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' > >>> > >>> > >>> Thanks, > >>> Kaiwang > >>> > >>> 2011/12/27 Kaiwang Chen : > >>>> 2011/12/25 ?: > >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> I found rsyslogd occasionally produced corrupted log entries like > >>>>>> > >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - > >>>>>> device eth0 entered promiscuous mode > >>>>>> > >>>>>> I believe that problem appeared in earlier versions including 5.8.2. > >>>>>> Looks like corruption never occur before hostname field. Is it a > >>>>>> reported bug? Any clue? > >>>>> > >>>>> > >>>>> the question is if this problem is in the raw message being sent > >>>>> to you, or is it something added by the rsyslog processing. > >>>>> > >>>>> I would suggest adding something along the following line. > >>>>> > >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG > >>>>> > >>>>> this will look for any messages with the hex 01 character in them > >>>>> and spit out all the info that you can use about the message in to > >>>>> the file /var/log/badmessages > >>>>> > >>>>> run this for a little while and look to see what the raw message > >>>>> that is being received over the wire looks like. If it's bad, then > >>>>> you need to look at the sender. If the raw message looks sane, but > >>>>> rsyslog isn't handling it right, then we can troubleshoot from there. > >>>> > >>>> Looks like it's related to tcp transmission. On the original server > >>>> that get input from unix socket, it is OK; while on the centeral > >>>> log server, the raw message is already corrupted. > >>>> > >>>> ===== on central log server > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection > >>>> from > >>>> UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > SNMP > >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - > >>>> Connection from UDP: [172.25.0.230]:53547 > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', > >>>> APP-NAME: 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 > >>>> snmpd > >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', > >>>> APP-NAME: 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Received SNMP packet(s) from UDP: > [172.25.0.230]:53547' > >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 > >>>> snmpd > >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', programname: > 'snmpd', APP-NAME: > >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 > >>>> snmpd > >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection > >>>> from > >>>> UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > SNMP > >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection > >>>> from > >>>> UDP: [172.25.0.230]:53547 > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: > >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>>> 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>>> Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: > >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>>> 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Received SNMP packet(s) from UDP: > [172.25.0.230]:53547' > >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: > >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>>> 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>>> Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> > >>>> Thanks, > >>>> Kaiwang > >>>>> > >>>>> David Lang > >>>>> > >>>>> ?> rsyslogd 5.8.6, compiled with: > >>>>>> > >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No > >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes > >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes > >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No > >>>>>> > >>>>>> /etc/rsyslog.conf > >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>>>>> > >>>>>> $ModLoad imklog > >>>>>> $ModLoad imuxsock > >>>>>> $ModLoad impstats > >>>>>> > >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off > >>>>>> $SystemLogUsePIDFromSystem on > >>>>>> > >>>>>> $PStatInterval 600 > >>>>>> $PStatSeverity 7 > >>>>>> > >>>>>> $WorkDirectory /var/spool/rsyslog > >>>>>> > >>>>>> $MainMsgQueueSaveOnShutdown on > >>>>>> $MainMsgQueueFileName mq > >>>>>> $MainMsgQueueMaxFileSize 5m > >>>>>> > >>>>>> $ActionQueueType LinkedList > >>>>>> $ActionQueueSaveOnShutdown on > >>>>>> $ActionQueueFileName dbq > >>>>>> $ActionQueueMaxFileSize 10m > >>>>>> $ActionResumeRetryCount -1 > >>>>>> *.* ? ? ? @@(o)10.2.3.4 > >>>>>> > >>>>>> > >>>>>> # Log all kernel messages to the console. > >>>>>> # Logging much else clutters up the screen. > >>>>>> #kern.* > >>>>>> /dev/console > >>>>>> > >>>>>> # Log anything (except mail) of level info or higher. > >>>>>> # Don't log private authentication messages! > >>>>>> *.info;mail.none;authpriv.none;cron.none > >>>>>> /var/log/messages > >>>>>> > >>>>>> # The authpriv file has restricted access. > >>>>>> authpriv.* > >>>>>> /var/log/secure > >>>>>> > >>>>>> # Log all the mail messages in one place. > >>>>>> mail.* > >>>>>> -/var/log/maillog > >>>>>> > >>>>>> > >>>>>> # Log cron stuff > >>>>>> cron.* > >>>>>> /var/log/cron > >>>>>> > >>>>>> # Everybody gets emergency messages > *.emerg > >>>>>> * > >>>>>> > >>>>>> # Save news errors of level crit and higher in a special file. > >>>>>> uucp,news.crit > >>>>>> /var/log/spooler > >>>>>> > >>>>>> # Save boot messages also to boot.log > >>>>>> local7.* > >>>>>> /var/log/boot.log > >>>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> Kaiwang > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From friedl at hq.adiscon.com Mon Jan 9 16:39:04 2012 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 9 Jan 2012 16:39:04 +0100 Subject: [rsyslog] rsyslog 6.2.0 (stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72816DC@GRFEXC.intern.adiscon.com> This is the initial stable release of rsyslog v6. It is basically the last beta version plus some more fixes. This version provides functional and performance enhancements, for example the Hadoop (HDFS) support has been considerably speeded up by supporting batched insert mode. Also, TCP transmission overhead for TLS has been dramatically improved. TCP now also supports input worker thread pools. Most importantly, rsyslog now supports log normalization via liblognorm rule bases. This permits very high performance normalization of semantically equal messages from different devices (and thus in different syntaxes). Note that config scoping, available in the beta versions, is NOT supported by v6-stable. This was decided because it would have been functionality equivalent to the new config language upcoming in v6.3 (already available as part of the devel version). As scoping was not available in any earlier versions, introducing it in v6.2 would have added, in the long term, just another method of doing some identicaly thing via the ugly old config language. This would have lead to user confusion and more complex than necessary code. If you are interested in the cleaner config language, we strongly encourage you to have a look at rsyslog 6.3. With the arrival of the stable v6 version, version 4 will be retired and is no longer officially supported (but support is provided under maintenance contracts, of course). ChangeLog: http://www.rsyslog.com/changelog-for-6-2-0-v6-stable/ Download: http://www.rsyslog.com/rsyslog-6-2-0-v6-stable/ As always, feedback is appreciated. Best regards, Florian Riedl From david at lang.hm Tue Jan 10 03:03:29 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 9 Jan 2012 18:03:29 -0800 (PST) Subject: [rsyslog] Use of the "discard" action In-Reply-To: <20120109093519.GE6656@isafjordur.dyndns.org> References: <20120109093519.GE6656@isafjordur.dyndns.org> Message-ID: On Mon, 9 Jan 2012, Francois Marier wrote: > Hi, > > I must be missing something because I'm trying to have messages received > (via UDP) from 192.168.1.1: > > 1- go to a separate file, and > 2- not show up in /var/log/syslog > > by loosely following the instructions in the cookbook [1] and the config > file documentation [2]: > > $ModLoad imudp > $UDPServerRun 514 > if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log > & ~ > > Unfortunately, only #1 works. I've got the log entries from 192.168.1.1 both > in gargoyle-router.log and in /var/log/syslog just like when I use this > config. unfortunantly, this config snippet is not sufficient for us to understand what's happening because it doesn't show anything writing to /var/log/syslog. the output to /var/log/syslog would need to be after the discard step (otherwise you are writing to /var/log/syslog before you even run the if statement) also note that doing a full "if then" is slower than doing the property-based conditional :fromhost-ip, startswith, "192.168.1.1" /var/log/gargoyle-router.log and finally, this will match 192.168.1.100 as well as 192.168.1.1 so you probably want equals instead of startswith. David Lang > I've also tried to get #2 working by itself (not logging to the > gargoyle-router.log file and not logging in /var/log/syslog) using this > config: > > $ModLoad imudp > $UDPServerRun 514 > if $fromhost-ip startswith '192.168.1.1' then ~ > > but it also didn't work. > > I suspect that there is something I don't understand about the discard > command because I can't figure out why it's not discarding the message. Any > ideas on how I should go about debugging this further? > > (I'm running rsyslog 4.2.0-2ubuntu8.1 on Ubuntu Lucid and I have attached my > full concatenated rsyslog.conf to this email.) > > Cheers, > Francois > > [1] http://rsyslog.com/writing-specific-messages-to-a-file-and-discarding-them/ > [2] http://rsyslog.com/doc/rsyslog_conf_actions.html > > -------------- next part -------------- # /etc/rsyslog.conf Configuration file for rsyslog. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html # # Default logging rules can be found in /etc/rsyslog.d/50-default.conf ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability $KLogPath /proc/kmsg # provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log & ~ # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use traditional timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # Filter duplicated messages $RepeatedMsgReduction on # # Set the default permissions for all log files. # $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 $PrivDropToUser syslog $PrivDropToGroup syslog # Default rules for rsyslog. # # For more information see rsyslog.conf(5) and /etc/rsyslog.conf # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # daemon.*;mail.*;\ news.err;\ *.=debug;*.=info;\ *.=notice;*.=warn |/dev/xconsole # Create an additional socket in postfix's chroot in order not to break # mail logging when rsyslog is restarted. If the directory is missing, # rsyslog will silently skip creating the socket. $AddUnixListenSocket /var/spool/postfix/dev/log -------------- next part -------------- _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ From kaiwang.chen at gmail.com Tue Jan 10 10:29:14 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Tue, 10 Jan 2012 17:29:14 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Message-ID: Applied to four hosts where coredump had been spotted; rpm was built with the following instructions: git clone git://git.adiscon.com/git/rsyslog.git cd rsyslog git checkout v5-stable ./autogen.sh cd .. mv rsyslog rsyslog-5.8.7 tar czf rsyslog-5.8.7-beta.tar.gz rsyslog-5.8.7 --exclude '.git*' rpmbuild -bs rsyslog.spec mock /usr/src/redhat/SRPMS/rsyslog-5.8.7-beta.el5.kc.src.rpm I will report back in two days. Thanks, Kaiwang 2012/1/9 Rainer Gerhards : > Thanks for reporting and tracking down this bug. I was on vacation and have > just returned. I applied your patch, it is now merged: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e338a6e5 > fa14c0394bde6f1d > > I'd appreciate if you could double-check that the git version is correct (you > never know...). I will release this version during the next days. > > Once again for your help! > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen >> Sent: Monday, January 09, 2012 9:29 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption >> >> To report that coredump and message corruption disappeared after the >> application of my patch; I guess it's fixed. >> >> Thanks, >> Kaiwang >> >> 2012/1/5 Kaiwang Chen : >> > Hello, >> > >> > I check ChangeLog and found 5.8.4 fixed some bug related to race >> > condition as discussed in >> > http://bugzilla.adiscon.com/show_bug.cgi?id=275 >> > >> > with patch protecting modification in the getMSGID(): >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 >> > 6cc230685c5b7c90f8590b4 >> > >> > >> > I guess the situation is almost identical to what in my case: >> > corruption is always at the APPNAME field and there is, I believe, a >> > typo in line 1955 from getAPPNAME() effectively ruins the mutex >> > proctection: >> > 1947 /* rgerhards, 2005-11-24 >> > 1948 ?*/ >> > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) >> > 1950 { >> > 1951 ? ? ? ? uchar *pszRet; >> > 1952 >> > 1953 ? ? ? ? assert(pM != NULL); >> > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >> > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ >> > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >> > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) >> > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); >> > 1959 ? ? ? ? else >> > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); >> > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >> > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); >> > 1963 ? ? ? ? return (char*)pszRet; >> > 1964 } >> > >> > Similar cases were spotted in getPROCID(), getStructuredData() and >> > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball >> > should solve the problem, I will report back later. >> > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 >> > +0800 >> > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 >> > @@ -1609,7 +1609,7 @@ >> > >> > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); >> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >> > - ? ? ? ? ? ? ? MsgUnlock(pM); >> > + ? ? ? ? ? ? ? MsgLock(pM); >> > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); >> > ? ? ? ?if(pM->pCSPROCID == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ >> > ?{ >> > ? ? ? ?uchar *pszRet; >> > >> > - ? ? ? MsgUnlock(pM); >> > + ? ? ? MsgLock(pM); >> > ? ? ? ?if(pM->pCSStrucData == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); >> > ? ? ? ?else >> > @@ -1894,7 +1894,7 @@ >> > ? ? ? ?uchar *pszRet; >> > >> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >> > - ? ? ? ? ? ? ? MsgUnlock(pM); >> > + ? ? ? ? ? ? ? MsgLock(pM); >> > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); >> > ? ? ? ?if(pM->pCSProgName == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ >> > >> > ? ? ? ?assert(pM != NULL); >> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >> > - ? ? ? ? ? ? ? MsgUnlock(pM); >> > + ? ? ? ? ? ? ? MsgLock(pM); >> > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >> > ? ? ? ?if(pM->pCSAPPNAME == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); >> > >> > >> > By the way, many crashes led to the convergence, for example //// db1 >> > core.26581 signal 11, Segmentation fault. >> > (gdb) p *pThis >> > $1 = { >> > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", >> > ?pszBuf = 0x7f7f5c000dd0 "sshd", >> > ?iBufSize = 4, >> > ?iStrLen = 4 >> > } >> > (gdb) where >> > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at >> > stringbuf.c:334 >> > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 >> > '\001') at msg.c:1960 >> > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, >> pTpe=0x12ef350, >> > propID=115 's', pPropLen=0x43fb8f00, >> > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 >> > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, >> > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 >> > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, >> > pBatch=0x12f7898, pbShutdownImmediate=) >> > ? ?at ../action.c:738 >> > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, >> > pbShutdownImmediate=) at ../action.c:1163 >> > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, >> > pWti=0x12f7870) at queue.c:1705 >> > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 >> > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 >> > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 >> > #10 0x0000000000000000 in ?? () >> > >> > >> > >> > Thanks, >> > Kaiwang >> > >> > 2011/12/31 Kaiwang Chen : >> >> Even worse case >> >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog >> >> 5.8.6, log source = /proc/kmsg started. >> >> ...several SYN flooding entries in the same second... >> >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size >> >> (fast): 0x00007fafb8000e00 *** >> >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: >> >> 0x00007fafb8000e20 *** >> >> >> >> ?And possibly hours later the host was not responsive to ssh >> >> connection requests, and even TTY login attempts failed... >> >> Interestingly the debug log in another file continues a little beyond: >> >> >> >> Debug line with all properties: >> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >> >> 'rsyslogd-pstats', >> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 >> maxqsize=0 ' >> >> >> >> Debug line with all properties: >> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >> >> 'rsyslogd-pstats', >> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 >> maxqsize=5 ' >> >> >> >> Fortunately the squid service on that host was not badly affected. So >> >> I did graceful failover and reboot the host, everything became OK. >> >> Although there was no direct timestamp associated, I guess the >> >> problem occured soon after rsyslogd started. I also spotted several >> >> corrupted messages on other hosts when rsyslogd started. They are same >> version: >> >> >> >> # rsyslogd -v >> >> rsyslogd 5.8.6, compiled with: >> >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >> >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No >> >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes >> >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes >> >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> >> >> >> Thanks, >> >> Kaiwang >> >> >> >> 2011/12/31 Kaiwang Chen : >> >>> Also spotted corrupted messages from imuxsock: >> >>> >> >>> Debug line with all properties: >> >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', >> >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >> >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >> >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >> >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> >>> ratelimit.numratelimiters=213 ' >> >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> >>> ratelimit.numratelimiters=213 ' >> >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 >> >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >> >>> >> >>> >> >>> Thanks, >> >>> Kaiwang >> >>> >> >>> 2011/12/27 Kaiwang Chen : >> >>>> 2011/12/25 ?: >> >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >> >>>>> >> >>>>>> Hi all, >> >>>>>> >> >>>>>> I found rsyslogd occasionally produced corrupted log entries like >> >>>>>> >> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >> >>>>>> device eth0 entered promiscuous mode >> >>>>>> >> >>>>>> I believe that problem appeared in earlier versions including 5.8.2. >> >>>>>> Looks like corruption never occur before hostname field. Is it a >> >>>>>> reported bug? Any clue? >> >>>>> >> >>>>> >> >>>>> the question is if this problem is in the raw message being sent >> >>>>> to you, or is it something added by the rsyslog processing. >> >>>>> >> >>>>> I would suggest adding something along the following line. >> >>>>> >> >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >> >>>>> >> >>>>> this will look for any messages with the hex 01 character in them >> >>>>> and spit out all the info that you can use about the message in to >> >>>>> the file /var/log/badmessages >> >>>>> >> >>>>> run this for a little while and look to see what the raw message >> >>>>> that is being received over the wire looks like. If it's bad, then >> >>>>> you need to look at the sender. If the raw message looks sane, but >> >>>>> rsyslog isn't handling it right, then we can troubleshoot from there. >> >>>> >> >>>> Looks like it's related to tcp transmission. On the original server >> >>>> that get input from unix socket, it is OK; while on the centeral >> >>>> log server, the raw message is already corrupted. >> >>>> >> >>>> ===== on central log server >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >> >>>> from >> >>>> UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >> SNMP >> >>>> packet(s) from UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >> >>>> Connection from UDP: [172.25.0.230]:53547 >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >> >>>> APP-NAME: 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >> >>>> snmpd >> >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >> >>>> APP-NAME: 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >> [172.25.0.230]:53547' >> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >> >>>> snmpd >> >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', programname: >> 'snmpd', APP-NAME: >> >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >> >>>> snmpd >> >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >> >>>> from >> >>>> UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >> SNMP >> >>>> packet(s) from UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >> >>>> from >> >>>> UDP: [172.25.0.230]:53547 >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >> >>>> 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >> >>>> Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >> >>>> 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >> [172.25.0.230]:53547' >> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >> >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >> >>>> 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >> >>>> Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> >> >>>> Thanks, >> >>>> Kaiwang >> >>>>> >> >>>>> David Lang >> >>>>> >> >>>>> ?> rsyslogd 5.8.6, compiled with: >> >>>>>> >> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >> >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >> >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >> >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >> >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >> >>>>>> >> >>>>>> /etc/rsyslog.conf >> >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >> >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >> >>>>>> >> >>>>>> $ModLoad imklog >> >>>>>> $ModLoad imuxsock >> >>>>>> $ModLoad impstats >> >>>>>> >> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >> >>>>>> $SystemLogUsePIDFromSystem on >> >>>>>> >> >>>>>> $PStatInterval 600 >> >>>>>> $PStatSeverity 7 >> >>>>>> >> >>>>>> $WorkDirectory /var/spool/rsyslog >> >>>>>> >> >>>>>> $MainMsgQueueSaveOnShutdown on >> >>>>>> $MainMsgQueueFileName mq >> >>>>>> $MainMsgQueueMaxFileSize 5m >> >>>>>> >> >>>>>> $ActionQueueType LinkedList >> >>>>>> $ActionQueueSaveOnShutdown on >> >>>>>> $ActionQueueFileName dbq >> >>>>>> $ActionQueueMaxFileSize 10m >> >>>>>> $ActionResumeRetryCount -1 >> >>>>>> *.* ? ? ? @@(o)10.2.3.4 >> >>>>>> >> >>>>>> >> >>>>>> # Log all kernel messages to the console. >> >>>>>> # Logging much else clutters up the screen. >> >>>>>> #kern.* >> >>>>>> /dev/console >> >>>>>> >> >>>>>> # Log anything (except mail) of level info or higher. >> >>>>>> # Don't log private authentication messages! >> >>>>>> *.info;mail.none;authpriv.none;cron.none >> >>>>>> /var/log/messages >> >>>>>> >> >>>>>> # The authpriv file has restricted access. >> >>>>>> authpriv.* >> >>>>>> /var/log/secure >> >>>>>> >> >>>>>> # Log all the mail messages in one place. >> >>>>>> mail.* >> >>>>>> -/var/log/maillog >> >>>>>> >> >>>>>> >> >>>>>> # Log cron stuff >> >>>>>> cron.* >> >>>>>> /var/log/cron >> >>>>>> >> >>>>>> # Everybody gets emergency messages >> *.emerg >> >>>>>> * >> >>>>>> >> >>>>>> # Save news errors of level crit and higher in a special file. >> >>>>>> uucp,news.crit >> >>>>>> /var/log/spooler >> >>>>>> >> >>>>>> # Save boot messages also to boot.log >> >>>>>> local7.* >> >>>>>> /var/log/boot.log >> >>>>>> >> >>>>>> >> >>>>>> Thanks, >> >>>>>> Kaiwang >> >>>>>> _______________________________________________ >> >>>>>> rsyslog mailing list >> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>>>> http://www.rsyslog.com/professional-services/ >> >>>>>> >> >>>>> _______________________________________________ >> >>>>> rsyslog mailing list >> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>>> http://www.rsyslog.com/professional-services/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From francois at fmarier.org Wed Jan 11 09:05:10 2012 From: francois at fmarier.org (Francois Marier) Date: Wed, 11 Jan 2012 21:05:10 +1300 Subject: [rsyslog] Use of the "discard" action In-Reply-To: References: <20120109093519.GE6656@isafjordur.dyndns.org> Message-ID: <20120111080510.GE20032@isafjordur.dyndns.org> On 2012-01-09 at 18:03:29, david at lang.hm wrote: > :fromhost-ip, startswith, "192.168.1.1" /var/log/gargoyle-router.log > > and finally, this will match 192.168.1.100 as well as 192.168.1.1 so > you probably want equals instead of startswith. Thanks to your suggestion, I now have a config that works: $ModLoad imudp $UDPServerRun 514 :fromhost-ip, isequal, "192.168.1.1" -/var/log/gargoyle-router.log & ~ (just before the line that outputs to /var/log/syslog) Cheers, Francois -- Francois Marier identi.ca/fmarier http://fmarier.org twitter.com/fmarier From david at lang.hm Wed Jan 11 12:12:36 2012 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Jan 2012 03:12:36 -0800 (PST) Subject: [rsyslog] Use of the "discard" action In-Reply-To: <20120111080510.GE20032@isafjordur.dyndns.org> References: <20120109093519.GE6656@isafjordur.dyndns.org> <20120111080510.GE20032@isafjordur.dyndns.org> Message-ID: note that in rsyslog, it defaults to async writes so the - is not needed ahead of the filename (and there are various config paramters to set if you want more safety at the expense of speed) David Lang On Wed, 11 Jan 2012, Francois Marier wrote: > On 2012-01-09 at 18:03:29, david at lang.hm wrote: >> :fromhost-ip, startswith, "192.168.1.1" /var/log/gargoyle-router.log >> >> and finally, this will match 192.168.1.100 as well as 192.168.1.1 so >> you probably want equals instead of startswith. > > Thanks to your suggestion, I now have a config that works: > > $ModLoad imudp > $UDPServerRun 514 > :fromhost-ip, isequal, "192.168.1.1" -/var/log/gargoyle-router.log > & ~ > > (just before the line that outputs to /var/log/syslog) > > Cheers, > Francois From kaushalshriyan at gmail.com Thu Jan 12 00:45:13 2012 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Thu, 12 Jan 2012 05:15:13 +0530 Subject: [rsyslog] rsyslog Centralized Logging Server along with rsyslog clients Message-ID: Hi I have Centralized Rsyslog server running on Ubuntu Linux Server 10.04 and also rsyslog client running on it. http://pastebin.ubuntu.com/801187/ -> Centralised Rsyslog Server http://pastebin.ubuntu.com/801189/ -> Rsyslog Client and http://pastebin.ubuntu.com/801190/ -> cat /etc/rsyslog.d/50-default.conf Please help me understand if my above set up is correct and correct me if i am wrong. Regards Kaushal From david at lang.hm Thu Jan 12 01:38:43 2012 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Jan 2012 16:38:43 -0800 (PST) Subject: [rsyslog] rsyslog Centralized Logging Server along with rsyslog clients In-Reply-To: References: Message-ID: On Thu, 12 Jan 2012, Kaushal Shriyan wrote: > Hi > > I have Centralized Rsyslog server running on Ubuntu Linux Server 10.04 and > also rsyslog client running on it. > > http://pastebin.ubuntu.com/801187/ -> Centralised Rsyslog Server > http://pastebin.ubuntu.com/801189/ -> Rsyslog Client and > http://pastebin.ubuntu.com/801190/ -> cat /etc/rsyslog.d/50-default.conf > > Please help me understand if my above set up is correct and correct me if i > am wrong. before anyone could tell you if it is correct, you would need to explain more about what you are trying to do. Also, 'correct' can vary a lot based on the details of your network. you are really better off trying something, and if it doesn't work the way you expect it to, then asking for help rather than asking people to create your configs (or validate your configs) cold without any indication of a problem. I doubt that you intend it to come across this way, but you are sounding very much like "please do my homework for me" (both here and on the SEC list). There is a lot of documentation for both products, including examples. You need to spend some time looking through them before asking for help on the lists. David Lang From kaiwang.chen at gmail.com Thu Jan 12 08:42:37 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Thu, 12 Jan 2012 15:42:37 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Message-ID: To report that such problem is not spotted on these four hosts since upgraded to 5.8.7-beta. Thanks, Kaiwang 2012/1/10 Kaiwang Chen : > Applied to four hosts where coredump had been spotted; rpm was built > with the following instructions: > > git clone git://git.adiscon.com/git/rsyslog.git > cd rsyslog > git checkout v5-stable > ./autogen.sh > cd .. > mv rsyslog rsyslog-5.8.7 > tar czf rsyslog-5.8.7-beta.tar.gz rsyslog-5.8.7 --exclude '.git*' > > rpmbuild -bs rsyslog.spec > mock /usr/src/redhat/SRPMS/rsyslog-5.8.7-beta.el5.kc.src.rpm > > > I will report back in two days. > > Thanks, > Kaiwang > > 2012/1/9 Rainer Gerhards : >> Thanks for reporting and tracking down this bug. I was on vacation and have >> just returned. I applied your patch, it is now merged: >> >> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e338a6e5 >> fa14c0394bde6f1d >> >> I'd appreciate if you could double-check that the git version is correct (you >> never know...). I will release this version during the next days. >> >> Once again for your help! >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen >>> Sent: Monday, January 09, 2012 9:29 AM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption >>> >>> To report that coredump and message corruption disappeared after the >>> application of my patch; I guess it's fixed. >>> >>> Thanks, >>> Kaiwang >>> >>> 2012/1/5 Kaiwang Chen : >>> > Hello, >>> > >>> > I check ChangeLog and found 5.8.4 fixed some bug related to race >>> > condition as discussed in >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=275 >>> > >>> > with patch protecting modification in the getMSGID(): >>> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 >>> > 6cc230685c5b7c90f8590b4 >>> > >>> > >>> > I guess the situation is almost identical to what in my case: >>> > corruption is always at the APPNAME field and there is, I believe, a >>> > typo in line 1955 from getAPPNAME() effectively ruins the mutex >>> > proctection: >>> > 1947 /* rgerhards, 2005-11-24 >>> > 1948 ?*/ >>> > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) >>> > 1950 { >>> > 1951 ? ? ? ? uchar *pszRet; >>> > 1952 >>> > 1953 ? ? ? ? assert(pM != NULL); >>> > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >>> > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ >>> > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >>> > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) >>> > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); >>> > 1959 ? ? ? ? else >>> > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); >>> > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >>> > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); >>> > 1963 ? ? ? ? return (char*)pszRet; >>> > 1964 } >>> > >>> > Similar cases were spotted in getPROCID(), getStructuredData() and >>> > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball >>> > should solve the problem, I will report back later. >>> > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 >>> > +0800 >>> > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 >>> > @@ -1609,7 +1609,7 @@ >>> > >>> > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); >>> > + ? ? ? ? ? ? ? MsgLock(pM); >>> > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); >>> > ? ? ? ?if(pM->pCSPROCID == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ >>> > ?{ >>> > ? ? ? ?uchar *pszRet; >>> > >>> > - ? ? ? MsgUnlock(pM); >>> > + ? ? ? MsgLock(pM); >>> > ? ? ? ?if(pM->pCSStrucData == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); >>> > ? ? ? ?else >>> > @@ -1894,7 +1894,7 @@ >>> > ? ? ? ?uchar *pszRet; >>> > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); >>> > + ? ? ? ? ? ? ? MsgLock(pM); >>> > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); >>> > ? ? ? ?if(pM->pCSProgName == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ >>> > >>> > ? ? ? ?assert(pM != NULL); >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); >>> > + ? ? ? ? ? ? ? MsgLock(pM); >>> > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >>> > ? ? ? ?if(pM->pCSAPPNAME == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); >>> > >>> > >>> > By the way, many crashes led to the convergence, for example //// db1 >>> > core.26581 signal 11, Segmentation fault. >>> > (gdb) p *pThis >>> > $1 = { >>> > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", >>> > ?pszBuf = 0x7f7f5c000dd0 "sshd", >>> > ?iBufSize = 4, >>> > ?iStrLen = 4 >>> > } >>> > (gdb) where >>> > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at >>> > stringbuf.c:334 >>> > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 >>> > '\001') at msg.c:1960 >>> > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, >>> pTpe=0x12ef350, >>> > propID=115 's', pPropLen=0x43fb8f00, >>> > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 >>> > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, >>> > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 >>> > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, >>> > pBatch=0x12f7898, pbShutdownImmediate=) >>> > ? ?at ../action.c:738 >>> > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, >>> > pbShutdownImmediate=) at ../action.c:1163 >>> > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, >>> > pWti=0x12f7870) at queue.c:1705 >>> > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 >>> > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 >>> > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 >>> > #10 0x0000000000000000 in ?? () >>> > >>> > >>> > >>> > Thanks, >>> > Kaiwang >>> > >>> > 2011/12/31 Kaiwang Chen : >>> >> Even worse case >>> >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog >>> >> 5.8.6, log source = /proc/kmsg started. >>> >> ...several SYN flooding entries in the same second... >>> >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size >>> >> (fast): 0x00007fafb8000e00 *** >>> >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: >>> >> 0x00007fafb8000e20 *** >>> >> >>> >> ?And possibly hours later the host was not responsive to ssh >>> >> connection requests, and even TTY login attempts failed... >>> >> Interestingly the debug log in another file continues a little beyond: >>> >> >>> >> Debug line with all properties: >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >>> >> 'rsyslogd-pstats', >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >>> >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >>> >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >>> >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 >>> maxqsize=0 ' >>> >> >>> >> Debug line with all properties: >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >>> >> 'rsyslogd-pstats', >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >>> >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >>> >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >>> >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 >>> maxqsize=5 ' >>> >> >>> >> Fortunately the squid service on that host was not badly affected. So >>> >> I did graceful failover and reboot the host, everything became OK. >>> >> Although there was no direct timestamp associated, I guess the >>> >> problem occured soon after rsyslogd started. I also spotted several >>> >> corrupted messages on other hosts when rsyslogd started. They are same >>> version: >>> >> >>> >> # rsyslogd -v >>> >> rsyslogd 5.8.6, compiled with: >>> >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>> >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>> >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>> >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No >>> >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes >>> >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes >>> >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No >>> >> >>> >> >>> >> Thanks, >>> >> Kaiwang >>> >> >>> >> 2011/12/31 Kaiwang Chen : >>> >>> Also spotted corrupted messages from imuxsock: >>> >>> >>> >>> Debug line with all properties: >>> >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', >>> >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >>> >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >>> >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >>> >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> >>> ratelimit.numratelimiters=213 ' >>> >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> >>> ratelimit.numratelimiters=213 ' >>> >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 >>> >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >>> >>> >>> >>> >>> >>> Thanks, >>> >>> Kaiwang >>> >>> >>> >>> 2011/12/27 Kaiwang Chen : >>> >>>> 2011/12/25 ?: >>> >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >>> >>>>> >>> >>>>>> Hi all, >>> >>>>>> >>> >>>>>> I found rsyslogd occasionally produced corrupted log entries like >>> >>>>>> >>> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >>> >>>>>> device eth0 entered promiscuous mode >>> >>>>>> >>> >>>>>> I believe that problem appeared in earlier versions including 5.8.2. >>> >>>>>> Looks like corruption never occur before hostname field. Is it a >>> >>>>>> reported bug? Any clue? >>> >>>>> >>> >>>>> >>> >>>>> the question is if this problem is in the raw message being sent >>> >>>>> to you, or is it something added by the rsyslog processing. >>> >>>>> >>> >>>>> I would suggest adding something along the following line. >>> >>>>> >>> >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >>> >>>>> >>> >>>>> this will look for any messages with the hex 01 character in them >>> >>>>> and spit out all the info that you can use about the message in to >>> >>>>> the file /var/log/badmessages >>> >>>>> >>> >>>>> run this for a little while and look to see what the raw message >>> >>>>> that is being received over the wire looks like. If it's bad, then >>> >>>>> you need to look at the sender. If the raw message looks sane, but >>> >>>>> rsyslog isn't handling it right, then we can troubleshoot from there. >>> >>>> >>> >>>> Looks like it's related to tcp transmission. On the original server >>> >>>> that get input from unix socket, it is OK; while on the centeral >>> >>>> log server, the raw message is already corrupted. >>> >>>> >>> >>>> ===== on central log server >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >>> >>>> from >>> >>>> UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >>> SNMP >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >>> >>>> Connection from UDP: [172.25.0.230]:53547 >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >>> >>>> APP-NAME: 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >>> >>>> snmpd >>> >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >>> >>>> APP-NAME: 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >>> [172.25.0.230]:53547' >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >>> >>>> snmpd >>> >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', programname: >>> 'snmpd', APP-NAME: >>> >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >>> >>>> snmpd >>> >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >>> >>>> from >>> >>>> UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >>> SNMP >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >>> >>>> from >>> >>>> UDP: [172.25.0.230]:53547 >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >>> >>>> 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> >>>> Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >>> >>>> 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >>> [172.25.0.230]:53547' >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >>> >>>> 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> >>>> Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> >>> >>>> Thanks, >>> >>>> Kaiwang >>> >>>>> >>> >>>>> David Lang >>> >>>>> >>> >>>>> ?> rsyslogd 5.8.6, compiled with: >>> >>>>>> >>> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>> >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>> >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>> >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >>> >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >>> >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >>> >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >>> >>>>>> >>> >>>>>> /etc/rsyslog.conf >>> >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >>> >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >>> >>>>>> >>> >>>>>> $ModLoad imklog >>> >>>>>> $ModLoad imuxsock >>> >>>>>> $ModLoad impstats >>> >>>>>> >>> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >>> >>>>>> $SystemLogUsePIDFromSystem on >>> >>>>>> >>> >>>>>> $PStatInterval 600 >>> >>>>>> $PStatSeverity 7 >>> >>>>>> >>> >>>>>> $WorkDirectory /var/spool/rsyslog >>> >>>>>> >>> >>>>>> $MainMsgQueueSaveOnShutdown on >>> >>>>>> $MainMsgQueueFileName mq >>> >>>>>> $MainMsgQueueMaxFileSize 5m >>> >>>>>> >>> >>>>>> $ActionQueueType LinkedList >>> >>>>>> $ActionQueueSaveOnShutdown on >>> >>>>>> $ActionQueueFileName dbq >>> >>>>>> $ActionQueueMaxFileSize 10m >>> >>>>>> $ActionResumeRetryCount -1 >>> >>>>>> *.* ? ? ? @@(o)10.2.3.4 >>> >>>>>> >>> >>>>>> >>> >>>>>> # Log all kernel messages to the console. >>> >>>>>> # Logging much else clutters up the screen. >>> >>>>>> #kern.* >>> >>>>>> /dev/console >>> >>>>>> >>> >>>>>> # Log anything (except mail) of level info or higher. >>> >>>>>> # Don't log private authentication messages! >>> >>>>>> *.info;mail.none;authpriv.none;cron.none >>> >>>>>> /var/log/messages >>> >>>>>> >>> >>>>>> # The authpriv file has restricted access. >>> >>>>>> authpriv.* >>> >>>>>> /var/log/secure >>> >>>>>> >>> >>>>>> # Log all the mail messages in one place. >>> >>>>>> mail.* >>> >>>>>> -/var/log/maillog >>> >>>>>> >>> >>>>>> >>> >>>>>> # Log cron stuff >>> >>>>>> cron.* >>> >>>>>> /var/log/cron >>> >>>>>> >>> >>>>>> # Everybody gets emergency messages >>> *.emerg >>> >>>>>> * >>> >>>>>> >>> >>>>>> # Save news errors of level crit and higher in a special file. >>> >>>>>> uucp,news.crit >>> >>>>>> /var/log/spooler >>> >>>>>> >>> >>>>>> # Save boot messages also to boot.log >>> >>>>>> local7.* >>> >>>>>> /var/log/boot.log >>> >>>>>> >>> >>>>>> >>> >>>>>> Thanks, >>> >>>>>> Kaiwang >>> >>>>>> _______________________________________________ >>> >>>>>> rsyslog mailing list >>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>>>>> http://www.rsyslog.com/professional-services/ >>> >>>>>> >>> >>>>> _______________________________________________ >>> >>>>> rsyslog mailing list >>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>>>> http://www.rsyslog.com/professional-services/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 09:40:03 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 03:40:03 -0500 Subject: [rsyslog] udp forwarding performance Message-ID: Hi all, I am wondering if anybody has performance testing numbers for rsyslog forwarding (using UDP) ? Currently, I am forwarding all of my rsyslog messages from one host to another host (also running rsyslog). Both hosts are running version 4.2.0. I am seeing throughput of about 2k-3k messages/second on receiver. If I try to send more messages from sender, I find that receiver's rsyslog is dropping logs. Both hosts share same virtual network with network latency of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it seems that rsyslog is capable of performing processing 100k messages/second. Can someone please clarify that this number (100k) is for local syslog messages only? (as opposed to receiving 100k messages/second over network) My intent is to find out what is the expected number of messages that rsyslog can forward to other rsyslog host? (assuming average host spec- dual core, 2 GB RAM) Cheers, Gaurav From rgerhards at hq.adiscon.com Thu Jan 12 09:45:01 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 09:45:01 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> I guess your problem is rooted in name resolution. A full blown DNS cache has only recently been introduced in v6. Previous version had limited caching, which usually works well, but not in all cases. Also 4.2.0 is *way outdated*, so probably the issue (if it is one rooted in code) may be solved by running the current 4.8.6 version. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > Sent: Thursday, January 12, 2012 9:40 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] udp forwarding performance > > Hi all, > I am wondering if anybody has performance testing numbers for rsyslog > forwarding (using UDP) ? > > Currently, I am forwarding all of my rsyslog messages from one host to > another host (also running rsyslog). Both hosts are running version > 4.2.0. > I am seeing throughput of about 2k-3k messages/second on receiver. If I > try > to send more messages from sender, I find that receiver's rsyslog is > dropping logs. Both hosts share same virtual network with network > latency > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > seems > that rsyslog is capable of performing processing 100k messages/second. > > Can someone please clarify that this number (100k) is for local syslog > messages only? (as opposed to receiving 100k messages/second over > network) > > My intent is to find out what is the expected number of messages that > rsyslog can forward to other rsyslog host? (assuming average host spec- > dual core, 2 GB RAM) > > Cheers, > Gaurav > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From rgerhards at hq.adiscon.com Thu Jan 12 09:45:40 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 09:45:40 +0100 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281716@GRFEXC.intern.adiscon.com> Thanks for that info and all your help. I'll probably very soon do an official 5.8.7 release. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen > Sent: Thursday, January 12, 2012 8:43 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption > > To report that such problem is not spotted on these four hosts since > upgraded to 5.8.7-beta. > > Thanks, > Kaiwang > > 2012/1/10 Kaiwang Chen : > > Applied to four hosts where coredump had been spotted; rpm was built > > with the following instructions: > > > > git clone git://git.adiscon.com/git/rsyslog.git > > cd rsyslog > > git checkout v5-stable > > ./autogen.sh > > cd .. > > mv rsyslog rsyslog-5.8.7 > > tar czf rsyslog-5.8.7-beta.tar.gz rsyslog-5.8.7 --exclude '.git*' > > > > rpmbuild -bs rsyslog.spec > > mock /usr/src/redhat/SRPMS/rsyslog-5.8.7-beta.el5.kc.src.rpm > > > > > > I will report back in two days. > > > > Thanks, > > Kaiwang > > > > 2012/1/9 Rainer Gerhards : > >> Thanks for reporting and tracking down this bug. I was on vacation > and have > >> just returned. I applied your patch, it is now merged: > >> > >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e3 > 38a6e5 > >> fa14c0394bde6f1d > >> > >> I'd appreciate if you could double-check that the git version is > correct (you > >> never know...). I will release this version during the next days. > >> > >> Once again for your help! > >> > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen > >>> Sent: Monday, January 09, 2012 9:29 AM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption > >>> > >>> To report that coredump and message corruption disappeared after > the > >>> application of my patch; I guess it's fixed. > >>> > >>> Thanks, > >>> Kaiwang > >>> > >>> 2012/1/5 Kaiwang Chen : > >>> > Hello, > >>> > > >>> > I check ChangeLog and found 5.8.4 fixed some bug related to race > >>> > condition as discussed in > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=275 > >>> > > >>> > with patch protecting modification in the getMSGID(): > >>> > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 > >>> > 6cc230685c5b7c90f8590b4 > >>> > > >>> > > >>> > I guess the situation is almost identical to what in my case: > >>> > corruption is always at the APPNAME field and there is, I > believe, a > >>> > typo in line 1955 from getAPPNAME() effectively ruins the mutex > >>> > proctection: > >>> > 1947 /* rgerhards, 2005-11-24 > >>> > 1948 ?*/ > >>> > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) > >>> > 1950 { > >>> > 1951 ? ? ? ? uchar *pszRet; > >>> > 1952 > >>> > 1953 ? ? ? ? assert(pM != NULL); > >>> > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > >>> > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ > >>> > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > >>> > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) > >>> > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); > >>> > 1959 ? ? ? ? else > >>> > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM- > >pCSAPPNAME); > >>> > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > >>> > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > 1963 ? ? ? ? return (char*)pszRet; > >>> > 1964 } > >>> > > >>> > Similar cases were spotted in getPROCID(), getStructuredData() > and > >>> > getProgramName(). ?Hopefully the following patch against 5.8.6 > tarball > >>> > should solve the problem, I will report back later. > >>> > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 > 17:53:02.000000000 > >>> > +0800 > >>> > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 > +0800 > >>> > @@ -1609,7 +1609,7 @@ > >>> > > >>> > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > + ? ? ? ? ? ? ? MsgLock(pM); > >>> > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); > >>> > ? ? ? ?if(pM->pCSPROCID == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ > >>> > ?{ > >>> > ? ? ? ?uchar *pszRet; > >>> > > >>> > - ? ? ? MsgUnlock(pM); > >>> > + ? ? ? MsgLock(pM); > >>> > ? ? ? ?if(pM->pCSStrucData == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); > >>> > ? ? ? ?else > >>> > @@ -1894,7 +1894,7 @@ > >>> > ? ? ? ?uchar *pszRet; > >>> > > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > + ? ? ? ? ? ? ? MsgLock(pM); > >>> > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); > >>> > ? ? ? ?if(pM->pCSProgName == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ > >>> > > >>> > ? ? ? ?assert(pM != NULL); > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > + ? ? ? ? ? ? ? MsgLock(pM); > >>> > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > >>> > ? ? ? ?if(pM->pCSAPPNAME == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > >>> > > >>> > > >>> > By the way, many crashes led to the convergence, for example //// > db1 > >>> > core.26581 signal 11, Segmentation fault. > >>> > (gdb) p *pThis > >>> > $1 = { > >>> > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", > >>> > ?pszBuf = 0x7f7f5c000dd0 "sshd", > >>> > ?iBufSize = 4, > >>> > ?iStrLen = 4 > >>> > } > >>> > (gdb) where > >>> > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at > >>> > stringbuf.c:334 > >>> > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 > >>> > '\001') at msg.c:1960 > >>> > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, > >>> pTpe=0x12ef350, > >>> > propID=115 's', pPropLen=0x43fb8f00, > >>> > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 > >>> > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, > pMsg=0x131c140, > >>> > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 > >>> > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, > >>> > pBatch=0x12f7898, pbShutdownImmediate=) > >>> > ? ?at ../action.c:738 > >>> > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, > >>> > pbShutdownImmediate=) at ../action.c:1163 > >>> > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, > >>> > pWti=0x12f7870) at queue.c:1705 > >>> > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at > wti.c:315 > >>> > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 > >>> > #9 ?0x00000031d5a06367 in __nptl_setxid () from > /lib64/libpthread.so.0 > >>> > #10 0x0000000000000000 in ?? () > >>> > > >>> > > >>> > > >>> > Thanks, > >>> > Kaiwang > >>> > > >>> > 2011/12/31 Kaiwang Chen : > >>> >> Even worse case > >>> >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog > >>> >> 5.8.6, log source = /proc/kmsg started. > >>> >> ...several SYN flooding entries in the same second... > >>> >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size > >>> >> (fast): 0x00007fafb8000e00 *** > >>> >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory > corruption: > >>> >> 0x00007fafb8000e20 *** > >>> >> > >>> >> ?And possibly hours later the host was not responsive to ssh > >>> >> connection requests, and even TTY login attempts failed... > >>> >> Interestingly the debug log in another file continues a little > beyond: > >>> >> > >>> >> Debug line with all properties: > >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: > 'squid6', > >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >>> >> 'rsyslogd-pstats', > >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >>> >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >>> >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >>> >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 > full=0 > >>> maxqsize=0 ' > >>> >> > >>> >> Debug line with all properties: > >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: > 'squid6', > >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >>> >> 'rsyslogd-pstats', > >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >>> >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >>> >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >>> >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 > >>> maxqsize=5 ' > >>> >> > >>> >> Fortunately the squid service on that host was not badly > affected. So > >>> >> I did graceful failover and reboot the host, everything became > OK. > >>> >> Although there was no direct timestamp associated, I guess the > >>> >> problem occured soon after rsyslogd started. I also spotted > several > >>> >> corrupted messages on other hosts when rsyslogd started. They > are same > >>> version: > >>> >> > >>> >> # rsyslogd -v > >>> >> rsyslogd 5.8.6, compiled with: > >>> >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >>> >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >>> >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >>> >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No > >>> >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes > >>> >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes > >>> >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No > >>> >> > >>> >> > >>> >> Thanks, > >>> >> Kaiwang > >>> >> > >>> >> 2011/12/31 Kaiwang Chen : > >>> >>> Also spotted corrupted messages from imuxsock: > >>> >>> > >>> >>> Debug line with all properties: > >>> >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: > 'squid9', > >>> >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > '<88>+?<99>2 > >>> >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: > '-', > >>> >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', > >>> >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> >>> ratelimit.numratelimiters=213 ' > >>> >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> >>> ratelimit.numratelimiters=213 ' > >>> >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 > >>> >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' > >>> >>> > >>> >>> > >>> >>> Thanks, > >>> >>> Kaiwang > >>> >>> > >>> >>> 2011/12/27 Kaiwang Chen : > >>> >>>> 2011/12/25 ?: > >>> >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: > >>> >>>>> > >>> >>>>>> Hi all, > >>> >>>>>> > >>> >>>>>> I found rsyslogd occasionally produced corrupted log entries > like > >>> >>>>>> > >>> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel > ?- - > >>> >>>>>> device eth0 entered promiscuous mode > >>> >>>>>> > >>> >>>>>> I believe that problem appeared in earlier versions > including 5.8.2. > >>> >>>>>> Looks like corruption never occur before hostname field. Is > it a > >>> >>>>>> reported bug? Any clue? > >>> >>>>> > >>> >>>>> > >>> >>>>> the question is if this problem is in the raw message being > sent > >>> >>>>> to you, or is it something added by the rsyslog processing. > >>> >>>>> > >>> >>>>> I would suggest adding something along the following line. > >>> >>>>> > >>> >>>>> :rawmesg, :contains, "#001" > ?/var/log/badmessages:RSYSLOG_DEBUG > >>> >>>>> > >>> >>>>> this will look for any messages with the hex 01 character in > them > >>> >>>>> and spit out all the info that you can use about the message > in to > >>> >>>>> the file /var/log/badmessages > >>> >>>>> > >>> >>>>> run this for a little while and look to see what the raw > message > >>> >>>>> that is being received over the wire looks like. If it's bad, > then > >>> >>>>> you need to look at the sender. If the raw message looks > sane, but > >>> >>>>> rsyslog isn't handling it right, then we can troubleshoot > from there. > >>> >>>> > >>> >>>> Looks like it's related to tcp transmission. On the original > server > >>> >>>> that get input from unix socket, it is OK; while on the > centeral > >>> >>>> log server, the raw message is already corrupted. > >>> >>>> > >>> >>>> ===== on central log server > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - > ?Connection > >>> >>>> from > >>> >>>> UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > >>> SNMP > >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - > >>> >>>> Connection from UDP: [172.25.0.230]:53547 > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: > 'snmpd', > >>> >>>> APP-NAME: 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 > gw71 > >>> >>>> snmpd > >>> >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: > 'snmpd', > >>> >>>> APP-NAME: 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: > >>> [172.25.0.230]:53547' > >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 > gw71 > >>> >>>> snmpd > >>> >>>> 24254 - - ?Received SNMP packet(s) from UDP: > [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', > programname: > >>> 'snmpd', APP-NAME: > >>> >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 > gw71 > >>> >>>> snmpd > >>> >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - > ?Connection > >>> >>>> from > >>> >>>> UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > >>> SNMP > >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - > ?Connection > >>> >>>> from > >>> >>>> UDP: [172.25.0.230]:53547 > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', > PRI: > >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>> >>>> 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>> >>>> Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', > PRI: > >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>> >>>> 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: > >>> [172.25.0.230]:53547' > >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>> >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', > PRI: > >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>> >>>> 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>> >>>> Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> > >>> >>>> Thanks, > >>> >>>> Kaiwang > >>> >>>>> > >>> >>>>> David Lang > >>> >>>>> > >>> >>>>> ?> rsyslogd 5.8.6, compiled with: > >>> >>>>>> > >>> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >>> >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >>> >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >>> >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No > >>> >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes > >>> >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes > >>> >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No > >>> >>>>>> > >>> >>>>>> /etc/rsyslog.conf > >>> >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>> >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>> >>>>>> > >>> >>>>>> $ModLoad imklog > >>> >>>>>> $ModLoad imuxsock > >>> >>>>>> $ModLoad impstats > >>> >>>>>> > >>> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off > >>> >>>>>> $SystemLogUsePIDFromSystem on > >>> >>>>>> > >>> >>>>>> $PStatInterval 600 > >>> >>>>>> $PStatSeverity 7 > >>> >>>>>> > >>> >>>>>> $WorkDirectory /var/spool/rsyslog > >>> >>>>>> > >>> >>>>>> $MainMsgQueueSaveOnShutdown on > >>> >>>>>> $MainMsgQueueFileName mq > >>> >>>>>> $MainMsgQueueMaxFileSize 5m > >>> >>>>>> > >>> >>>>>> $ActionQueueType LinkedList > >>> >>>>>> $ActionQueueSaveOnShutdown on > >>> >>>>>> $ActionQueueFileName dbq > >>> >>>>>> $ActionQueueMaxFileSize 10m > >>> >>>>>> $ActionResumeRetryCount -1 > >>> >>>>>> *.* ? ? ? @@(o)10.2.3.4 > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> # Log all kernel messages to the console. > >>> >>>>>> # Logging much else clutters up the screen. > >>> >>>>>> #kern.* > >>> >>>>>> /dev/console > >>> >>>>>> > >>> >>>>>> # Log anything (except mail) of level info or higher. > >>> >>>>>> # Don't log private authentication messages! > >>> >>>>>> *.info;mail.none;authpriv.none;cron.none > >>> >>>>>> /var/log/messages > >>> >>>>>> > >>> >>>>>> # The authpriv file has restricted access. > >>> >>>>>> authpriv.* > >>> >>>>>> /var/log/secure > >>> >>>>>> > >>> >>>>>> # Log all the mail messages in one place. > >>> >>>>>> mail.* > >>> >>>>>> -/var/log/maillog > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> # Log cron stuff > >>> >>>>>> cron.* > >>> >>>>>> /var/log/cron > >>> >>>>>> > >>> >>>>>> # Everybody gets emergency messages > >>> *.emerg > >>> >>>>>> * > >>> >>>>>> > >>> >>>>>> # Save news errors of level crit and higher in a special > file. > >>> >>>>>> uucp,news.crit > >>> >>>>>> /var/log/spooler > >>> >>>>>> > >>> >>>>>> # Save boot messages also to boot.log > >>> >>>>>> local7.* > >>> >>>>>> /var/log/boot.log > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Thanks, > >>> >>>>>> Kaiwang > >>> >>>>>> _______________________________________________ > >>> >>>>>> rsyslog mailing list > >>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> >>>>>> http://www.rsyslog.com/professional-services/ > >>> >>>>>> > >>> >>>>> _______________________________________________ > >>> >>>>> rsyslog mailing list > >>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> >>>>> http://www.rsyslog.com/professional-services/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 09:48:44 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 03:48:44 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> Message-ID: Thanks Rainer! I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets install by apt-get. I will try to install newer version. BTW, what should be the expected performance (messages/sec) for sending and receiving messages over UDP? Thanks again, Gaurav On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards wrote: > I guess your problem is rooted in name resolution. A full blown DNS cache > has > only recently been introduced in v6. Previous version had limited caching, > which usually works well, but not in all cases. Also 4.2.0 is *way > outdated*, > so probably the issue (if it is one rooted in code) may be solved by > running > the current 4.8.6 version. > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > Sent: Thursday, January 12, 2012 9:40 AM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] udp forwarding performance > > > > Hi all, > > I am wondering if anybody has performance testing numbers for rsyslog > > forwarding (using UDP) ? > > > > Currently, I am forwarding all of my rsyslog messages from one host to > > another host (also running rsyslog). Both hosts are running version > > 4.2.0. > > I am seeing throughput of about 2k-3k messages/second on receiver. If I > > try > > to send more messages from sender, I find that receiver's rsyslog is > > dropping logs. Both hosts share same virtual network with network > > latency > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > > seems > > that rsyslog is capable of performing processing 100k messages/second. > > > > Can someone please clarify that this number (100k) is for local syslog > > messages only? (as opposed to receiving 100k messages/second over > > network) > > > > My intent is to find out what is the expected number of messages that > > rsyslog can forward to other rsyslog host? (assuming average host spec- > > dual core, 2 GB RAM) > > > > Cheers, > > Gaurav > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Thu Jan 12 09:52:12 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 09:52:12 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > Sent: Thursday, January 12, 2012 9:49 AM > To: rsyslog-users > Subject: Re: [rsyslog] udp forwarding performance > > Thanks Rainer! > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets install > by > apt-get. I will try to install newer version. > > BTW, what should be the expected performance (messages/sec) for sending > and > receiving messages over UDP? Wiht v6 > 500k/second, but you possibly need to enable realtime priority in order for the OS to know that rsyslog must be sufficiently often activated. This number assumes lightweight processing (no database inserts or such). Try first without realtime. Rainer > > Thanks again, > Gaurav > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > wrote: > > > I guess your problem is rooted in name resolution. A full blown DNS > cache > > has > > only recently been introduced in v6. Previous version had limited > caching, > > which usually works well, but not in all cases. Also 4.2.0 is *way > > outdated*, > > so probably the issue (if it is one rooted in code) may be solved by > > running > > the current 4.8.6 version. > > > > HTH > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > Sent: Thursday, January 12, 2012 9:40 AM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] udp forwarding performance > > > > > > Hi all, > > > I am wondering if anybody has performance testing numbers for > rsyslog > > > forwarding (using UDP) ? > > > > > > Currently, I am forwarding all of my rsyslog messages from one host > to > > > another host (also running rsyslog). Both hosts are running version > > > 4.2.0. > > > I am seeing throughput of about 2k-3k messages/second on receiver. > If I > > > try > > > to send more messages from sender, I find that receiver's rsyslog > is > > > dropping logs. Both hosts share same virtual network with network > > > latency > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > > > seems > > > that rsyslog is capable of performing processing 100k > messages/second. > > > > > > Can someone please clarify that this number (100k) is for local > syslog > > > messages only? (as opposed to receiving 100k messages/second over > > > network) > > > > > > My intent is to find out what is the expected number of messages > that > > > rsyslog can forward to other rsyslog host? (assuming average host > spec- > > > dual core, 2 GB RAM) > > > > > > Cheers, > > > Gaurav > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 10:32:55 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 04:32:55 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, I am now running version 5.8.1. I was running into rate-limiting issues so I've set following- $IMUXSockRateLimitInterval 0 $SystemLogRateLimitInterval 0 After this, the sender host is sending only about 300-400 messages, then dropping a lot of them (literally thousands). I ran Wireshark to confirm whether if it is receiver's issue or sender's and found that indeed sender is not sending messages. Can you please suggest which directives I should set in config to troubleshoot this? Thanks, Gaurav On Thu, Jan 12, 2012 at 3:52 AM, Rainer Gerhards wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > Sent: Thursday, January 12, 2012 9:49 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] udp forwarding performance > > > > Thanks Rainer! > > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets install > > by > > apt-get. I will try to install newer version. > > > > BTW, what should be the expected performance (messages/sec) for sending > > and > > receiving messages over UDP? > > Wiht v6 > 500k/second, but you possibly need to enable realtime priority in > order for the OS to know that rsyslog must be sufficiently often activated. > This number assumes lightweight processing (no database inserts or such). > > Try first without realtime. > > Rainer > > > > Thanks again, > > Gaurav > > > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > > wrote: > > > > > I guess your problem is rooted in name resolution. A full blown DNS > > cache > > > has > > > only recently been introduced in v6. Previous version had limited > > caching, > > > which usually works well, but not in all cases. Also 4.2.0 is *way > > > outdated*, > > > so probably the issue (if it is one rooted in code) may be solved by > > > running > > > the current 4.8.6 version. > > > > > > HTH > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > Sent: Thursday, January 12, 2012 9:40 AM > > > > To: rsyslog at lists.adiscon.com > > > > Subject: [rsyslog] udp forwarding performance > > > > > > > > Hi all, > > > > I am wondering if anybody has performance testing numbers for > > rsyslog > > > > forwarding (using UDP) ? > > > > > > > > Currently, I am forwarding all of my rsyslog messages from one host > > to > > > > another host (also running rsyslog). Both hosts are running version > > > > 4.2.0. > > > > I am seeing throughput of about 2k-3k messages/second on receiver. > > If I > > > > try > > > > to send more messages from sender, I find that receiver's rsyslog > > is > > > > dropping logs. Both hosts share same virtual network with network > > > > latency > > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > > > > seems > > > > that rsyslog is capable of performing processing 100k > > messages/second. > > > > > > > > Can someone please clarify that this number (100k) is for local > > syslog > > > > messages only? (as opposed to receiving 100k messages/second over > > > > network) > > > > > > > > My intent is to find out what is the expected number of messages > > that > > > > rsyslog can forward to other rsyslog host? (assuming average host > > spec- > > > > dual core, 2 GB RAM) > > > > > > > > Cheers, > > > > Gaurav > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Thu Jan 12 10:55:53 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 10:55:53 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728171A@GRFEXC.intern.adiscon.com> Have your turned off name resolution for a test? Why not use 5.8.6? rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > Sent: Thursday, January 12, 2012 10:33 AM > To: rsyslog-users > Subject: Re: [rsyslog] udp forwarding performance > > Hi Rainer, > I am now running version 5.8.1. I was running into rate-limiting issues > so > I've set following- > $IMUXSockRateLimitInterval 0 > $SystemLogRateLimitInterval 0 > > After this, the sender host is sending only about 300-400 messages, > then > dropping a lot of them (literally thousands). I ran Wireshark to > confirm > whether if it is receiver's issue or sender's and found that indeed > sender > is not sending messages. > > Can you please suggest which directives I should set in config to > troubleshoot this? > > Thanks, > Gaurav > > > > On Thu, Jan 12, 2012 at 3:52 AM, Rainer Gerhards > wrote: > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > Sent: Thursday, January 12, 2012 9:49 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] udp forwarding performance > > > > > > Thanks Rainer! > > > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets > install > > > by > > > apt-get. I will try to install newer version. > > > > > > BTW, what should be the expected performance (messages/sec) for > sending > > > and > > > receiving messages over UDP? > > > > Wiht v6 > 500k/second, but you possibly need to enable realtime > priority in > > order for the OS to know that rsyslog must be sufficiently often > activated. > > This number assumes lightweight processing (no database inserts or > such). > > > > Try first without realtime. > > > > Rainer > > > > > > Thanks again, > > > Gaurav > > > > > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > > > wrote: > > > > > > > I guess your problem is rooted in name resolution. A full blown > DNS > > > cache > > > > has > > > > only recently been introduced in v6. Previous version had limited > > > caching, > > > > which usually works well, but not in all cases. Also 4.2.0 is > *way > > > > outdated*, > > > > so probably the issue (if it is one rooted in code) may be solved > by > > > > running > > > > the current 4.8.6 version. > > > > > > > > HTH > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > > Sent: Thursday, January 12, 2012 9:40 AM > > > > > To: rsyslog at lists.adiscon.com > > > > > Subject: [rsyslog] udp forwarding performance > > > > > > > > > > Hi all, > > > > > I am wondering if anybody has performance testing numbers for > > > rsyslog > > > > > forwarding (using UDP) ? > > > > > > > > > > Currently, I am forwarding all of my rsyslog messages from one > host > > > to > > > > > another host (also running rsyslog). Both hosts are running > version > > > > > 4.2.0. > > > > > I am seeing throughput of about 2k-3k messages/second on > receiver. > > > If I > > > > > try > > > > > to send more messages from sender, I find that receiver's > rsyslog > > > is > > > > > dropping logs. Both hosts share same virtual network with > network > > > > > latency > > > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB > RAM. > > > > > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog- > performance.html it > > > > > seems > > > > > that rsyslog is capable of performing processing 100k > > > messages/second. > > > > > > > > > > Can someone please clarify that this number (100k) is for local > > > syslog > > > > > messages only? (as opposed to receiving 100k messages/second > over > > > > > network) > > > > > > > > > > My intent is to find out what is the expected number of > messages > > > that > > > > > rsyslog can forward to other rsyslog host? (assuming average > host > > > spec- > > > > > dual core, 2 GB RAM) > > > > > > > > > > Cheers, > > > > > Gaurav > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com/professional-services/ > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 11:05:03 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 05:05:03 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728171A@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA728171A@GRFEXC.intern.adiscon.com> Message-ID: I will try to install 5.8.6, btw, disabling dns resolution didn't help. On Thu, Jan 12, 2012 at 4:55 AM, Rainer Gerhards wrote: > Have your turned off name resolution for a test? Why not use 5.8.6? > > rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > Sent: Thursday, January 12, 2012 10:33 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] udp forwarding performance > > > > Hi Rainer, > > I am now running version 5.8.1. I was running into rate-limiting issues > > so > > I've set following- > > $IMUXSockRateLimitInterval 0 > > $SystemLogRateLimitInterval 0 > > > > After this, the sender host is sending only about 300-400 messages, > > then > > dropping a lot of them (literally thousands). I ran Wireshark to > > confirm > > whether if it is receiver's issue or sender's and found that indeed > > sender > > is not sending messages. > > > > Can you please suggest which directives I should set in config to > > troubleshoot this? > > > > Thanks, > > Gaurav > > > > > > > > On Thu, Jan 12, 2012 at 3:52 AM, Rainer Gerhards > > wrote: > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > Sent: Thursday, January 12, 2012 9:49 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] udp forwarding performance > > > > > > > > Thanks Rainer! > > > > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets > > install > > > > by > > > > apt-get. I will try to install newer version. > > > > > > > > BTW, what should be the expected performance (messages/sec) for > > sending > > > > and > > > > receiving messages over UDP? > > > > > > Wiht v6 > 500k/second, but you possibly need to enable realtime > > priority in > > > order for the OS to know that rsyslog must be sufficiently often > > activated. > > > This number assumes lightweight processing (no database inserts or > > such). > > > > > > Try first without realtime. > > > > > > Rainer > > > > > > > > Thanks again, > > > > Gaurav > > > > > > > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > > > > wrote: > > > > > > > > > I guess your problem is rooted in name resolution. A full blown > > DNS > > > > cache > > > > > has > > > > > only recently been introduced in v6. Previous version had limited > > > > caching, > > > > > which usually works well, but not in all cases. Also 4.2.0 is > > *way > > > > > outdated*, > > > > > so probably the issue (if it is one rooted in code) may be solved > > by > > > > > running > > > > > the current 4.8.6 version. > > > > > > > > > > HTH > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > > > Sent: Thursday, January 12, 2012 9:40 AM > > > > > > To: rsyslog at lists.adiscon.com > > > > > > Subject: [rsyslog] udp forwarding performance > > > > > > > > > > > > Hi all, > > > > > > I am wondering if anybody has performance testing numbers for > > > > rsyslog > > > > > > forwarding (using UDP) ? > > > > > > > > > > > > Currently, I am forwarding all of my rsyslog messages from one > > host > > > > to > > > > > > another host (also running rsyslog). Both hosts are running > > version > > > > > > 4.2.0. > > > > > > I am seeing throughput of about 2k-3k messages/second on > > receiver. > > > > If I > > > > > > try > > > > > > to send more messages from sender, I find that receiver's > > rsyslog > > > > is > > > > > > dropping logs. Both hosts share same virtual network with > > network > > > > > > latency > > > > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB > > RAM. > > > > > > > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog- > > performance.html it > > > > > > seems > > > > > > that rsyslog is capable of performing processing 100k > > > > messages/second. > > > > > > > > > > > > Can someone please clarify that this number (100k) is for local > > > > syslog > > > > > > messages only? (as opposed to receiving 100k messages/second > > over > > > > > > network) > > > > > > > > > > > > My intent is to find out what is the expected number of > > messages > > > > that > > > > > > rsyslog can forward to other rsyslog host? (assuming average > > host > > > > spec- > > > > > > dual core, 2 GB RAM) > > > > > > > > > > > > Cheers, > > > > > > Gaurav > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com/professional-services/ > > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > -- From rgerhards at hq.adiscon.com Thu Jan 12 15:14:55 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 15:14:55 +0100 Subject: [rsyslog] update on rsyslog licensing and funding Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> Hi all, we had a lengthy discussion last November on rsyslog funding and licensing. At that time, I promised to keep you updated. I have blogged about the latest state: http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html Feedback is appreciated. Rainer From david at lang.hm Thu Jan 12 20:51:37 2012 From: david at lang.hm (david at lang.hm) Date: Thu, 12 Jan 2012 11:51:37 -0800 (PST) Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: On Thu, 12 Jan 2012, Gaurav Kumar wrote: > Hi all, > I am wondering if anybody has performance testing numbers for rsyslog > forwarding (using UDP) ? > > Currently, I am forwarding all of my rsyslog messages from one host to > another host (also running rsyslog). Both hosts are running version 4.2.0. > I am seeing throughput of about 2k-3k messages/second on receiver. If I try > to send more messages from sender, I find that receiver's rsyslog is > dropping logs. Both hosts share same virtual network with network latency > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it seems > that rsyslog is capable of performing processing 100k messages/second. > > Can someone please clarify that this number (100k) is for local syslog > messages only? (as opposed to receiving 100k messages/second over network) I have received >380K logs/sec (pretty close to gig-e wire speed) with no packet loss. I routinely do tens of thousands of logs/sec between machines. > My intent is to find out what is the expected number of messages that > rsyslog can forward to other rsyslog host? (assuming average host spec- > dual core, 2 GB RAM) it is going to depend a lot on what processing you are doing. disable name resolution (-x on the command line) when you have the systems dropping logs, what is the utilization of the rsyslog threads? David Lang From gk at pivotalsecurity.com Fri Jan 13 04:36:50 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 22:36:50 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: I was finally able to resolve the issue. The root cause was improper tcp/ip config on reciver host. Thanks all for the hints. On Jan 12, 2012 2:51 PM, wrote: > On Thu, 12 Jan 2012, Gaurav Kumar wrote: > > Hi all, >> I am wondering if anybody has performance testing numbers for rsyslog >> forwarding (using UDP) ? >> >> Currently, I am forwarding all of my rsyslog messages from one host to >> another host (also running rsyslog). Both hosts are running version 4.2.0. >> I am seeing throughput of about 2k-3k messages/second on receiver. If I >> try >> to send more messages from sender, I find that receiver's rsyslog is >> dropping logs. Both hosts share same virtual network with network latency >> of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. >> >> As per http://blog.gerhards.net/2008/**10/rsyslog-performance.htmlit seems >> that rsyslog is capable of performing processing 100k messages/second. >> >> Can someone please clarify that this number (100k) is for local syslog >> messages only? (as opposed to receiving 100k messages/second over network) >> > > I have received >380K logs/sec (pretty close to gig-e wire speed) with no > packet loss. I routinely do tens of thousands of logs/sec between machines. > > My intent is to find out what is the expected number of messages that >> rsyslog can forward to other rsyslog host? (assuming average host spec- >> dual core, 2 GB RAM) >> > > it is going to depend a lot on what processing you are doing. > > disable name resolution (-x on the command line) > > when you have the systems dropping logs, what is the utilization of the > rsyslog threads? > > David Lang > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From r.bhatia at ipax.at Fri Jan 13 09:32:10 2012 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 13 Jan 2012 09:32:10 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: <4F0FEC0A.9080204@ipax.at> On 2012-01-13 04:36, Gaurav Kumar wrote: > I was finally able to resolve the issue. The root cause was improper tcp/ip > config on reciver host. Thanks all for the hints. hi gaurav! could you please elaborate on the settings you had to tune/ which settings you are now using? thanks, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From michael at maymann.org Fri Jan 13 13:15:05 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 13 Jan 2012 13:15:05 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples Message-ID: Hi List, I'm new to rsyslog/syslog in general. I would like to syslog from all my 100+ network devices. Preferably I would like a FQDN.log file for each host (or a FQDN-dir containing logs from this host if more logfiles per host are best practice)... Can anyone give me an example of (or link to) best practice of this kind of setup. Thanks in advance :-) ! ~maymann From michael at maymann.org Fri Jan 13 14:43:06 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 13 Jan 2012 14:43:06 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Furthermore: would it be possible to validate FQDN from DNS and not from syslog-info hostname. We are getting a lot of weird logfiles as some applications are not including the hostname as the first parameter in the syslog-entries, e.g.: Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList text /tmp/hostCompatList"'#012#01212/16/11 09:47:40 Would be nice to validate FQDN from sender DNS query... Thanks in advance :-) ! ~maymann 2012/1/13 Michael Maymann > Hi List, > > I'm new to rsyslog/syslog in general. > > I would like to syslog from all my 100+ network devices. > Preferably I would like a FQDN.log file for each host (or a FQDN-dir > containing logs from this host if more logfiles per host are best > practice)... > > Can anyone give me an example of (or link to) best practice of this kind > of setup. > > > Thanks in advance :-) ! > > ~maymann > From rgerhards at hq.adiscon.com Fri Jan 13 18:38:40 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 13 Jan 2012 18:38:40 +0100 Subject: [rsyslog] update on rsyslog licensing and funding In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> Hi all, I created some confusion if the rsyslog project stays under GPLv3 with yesterday's post. In fact, it does! I wrote a new posting today in the hope to clarify some things: http://blog.gerhards.net/2012/01/rsyslog-will-remain-gplv3-licensed.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, January 12, 2012 3:15 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] update on rsyslog licensing and funding > > Hi all, > > we had a lengthy discussion last November on rsyslog funding and > licensing. > At that time, I promised to keep you updated. I have blogged about the > latest > state: > > http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html > > Feedback is appreciated. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From david at lang.hm Fri Jan 13 21:34:32 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Jan 2012 12:34:32 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: you need to be aware that doing the DNS queries is rather expensive (although I think I saw a comment that in the very latest 6.2 version there may now be a DNS cache that will drastically help) you would need to create a template with FROMHOST in it and use that as the filename to write to (look for dynafile in the documentation) note that if you are relaying logs from one machine to another, only the first machine will see the true source in FROMHOST, machines after that will only see the relay box. let me know if this doesn't give you enough clues to learn how to do this. David Lang On Fri, 13 Jan 2012, Michael Maymann wrote: > Date: Fri, 13 Jan 2012 14:43:06 +0100 > From: Michael Maymann > Reply-To: rsyslog-users > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Furthermore: would it be possible to validate FQDN from DNS and not from > syslog-info hostname. > We are getting a lot of weird logfiles as some applications are not > including the hostname as the first parameter in the syslog-entries, e.g.: > Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] > #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList > text /tmp/hostCompatList"'#012#01212/16/11 09:47:40 > > Would be nice to validate FQDN from sender DNS query... > > Thanks in advance :-) ! > ~maymann > > > 2012/1/13 Michael Maymann > >> Hi List, >> >> I'm new to rsyslog/syslog in general. >> >> I would like to syslog from all my 100+ network devices. >> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >> containing logs from this host if more logfiles per host are best >> practice)... >> >> Can anyone give me an example of (or link to) best practice of this kind >> of setup. >> >> >> Thanks in advance :-) ! >> >> ~maymann >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From david at lang.hm Fri Jan 13 21:44:13 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Jan 2012 12:44:13 -0800 (PST) Subject: [rsyslog] update on rsyslog licensing and funding In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> Message-ID: If the licensign text for rsyslog clearly states that the authors of rsyslog do not consider plugins to be defivitive works of rsyslog, that basically makes it so that even if someone tries to claim that the GPL did apply to the plugin, anyone could point to that statement and any judge would dismiss the case (I think the legal term is enstopple, basically you can't give people permission to do something and then sue them for doing it) some people call this an 'exception', but usually the people granting the 'exception' consider it common sense. with linux you have the 'userspace exception' that says that software running in userspace and using the standard system calls is not considered a derived work of the linux kernel. Most people consider this an obvious truth that doesn't really matter, but the fact that it has been clearly stated for a long time means that nobody can now claim otherwise. I belive that you have had a similar statement in place about the plugins for a long time, so it's not that it takes creative thinking to make plugins not be required to be GPL licensed, but it takes creative thinking to justify ignoring that statement and claiming that they are required to be under the GPL. David Lang On Fri, 13 Jan 2012, Rainer Gerhards wrote: > Date: Fri, 13 Jan 2012 18:38:40 +0100 > From: Rainer Gerhards > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] update on rsyslog licensing and funding > > Hi all, > > I created some confusion if the rsyslog project stays under GPLv3 with > yesterday's post. In fact, it does! I wrote a new posting today in the hope > to clarify some things: > > http://blog.gerhards.net/2012/01/rsyslog-will-remain-gplv3-licensed.html > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >> Sent: Thursday, January 12, 2012 3:15 PM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] update on rsyslog licensing and funding >> >> Hi all, >> >> we had a lengthy discussion last November on rsyslog funding and >> licensing. >> At that time, I promised to keep you updated. I have blogged about the >> latest >> state: >> >> http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html >> >> Feedback is appreciated. >> >> Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From gk at pivotalsecurity.com Fri Jan 13 22:34:06 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Fri, 13 Jan 2012 16:34:06 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <4F0FEC0A.9080204@ipax.at> References: <4F0FEC0A.9080204@ipax.at> Message-ID: Hi Raoul, I don't know which exact config resolved the issue. I followed general guidance on tuning TCP/IP stack - http://www.psc.edu/networking/projects/tcptune/historical.php Cheers, Gaurav On Fri, Jan 13, 2012 at 3:32 AM, Raoul Bhatia [IPAX] wrote: > On 2012-01-13 04:36, Gaurav Kumar wrote: > >> I was finally able to resolve the issue. The root cause was improper >> tcp/ip >> config on reciver host. Thanks all for the hints. >> > > hi gaurav! > > could you please elaborate on the settings you had to tune/ > which settings you are now using? > > thanks, > raoul > -- > ______________________________**______________________________**________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ______________________________**______________________________**________ > From david at lang.hm Sat Jan 14 07:46:55 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Jan 2012 22:46:55 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: http://rsyslog.com/article60/ David Lang On Sat, 14 Jan 2012, Michael Maymann wrote: > Date: Sat, 14 Jan 2012 07:23:57 +0100 > From: Michael Maymann > To: rsyslog-users , david at lang.hm, > Michael Maymann > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi David, > > thanks for you kind reply...:-) ! > --- > This didn't seem to get through to the archives for some reason...: > http://lists.adiscon.net/pipermail/rsyslog/2012-January/thread.html > Hope I will not dobbel-post... > --- > I don't use syslog-relays, so this will not cause me any problems. > Don't actually know what version we are running - can see this Monday > morning though... Thanks for this hint... will upgrade to 6.2 if not > already then. > I have to configure this into a already running live production system - > our previous syslog-admin left...:-(. > Could I perhaps ask you to be so kind as to give an configuration example > of how this is done, if I ask really nicely... :-) ? > > Thanks in advance :-) ! > ~maymann > > 2012/1/13 > >> you need to be aware that doing the DNS queries is rather expensive >> (although I think I saw a comment that in the very latest 6.2 version there >> may now be a DNS cache that will drastically help) >> >> you would need to create a template with FROMHOST in it and use that as >> the filename to write to (look for dynafile in the documentation) >> >> note that if you are relaying logs from one machine to another, only the >> first machine will see the true source in FROMHOST, machines after that >> will only see the relay box. >> >> let me know if this doesn't give you enough clues to learn how to do this. >> >> David Lang >> >> On Fri, 13 Jan 2012, Michael Maymann wrote: >> >> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> From: Michael Maymann >>> Reply-To: rsyslog-users >>> To: rsyslog at lists.adiscon.com >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >>> >>> >>> Furthermore: would it be possible to validate FQDN from DNS and not from >>> syslog-info hostname. >>> We are getting a lot of weird logfiles as some applications are not >>> including the hostname as the first parameter in the syslog-entries, e.g.: >>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >>> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >>> >>> Would be nice to validate FQDN from sender DNS query... >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> >>> 2012/1/13 Michael Maymann >>> >>> Hi List, >>>> >>>> I'm new to rsyslog/syslog in general. >>>> >>>> I would like to syslog from all my 100+ network devices. >>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>> containing logs from this host if more logfiles per host are best >>>> practice)... >>>> >>>> Can anyone give me an example of (or link to) best practice of this kind >>>> of setup. >>>> >>>> >>>> Thanks in advance :-) ! >>>> >>>> ~maymann >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>> http://www.rsyslog.com/**professional-services/ >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> > From david at lang.hm Sat Jan 14 09:13:58 2012 From: david at lang.hm (david at lang.hm) Date: Sat, 14 Jan 2012 00:13:58 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: On Sat, 14 Jan 2012, Michael Maymann wrote: > Hi David, > > thanks for this...this is super info...:-) ! > If I have to create different logs per host, will this be the a valid > configuration: > $template DynaFile_messages,?/logfiles_on_nfs/%HOSTNAME%/messages? > *.* -?DynaFile_messages > $template DynaFile_secure,?/logfiles_on_nfs/%HOSTNAME%/secure? > *.* -?DynaFile_secure > $template DynaFile_auth.log,?/logfiles_on_nfs/%HOSTNAME%/auth.log? > *.* -?DynaFile_auth.log I believe so. > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to > create every hosts dir upfront... ? it will create it for you (make sure it's running with the appropriate permissions, if you have rsyslog configured to drop privileges, the lower privileges need the ability to create the directories) > 2. Is DNS caching default enabled or do I have to enable this somewhere > first...? I don't know, I haven't had a chance to look into that yet. David Lang > > Thanks in advance :-) ! > ~maymann > > > 2012/1/14 > >> http://rsyslog.com/article60/ >> >> David Lang >> >> On Sat, 14 Jan 2012, Michael Maymann wrote: >> >> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> From: Michael Maymann >>> To: rsyslog-users , david at lang.hm, >>> Michael Maymann >>> >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >>> >>> Hi David, >>> >>> thanks for you kind reply...:-) ! >>> --- >>> This didn't seem to get through to the archives for some reason...: >>> http://lists.adiscon.net/**pipermail/rsyslog/2012-**January/thread.html >>> Hope I will not dobbel-post... >>> --- >>> I don't use syslog-relays, so this will not cause me any problems. >>> Don't actually know what version we are running - can see this Monday >>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>> already then. >>> I have to configure this into a already running live production system - >>> our previous syslog-admin left...:-(. >>> Could I perhaps ask you to be so kind as to give an configuration example >>> of how this is done, if I ask really nicely... :-) ? >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> 2012/1/13 >>> >>> you need to be aware that doing the DNS queries is rather expensive >>>> (although I think I saw a comment that in the very latest 6.2 version >>>> there >>>> may now be a DNS cache that will drastically help) >>>> >>>> you would need to create a template with FROMHOST in it and use that as >>>> the filename to write to (look for dynafile in the documentation) >>>> >>>> note that if you are relaying logs from one machine to another, only the >>>> first machine will see the true source in FROMHOST, machines after that >>>> will only see the relay box. >>>> >>>> let me know if this doesn't give you enough clues to learn how to do >>>> this. >>>> >>>> David Lang >>>> >>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>> >>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>> >>>>> From: Michael Maymann >>>>> Reply-To: rsyslog-users >>>>> To: rsyslog at lists.adiscon.com >>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>> howto/links/examples >>>>> >>>>> >>>>> Furthermore: would it be possible to validate FQDN from DNS and not from >>>>> syslog-info hostname. >>>>> We are getting a lot of weird logfiles as some applications are not >>>>> including the hostname as the first parameter in the syslog-entries, >>>>> e.g.: >>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>> HostCompatList >>>>> text /tmp/hostCompatList"'#012#****01212/16/11 09:47:40 >>>>> >>>>> >>>>> Would be nice to validate FQDN from sender DNS query... >>>>> >>>>> Thanks in advance :-) ! >>>>> ~maymann >>>>> >>>>> >>>>> 2012/1/13 Michael Maymann >>>>> >>>>> Hi List, >>>>> >>>>>> >>>>>> I'm new to rsyslog/syslog in general. >>>>>> >>>>>> I would like to syslog from all my 100+ network devices. >>>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>>> containing logs from this host if more logfiles per host are best >>>>>> practice)... >>>>>> >>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>> kind >>>>>> of setup. >>>>>> >>>>>> >>>>>> Thanks in advance :-) ! >>>>>> >>>>>> ~maymann >>>>>> >>>>>> ______________________________****_________________ >>>>>> >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>>> >>>>>> >>>>> http://www.rsyslog.com/****professional-services/ >>>>> >>>>>> >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>> >>>>> >>>> http://www.rsyslog.com/****professional-services/ >>>> >>>>> >>>> >>>> >>> > From michael at maymann.org Sat Jan 14 00:01:53 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 00:01:53 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: This didn't seem to get through to the archives or some reason...: http://lists.adiscon.net/pipermail/rsyslog/2012-January/thread.html Hope I will not dobbel-post... ~maymann 2012/1/13 Michael Maymann > Hi David, > > thanks for you kind reply...:-) ! > > I don't use syslog-relays, so this will not cause me any problems. > Don't actually know what version we are running - can see this Monday > morning though... Thanks for this hint... will upgrade to 6.2 if not > already then. > I have to configure this into a already running live production system - > our previous syslog-admin left...:-(. > Could I perhaps ask you to be so kind as to give an configuration example > of how this is done, if I ask really nicely... :-) ? > > > Thanks in advance :-) ! > ~maymann > > > 2012/1/13 > > you need to be aware that doing the DNS queries is rather expensive >> (although I think I saw a comment that in the very latest 6.2 version there >> may now be a DNS cache that will drastically help) >> >> you would need to create a template with FROMHOST in it and use that as >> the filename to write to (look for dynafile in the documentation) >> >> note that if you are relaying logs from one machine to another, only the >> first machine will see the true source in FROMHOST, machines after that >> will only see the relay box. >> >> let me know if this doesn't give you enough clues to learn how to do this. >> >> David Lang >> >> On Fri, 13 Jan 2012, Michael Maymann wrote: >> >> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> From: Michael Maymann >>> Reply-To: rsyslog-users >>> To: rsyslog at lists.adiscon.com >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> howto/links/examples >>> >>> >>> Furthermore: would it be possible to validate FQDN from DNS and not from >>> syslog-info hostname. >>> We are getting a lot of weird logfiles as some applications are not >>> including the hostname as the first parameter in the syslog-entries, >>> e.g.: >>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>> HostCompatList >>> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >>> >>> Would be nice to validate FQDN from sender DNS query... >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> >>> 2012/1/13 Michael Maymann >>> >>> Hi List, >>>> >>>> I'm new to rsyslog/syslog in general. >>>> >>>> I would like to syslog from all my 100+ network devices. >>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>> containing logs from this host if more logfiles per host are best >>>> practice)... >>>> >>>> Can anyone give me an example of (or link to) best practice of this kind >>>> of setup. >>>> >>>> >>>> Thanks in advance :-) ! >>>> >>>> ~maymann >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>> http://www.rsyslog.com/**professional-services/ >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> > > From michael at maymann.org Sat Jan 14 07:20:45 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 07:20:45 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for you kind reply...:-) ! I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to 6.2 if not already then. I have to configure this into a already running live production system - our previous syslog-admin left...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann 2012/1/13 > you need to be aware that doing the DNS queries is rather expensive > (although I think I saw a comment that in the very latest 6.2 version there > may now be a DNS cache that will drastically help) > > you would need to create a template with FROMHOST in it and use that as > the filename to write to (look for dynafile in the documentation) > > note that if you are relaying logs from one machine to another, only the > first machine will see the true source in FROMHOST, machines after that > will only see the relay box. > > let me know if this doesn't give you enough clues to learn how to do this. > > David Lang > > On Fri, 13 Jan 2012, Michael Maymann wrote: > > Date: Fri, 13 Jan 2012 14:43:06 +0100 >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> >> Furthermore: would it be possible to validate FQDN from DNS and not from >> syslog-info hostname. >> We are getting a lot of weird logfiles as some applications are not >> including the hostname as the first parameter in the syslog-entries, e.g.: >> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >> >> Would be nice to validate FQDN from sender DNS query... >> >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/13 Michael Maymann >> >> Hi List, >>> >>> I'm new to rsyslog/syslog in general. >>> >>> I would like to syslog from all my 100+ network devices. >>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>> containing logs from this host if more logfiles per host are best >>> practice)... >>> >>> Can anyone give me an example of (or link to) best practice of this kind >>> of setup. >>> >>> >>> Thanks in advance :-) ! >>> >>> ~maymann >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From rgerhards at hq.adiscon.com Sat Jan 14 11:30:40 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 14 Jan 2012 11:30:40 +0100 Subject: [rsyslog] update on rsyslog licensing and funding In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281742@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, January 13, 2012 9:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] update on rsyslog licensing and funding > > If the licensign text for rsyslog clearly states that the authors of > rsyslog do not consider plugins to be defivitive works of rsyslog, that > basically makes it so that even if someone tries to claim that the GPL > did > apply to the plugin, anyone could point to that statement and any judge > would dismiss the case (I think the legal term is enstopple, basically > you > can't give people permission to do something and then sue them for > doing > it) > > some people call this an 'exception', but usually the people granting > the > 'exception' consider it common sense. > > with linux you have the 'userspace exception' that says that software > running in userspace and using the standard system calls is not > considered > a derived work of the linux kernel. Most people consider this an > obvious > truth that doesn't really matter, but the fact that it has been clearly > stated for a long time means that nobody can now claim otherwise. > > I belive that you have had a similar statement in place about the > plugins > for a long time, so it's not that it takes creative thinking to make > plugins not be required to be GPL licensed, but it takes creative > thinking > to justify ignoring that statement and claiming that they are required > to > be under the GPL. I have searched the various documents (the past days and now again). While they express that we consider plugins as separate projects and don't care about their licensing, the exact text is somewhat weak. I agree, at least under German law, that this probably comes close enough to an exception. On the other hand, the ultimate answer can only be found in court (at least under German law, where a judge would need to interpret the statements). This level of uncertainty, how small it may be, drives some potential users away. We have actually seen this happen and it was a big loss for the project. With the licensing change, we try to solve, or at least reduce that uncertainty. That's also why I wrote that I think the situation does not change so much over how it is right now. Rainer > David Lang > > On Fri, 13 Jan 2012, Rainer Gerhards wrote: > > > Date: Fri, 13 Jan 2012 18:38:40 +0100 > > From: Rainer Gerhards > > Reply-To: rsyslog-users > > To: rsyslog-users > > Subject: Re: [rsyslog] update on rsyslog licensing and funding > > > > Hi all, > > > > I created some confusion if the rsyslog project stays under GPLv3 > with > > yesterday's post. In fact, it does! I wrote a new posting today in > the hope > > to clarify some things: > > > > http://blog.gerhards.net/2012/01/rsyslog-will-remain-gplv3- > licensed.html > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > >> Sent: Thursday, January 12, 2012 3:15 PM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] update on rsyslog licensing and funding > >> > >> Hi all, > >> > >> we had a lengthy discussion last November on rsyslog funding and > >> licensing. > >> At that time, I promised to keep you updated. I have blogged about > the > >> latest > >> state: > >> > >> http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html > >> > >> Feedback is appreciated. > >> > >> Rainer > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Sat Jan 14 08:50:06 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 08:50:06 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for this...this is super info...:-) ! If I have to create different logs per host, will this be the a valid configuration: $template DynaFile_messages,?/logfiles_on_nfs/%HOSTNAME%/messages? *.* -?DynaFile_messages $template DynaFile_secure,?/logfiles_on_nfs/%HOSTNAME%/secure? *.* -?DynaFile_secure $template DynaFile_auth.log,?/logfiles_on_nfs/%HOSTNAME%/auth.log? *.* -?DynaFile_auth.log 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to create every hosts dir upfront... ? 2. Is DNS caching default enabled or do I have to enable this somewhere first...? Thanks in advance :-) ! ~maymann 2012/1/14 > http://rsyslog.com/article60/ > > David Lang > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > Date: Sat, 14 Jan 2012 07:23:57 +0100 >> From: Michael Maymann >> To: rsyslog-users , david at lang.hm, >> Michael Maymann >> >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> Hi David, >> >> thanks for you kind reply...:-) ! >> --- >> This didn't seem to get through to the archives for some reason...: >> http://lists.adiscon.net/**pipermail/rsyslog/2012-**January/thread.html >> Hope I will not dobbel-post... >> --- >> I don't use syslog-relays, so this will not cause me any problems. >> Don't actually know what version we are running - can see this Monday >> morning though... Thanks for this hint... will upgrade to 6.2 if not >> already then. >> I have to configure this into a already running live production system - >> our previous syslog-admin left...:-(. >> Could I perhaps ask you to be so kind as to give an configuration example >> of how this is done, if I ask really nicely... :-) ? >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/13 >> >> you need to be aware that doing the DNS queries is rather expensive >>> (although I think I saw a comment that in the very latest 6.2 version >>> there >>> may now be a DNS cache that will drastically help) >>> >>> you would need to create a template with FROMHOST in it and use that as >>> the filename to write to (look for dynafile in the documentation) >>> >>> note that if you are relaying logs from one machine to another, only the >>> first machine will see the true source in FROMHOST, machines after that >>> will only see the relay box. >>> >>> let me know if this doesn't give you enough clues to learn how to do >>> this. >>> >>> David Lang >>> >>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>> >>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> >>>> From: Michael Maymann >>>> Reply-To: rsyslog-users >>>> To: rsyslog at lists.adiscon.com >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>> howto/links/examples >>>> >>>> >>>> Furthermore: would it be possible to validate FQDN from DNS and not from >>>> syslog-info hostname. >>>> We are getting a lot of weird logfiles as some applications are not >>>> including the hostname as the first parameter in the syslog-entries, >>>> e.g.: >>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>> HostCompatList >>>> text /tmp/hostCompatList"'#012#****01212/16/11 09:47:40 >>>> >>>> >>>> Would be nice to validate FQDN from sender DNS query... >>>> >>>> Thanks in advance :-) ! >>>> ~maymann >>>> >>>> >>>> 2012/1/13 Michael Maymann >>>> >>>> Hi List, >>>> >>>>> >>>>> I'm new to rsyslog/syslog in general. >>>>> >>>>> I would like to syslog from all my 100+ network devices. >>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>> containing logs from this host if more logfiles per host are best >>>>> practice)... >>>>> >>>>> Can anyone give me an example of (or link to) best practice of this >>>>> kind >>>>> of setup. >>>>> >>>>> >>>>> Thanks in advance :-) ! >>>>> >>>>> ~maymann >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>> >>>> > >>>> http://www.rsyslog.com/****professional-services/ >>>> >>>> > >>>> >>>> ______________________________****_________________ >>>> >>> rsyslog mailing list >>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>> >>> > >>> http://www.rsyslog.com/****professional-services/ >>> >>> > >>> >>> >> From michael at maymann.org Sat Jan 14 07:23:57 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 07:23:57 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for you kind reply...:-) ! --- This didn't seem to get through to the archives for some reason...: http://lists.adiscon.net/pipermail/rsyslog/2012-January/thread.html Hope I will not dobbel-post... --- I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to 6.2 if not already then. I have to configure this into a already running live production system - our previous syslog-admin left...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann 2012/1/13 > you need to be aware that doing the DNS queries is rather expensive > (although I think I saw a comment that in the very latest 6.2 version there > may now be a DNS cache that will drastically help) > > you would need to create a template with FROMHOST in it and use that as > the filename to write to (look for dynafile in the documentation) > > note that if you are relaying logs from one machine to another, only the > first machine will see the true source in FROMHOST, machines after that > will only see the relay box. > > let me know if this doesn't give you enough clues to learn how to do this. > > David Lang > > On Fri, 13 Jan 2012, Michael Maymann wrote: > > Date: Fri, 13 Jan 2012 14:43:06 +0100 >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> >> Furthermore: would it be possible to validate FQDN from DNS and not from >> syslog-info hostname. >> We are getting a lot of weird logfiles as some applications are not >> including the hostname as the first parameter in the syslog-entries, e.g.: >> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >> >> Would be nice to validate FQDN from sender DNS query... >> >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/13 Michael Maymann >> >> Hi List, >>> >>> I'm new to rsyslog/syslog in general. >>> >>> I would like to syslog from all my 100+ network devices. >>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>> containing logs from this host if more logfiles per host are best >>> practice)... >>> >>> Can anyone give me an example of (or link to) best practice of this kind >>> of setup. >>> >>> >>> Thanks in advance :-) ! >>> >>> ~maymann >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From michael at maymann.org Sat Jan 14 07:07:14 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 07:07:14 +0100 Subject: [rsyslog] Syslogging FQDN logfile,logdir howto,links,examples Message-ID: Hi David, thanks for you kind reply...:-) ! Don't know why this is not getting through when i reply-all...? I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to latest 6.2 stable if not already then. I have to configure this into a already running live production system - our previous syslog-admin left for other tasks...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann From michael at maymann.org Fri Jan 13 21:45:14 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 13 Jan 2012 21:45:14 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for you kind reply...:-) ! I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to 6.2 if not already then. I have to configure this into a already running live production system - our previous syslog-admin left...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann 2012/1/13 > you need to be aware that doing the DNS queries is rather expensive > (although I think I saw a comment that in the very latest 6.2 version there > may now be a DNS cache that will drastically help) > > you would need to create a template with FROMHOST in it and use that as > the filename to write to (look for dynafile in the documentation) > > note that if you are relaying logs from one machine to another, only the > first machine will see the true source in FROMHOST, machines after that > will only see the relay box. > > let me know if this doesn't give you enough clues to learn how to do this. > > David Lang > > On Fri, 13 Jan 2012, Michael Maymann wrote: > > Date: Fri, 13 Jan 2012 14:43:06 +0100 >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> >> Furthermore: would it be possible to validate FQDN from DNS and not from >> syslog-info hostname. >> We are getting a lot of weird logfiles as some applications are not >> including the hostname as the first parameter in the syslog-entries, e.g.: >> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >> >> Would be nice to validate FQDN from sender DNS query... >> >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/13 Michael Maymann >> >> Hi List, >>> >>> I'm new to rsyslog/syslog in general. >>> >>> I would like to syslog from all my 100+ network devices. >>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>> containing logs from this host if more logfiles per host are best >>> practice)... >>> >>> Can anyone give me an example of (or link to) best practice of this kind >>> of setup. >>> >>> >>> Thanks in advance :-) ! >>> >>> ~maymann >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From rodney.mckee at gmail.com Sun Jan 15 20:55:46 2012 From: rodney.mckee at gmail.com (Rodney McKee) Date: Mon, 16 Jan 2012 06:55:46 +1100 (EST) Subject: [rsyslog] dropping messages In-Reply-To: <287e447b-fd75-4f02-97b9-d7e2caf30606@wsrmckee> Message-ID: <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> Hello, Is their a way to match the text in a message and drop it based on the content? Rgds Rodney From mbiebl at gmail.com Sun Jan 15 21:07:46 2012 From: mbiebl at gmail.com (Michael Biebl) Date: Sun, 15 Jan 2012 21:07:46 +0100 Subject: [rsyslog] dropping messages In-Reply-To: <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> References: <287e447b-fd75-4f02-97b9-d7e2caf30606@wsrmckee> <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> Message-ID: Am 15. Januar 2012 20:55 schrieb Rodney McKee : > Hello, > > Is their a way to match the text in a message and drop it based on the content? Google for "rsyslog filter by message", click on the first hit. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rodney.mckee at gmail.com Sun Jan 15 21:39:57 2012 From: rodney.mckee at gmail.com (Rodney McKee) Date: Mon, 16 Jan 2012 07:39:57 +1100 Subject: [rsyslog] dropping messages In-Reply-To: References: <287e447b-fd75-4f02-97b9-d7e2caf30606@wsrmckee> <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> Message-ID: That looks good, cheers On 16/01/2012 7:07 AM, "Michael Biebl" wrote: > Am 15. Januar 2012 20:55 schrieb Rodney McKee : > > Hello, > > > > Is their a way to match the text in a message and drop it based on the > content? > > Google for "rsyslog filter by message", click on the first hit. > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From penght at cn.fujitsu.com Mon Jan 16 06:32:59 2012 From: penght at cn.fujitsu.com (Peng Haitao) Date: Mon, 16 Jan 2012 13:32:59 +0800 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID In-Reply-To: <4F069932.9070408@cn.fujitsu.com> References: <4F069932.9070408@cn.fujitsu.com> Message-ID: <4F13B68B.5070203@cn.fujitsu.com> Peng Haitao said the following on 2012-1-6 14:48: > [add list to the CC list] > > When HAVE_SETSID is not defined, rsyslogd will use ioctl() make itself to daemon, > but this can not make rsyslogd process become the process group leader of a new > process group. In RHEL6.1, the status is as follows: > # uname -a > Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux > # /etc/init.d/rsyslog restart > Shutting down system logger: [ OK ] > Starting system logger: [ OK ] > # ps axo pgrp,ppid,pid,comm | grep rsyslog > 6290 1 6301 rsyslogd > > When we send SIGTERM signal to 6290, rsyslogd will die:( So I think we should > call setpgid() before ioctl(). > hi, anyone can comment the patch? -- Best Regards, Peng > Signed-off-by: Peng Haitao > --- > tools/syslogd.c | 7 +++++++ > 1 files changed, 7 insertions(+), 0 deletions(-) > > diff --git a/tools/syslogd.c b/tools/syslogd.c > index d1224f2..ac954b7 100644 > --- a/tools/syslogd.c > +++ b/tools/syslogd.c > @@ -356,8 +356,15 @@ void untty(void) > #else > { > int i; > + pid_t pid; > > if(!Debug) { > + pid = getpid(); > + if (setpgid(pid, pid) < 0) { > + perror("setpgid"); > + exit(1); > + } > + > i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); > if (i >= 0) { > # if !defined(__hpux) From michael at maymann.org Mon Jan 16 08:57:26 2012 From: michael at maymann.org (Michael Maymann) Date: Mon, 16 Jan 2012 08:57:26 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: If I want DNS caching, should i use the new stable-6.2.0 or the older devel-6.3.6... ? Can see this feature mentioned here: http://rsyslog.com/features/ or here: http://rsyslog.com/project-status/ Thanks in advance :-) ! ~maymann 2012/1/14 > On Sat, 14 Jan 2012, Michael Maymann wrote: > > Hi David, >> >> thanks for this...this is super info...:-) ! >> If I have to create different logs per host, will this be the a valid >> configuration: >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >> *.* -?DynaFile_messages >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >> *.* -?DynaFile_secure >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >> *.* -?DynaFile_auth.log >> > > I believe so. > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to >> create every hosts dir upfront... ? >> > > it will create it for you (make sure it's running with the appropriate > permissions, if you have rsyslog configured to drop privileges, the lower > privileges need the ability to create the directories) > > > 2. Is DNS caching default enabled or do I have to enable this somewhere >> first...? >> > > I don't know, I haven't had a chance to look into that yet. > > David Lang > > >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/14 >> >> http://rsyslog.com/article60/ >>> >>> David Lang >>> >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> >>>> From: Michael Maymann >>>> To: rsyslog-users , david at lang.hm, >>>> Michael Maymann >>>> >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>> howto/links/examples >>>> >>>> Hi David, >>>> >>>> thanks for you kind reply...:-) ! >>>> --- >>>> This didn't seem to get through to the archives for some reason...: >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>>> January/thread.html >>>> >>> html >>>> > >>>> >>>> Hope I will not dobbel-post... >>>> --- >>>> I don't use syslog-relays, so this will not cause me any problems. >>>> Don't actually know what version we are running - can see this Monday >>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>> already then. >>>> I have to configure this into a already running live production system - >>>> our previous syslog-admin left...:-(. >>>> Could I perhaps ask you to be so kind as to give an configuration >>>> example >>>> of how this is done, if I ask really nicely... :-) ? >>>> >>>> Thanks in advance :-) ! >>>> ~maymann >>>> >>>> 2012/1/13 >>>> >>>> you need to be aware that doing the DNS queries is rather expensive >>>> >>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>> there >>>>> may now be a DNS cache that will drastically help) >>>>> >>>>> you would need to create a template with FROMHOST in it and use that as >>>>> the filename to write to (look for dynafile in the documentation) >>>>> >>>>> note that if you are relaying logs from one machine to another, only >>>>> the >>>>> first machine will see the true source in FROMHOST, machines after that >>>>> will only see the relay box. >>>>> >>>>> let me know if this doesn't give you enough clues to learn how to do >>>>> this. >>>>> >>>>> David Lang >>>>> >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>> >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>> >>>>> From: Michael Maymann >>>>>> Reply-To: rsyslog-users >>>>>> To: rsyslog at lists.adiscon.com >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>> howto/links/examples >>>>>> >>>>>> >>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>> from >>>>>> syslog-info hostname. >>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>> e.g.: >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>> [set_host_compat_list] >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>> HostCompatList >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>>>>> >>>>>> >>>>>> >>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>> >>>>>> Thanks in advance :-) ! >>>>>> ~maymann >>>>>> >>>>>> >>>>>> 2012/1/13 Michael Maymann >>>>>> >>>>>> Hi List, >>>>>> >>>>>> >>>>>>> I'm new to rsyslog/syslog in general. >>>>>>> >>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>>>> containing logs from this host if more logfiles per host are best >>>>>>> practice)... >>>>>>> >>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>> kind >>>>>>> of setup. >>>>>>> >>>>>>> >>>>>>> Thanks in advance :-) ! >>>>>>> >>>>>>> ~maymann >>>>>>> >>>>>>> ______________________________******_________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>>> >>>>>> > >>>>>> >>>>>> >>>>>> > >>>>>> >>>>>>> >>>>>>> http://www.rsyslog.com/******professional-services/ >>>>>> >>>>>> > >>>>>> >>>>>> >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>> ______________________________******_________________ >>>>>> >>>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>> >>>>> > >>>>> >>>>> >>>>> > >>>>> >>>>>> >>>>>> http://www.rsyslog.com/******professional-services/ >>>>> >>>>> > >>>>> >>>>> >>>>> > >>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >> From rgerhards at hq.adiscon.com Mon Jan 16 10:33:42 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 10:33:42 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> The cache is available since 6.3.1, so you need to go for the devel version. A good place to check those things is the ChangeLog itself, here is the current one: http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8575d085 a0fcf48f71339154813971;hb=HEAD Note that v6-devel is almost as stable as v6-stable except for the config read phase at startup. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > Sent: Monday, January 16, 2012 8:57 AM > To: david at lang.hm > Cc: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > howto/links/examples > > If I want DNS caching, should i use the new stable-6.2.0 or the older > devel-6.3.6... ? > Can see this feature mentioned here: > http://rsyslog.com/features/ > or here: > http://rsyslog.com/project-status/ > > > Thanks in advance :-) ! > ~maymann > > 2012/1/14 > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > Hi David, > >> > >> thanks for this...this is super info...:-) ! > >> If I have to create different logs per host, will this be the a > valid > >> configuration: > >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > >> *.* -?DynaFile_messages > >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > >> *.* -?DynaFile_secure > >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > >> *.* -?DynaFile_auth.log > >> > > > > I believe so. > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > have to > >> create every hosts dir upfront... ? > >> > > > > it will create it for you (make sure it's running with the > appropriate > > permissions, if you have rsyslog configured to drop privileges, the > lower > > privileges need the ability to create the directories) > > > > > > 2. Is DNS caching default enabled or do I have to enable this > somewhere > >> first...? > >> > > > > I don't know, I haven't had a chance to look into that yet. > > > > David Lang > > > > > >> Thanks in advance :-) ! > >> ~maymann > >> > >> > >> 2012/1/14 > >> > >> http://rsyslog.com/article60/ > >>> > >>> David Lang > >>> > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > >>> > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > >>> > >>>> From: Michael Maymann > >>>> To: rsyslog-users , david at lang.hm, > >>>> Michael Maymann > >>>> > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>>> howto/links/examples > >>>> > >>>> Hi David, > >>>> > >>>> thanks for you kind reply...:-) ! > >>>> --- > >>>> This didn't seem to get through to the archives for some > reason...: > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > >>>> > January/thread.html **January/thread.html> > >>>> January/thread.** > >>>> html January/thread.html> > >>>> > > >>>> > >>>> Hope I will not dobbel-post... > >>>> --- > >>>> I don't use syslog-relays, so this will not cause me any problems. > >>>> Don't actually know what version we are running - can see this > Monday > >>>> morning though... Thanks for this hint... will upgrade to 6.2 if > not > >>>> already then. > >>>> I have to configure this into a already running live production > system - > >>>> our previous syslog-admin left...:-(. > >>>> Could I perhaps ask you to be so kind as to give an configuration > >>>> example > >>>> of how this is done, if I ask really nicely... :-) ? > >>>> > >>>> Thanks in advance :-) ! > >>>> ~maymann > >>>> > >>>> 2012/1/13 > >>>> > >>>> you need to be aware that doing the DNS queries is rather > expensive > >>>> > >>>>> (although I think I saw a comment that in the very latest 6.2 > version > >>>>> there > >>>>> may now be a DNS cache that will drastically help) > >>>>> > >>>>> you would need to create a template with FROMHOST in it and use > that as > >>>>> the filename to write to (look for dynafile in the documentation) > >>>>> > >>>>> note that if you are relaying logs from one machine to another, > only > >>>>> the > >>>>> first machine will see the true source in FROMHOST, machines > after that > >>>>> will only see the relay box. > >>>>> > >>>>> let me know if this doesn't give you enough clues to learn how to > do > >>>>> this. > >>>>> > >>>>> David Lang > >>>>> > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > >>>>> > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > >>>>> > >>>>> From: Michael Maymann > >>>>>> Reply-To: rsyslog-users > >>>>>> To: rsyslog at lists.adiscon.com > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>>>>> howto/links/examples > >>>>>> > >>>>>> > >>>>>> Furthermore: would it be possible to validate FQDN from DNS and > not > >>>>>> from > >>>>>> syslog-info hostname. > >>>>>> We are getting a lot of weird logfiles as some applications are > not > >>>>>> including the hostname as the first parameter in the syslog- > entries, > >>>>>> e.g.: > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > >>>>>> [set_host_compat_list] > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData > >>>>>> HostCompatList > >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > >>>>>> > >>>>>> > >>>>>> > >>>>>> Would be nice to validate FQDN from sender DNS query... > >>>>>> > >>>>>> Thanks in advance :-) ! > >>>>>> ~maymann > >>>>>> > >>>>>> > >>>>>> 2012/1/13 Michael Maymann > >>>>>> > >>>>>> Hi List, > >>>>>> > >>>>>> > >>>>>>> I'm new to rsyslog/syslog in general. > >>>>>>> > >>>>>>> I would like to syslog from all my 100+ network devices. > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > FQDN-dir > >>>>>>> containing logs from this host if more logfiles per host are > best > >>>>>>> practice)... > >>>>>>> > >>>>>>> Can anyone give me an example of (or link to) best practice of > this > >>>>>>> kind > >>>>>>> of setup. > >>>>>>> > >>>>>>> > >>>>>>> Thanks in advance :-) ! > >>>>>>> > >>>>>>> ~maymann > >>>>>>> > >>>>>>> ______________________________******_________________ > >>>>>>> > >>>>>>> rsyslog mailing list > >>>>>> > http://lists.adiscon.net/******mailman/listinfo/rsyslog iscon.net/****mailman/listinfo/rsyslog> > >>>>>> > discon.net/**mailman/listinfo/rsyslog> > >>>>>> > > >>>>>> > discon.net/mailman/**listinfo/rsyslog> > >>>>>> > scon.net/mailman/listinfo/rsyslog> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>>> http://www.rsyslog.com/******professional- > services/ > >>>>>> services/ > >>>>>> > > >>>>>> **services/ > >>>>>> services/ > >>>>>> > > >>>>>> > >>>>>>> > >>>>>>> > >>>>>> ______________________________******_________________ > >>>>>> > >>>>>> rsyslog mailing list > >>>>> > http://lists.adiscon.net/******mailman/listinfo/rsyslog iscon.net/****mailman/listinfo/rsyslog> > >>>>> > discon.net/**mailman/listinfo/rsyslog> > >>>>> > > >>>>> > discon.net/mailman/**listinfo/rsyslog> > >>>>> > scon.net/mailman/listinfo/rsyslog> > >>>>> > > >>>>> > >>>>>> > >>>>>> http://www.rsyslog.com/******professional- > services/ > >>>>> services/ > >>>>> > > >>>>> **services/ > >>>>> services/ > >>>>> > > >>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Mon Jan 16 10:47:57 2012 From: michael at maymann.org (Michael Maymann) Date: Mon, 16 Jan 2012 10:47:57 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, Thanks for your reply. How do I install 6.3.6 on RHEL6 easiest ? Thanks in advance :-)! ~maymann 2012/1/16 Rainer Gerhards > The cache is available since 6.3.1, so you need to go for the devel > version. > A good place to check those things is the ChangeLog itself, here is the > current one: > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8575d085 > a0fcf48f71339154813971 > ;hb=HEAD > > Note that v6-devel is almost as stable as v6-stable except for the config > read phase at startup. > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Monday, January 16, 2012 8:57 AM > > To: david at lang.hm > > Cc: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > howto/links/examples > > > > If I want DNS caching, should i use the new stable-6.2.0 or the older > > devel-6.3.6... ? > > Can see this feature mentioned here: > > http://rsyslog.com/features/ > > or here: > > http://rsyslog.com/project-status/ > > > > > > Thanks in advance :-) ! > > ~maymann > > > > 2012/1/14 > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > Hi David, > > >> > > >> thanks for this...this is super info...:-) ! > > >> If I have to create different logs per host, will this be the a > > valid > > >> configuration: > > >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > >> *.* -?DynaFile_messages > > >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > >> *.* -?DynaFile_secure > > >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > >> *.* -?DynaFile_auth.log > > >> > > > > > > I believe so. > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > have to > > >> create every hosts dir upfront... ? > > >> > > > > > > it will create it for you (make sure it's running with the > > appropriate > > > permissions, if you have rsyslog configured to drop privileges, the > > lower > > > privileges need the ability to create the directories) > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > somewhere > > >> first...? > > >> > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > David Lang > > > > > > > > >> Thanks in advance :-) ! > > >> ~maymann > > >> > > >> > > >> 2012/1/14 > > >> > > >> http://rsyslog.com/article60/ > > >>> > > >>> David Lang > > >>> > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > >>> > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > >>> > > >>>> From: Michael Maymann > > >>>> To: rsyslog-users , david at lang.hm, > > >>>> Michael Maymann > > >>>> > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>> howto/links/examples > > >>>> > > >>>> Hi David, > > >>>> > > >>>> thanks for you kind reply...:-) ! > > >>>> --- > > >>>> This didn't seem to get through to the archives for some > > reason...: > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > >>>> > > January/thread.html > **January/thread.html> > > >>>> > January/thread.** > > >>>> html > January/thread.html> > > >>>> > > > >>>> > > >>>> Hope I will not dobbel-post... > > >>>> --- > > >>>> I don't use syslog-relays, so this will not cause me any problems. > > >>>> Don't actually know what version we are running - can see this > > Monday > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 if > > not > > >>>> already then. > > >>>> I have to configure this into a already running live production > > system - > > >>>> our previous syslog-admin left...:-(. > > >>>> Could I perhaps ask you to be so kind as to give an configuration > > >>>> example > > >>>> of how this is done, if I ask really nicely... :-) ? > > >>>> > > >>>> Thanks in advance :-) ! > > >>>> ~maymann > > >>>> > > >>>> 2012/1/13 > > >>>> > > >>>> you need to be aware that doing the DNS queries is rather > > expensive > > >>>> > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > version > > >>>>> there > > >>>>> may now be a DNS cache that will drastically help) > > >>>>> > > >>>>> you would need to create a template with FROMHOST in it and use > > that as > > >>>>> the filename to write to (look for dynafile in the documentation) > > >>>>> > > >>>>> note that if you are relaying logs from one machine to another, > > only > > >>>>> the > > >>>>> first machine will see the true source in FROMHOST, machines > > after that > > >>>>> will only see the relay box. > > >>>>> > > >>>>> let me know if this doesn't give you enough clues to learn how to > > do > > >>>>> this. > > >>>>> > > >>>>> David Lang > > >>>>> > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > >>>>> > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > >>>>> > > >>>>> From: Michael Maymann > > >>>>>> Reply-To: rsyslog-users > > >>>>>> To: rsyslog at lists.adiscon.com > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>>>> howto/links/examples > > >>>>>> > > >>>>>> > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS and > > not > > >>>>>> from > > >>>>>> syslog-info hostname. > > >>>>>> We are getting a lot of weird logfiles as some applications are > > not > > >>>>>> including the hostname as the first parameter in the syslog- > > entries, > > >>>>>> e.g.: > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > >>>>>> [set_host_compat_list] > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData > > >>>>>> HostCompatList > > >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > >>>>>> > > >>>>>> Thanks in advance :-) ! > > >>>>>> ~maymann > > >>>>>> > > >>>>>> > > >>>>>> 2012/1/13 Michael Maymann > > >>>>>> > > >>>>>> Hi List, > > >>>>>> > > >>>>>> > > >>>>>>> I'm new to rsyslog/syslog in general. > > >>>>>>> > > >>>>>>> I would like to syslog from all my 100+ network devices. > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > FQDN-dir > > >>>>>>> containing logs from this host if more logfiles per host are > > best > > >>>>>>> practice)... > > >>>>>>> > > >>>>>>> Can anyone give me an example of (or link to) best practice of > > this > > >>>>>>> kind > > >>>>>>> of setup. > > >>>>>>> > > >>>>>>> > > >>>>>>> Thanks in advance :-) ! > > >>>>>>> > > >>>>>>> ~maymann > > >>>>>>> > > >>>>>>> ______________________________******_________________ > > >>>>>>> > > >>>>>>> rsyslog mailing list > > >>>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > **services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> ______________________________******_________________ > > >>>>>> > > >>>>>> rsyslog mailing list > > >>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > **services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Mon Jan 16 11:10:34 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 11:10:34 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > Sent: Monday, January 16, 2012 10:48 AM > To: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi Rainer, > > Thanks for your reply. > How do I install 6.3.6 on RHEL6 easiest ? I have no specific instructions. Just grab the sources and compile, I'd say ;) Note, however, that you need to install libestr and probably libee first. Rainer > > Thanks in advance :-)! > ~maymann > > 2012/1/16 Rainer Gerhards > > > The cache is available since 6.3.1, so you need to go for the devel > > version. > > A good place to check those things is the ChangeLog itself, here is > > the current one: > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 > > 575d085 > > a0fcf48f71339154813971 > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> > > ;hb=HEAD > > > > Note that v6-devel is almost as stable as v6-stable except for the > > config read phase at startup. > > > > HTH > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > > Sent: Monday, January 16, 2012 8:57 AM > > > To: david at lang.hm > > > Cc: rsyslog-users > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > howto/links/examples > > > > > > If I want DNS caching, should i use the new stable-6.2.0 or the > > > older devel-6.3.6... ? > > > Can see this feature mentioned here: > > > http://rsyslog.com/features/ > > > or here: > > > http://rsyslog.com/project-status/ > > > > > > > > > Thanks in advance :-) ! > > > ~maymann > > > > > > 2012/1/14 > > > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > > > Hi David, > > > >> > > > >> thanks for this...this is super info...:-) ! > > > >> If I have to create different logs per host, will this be the a > > > valid > > > >> configuration: > > > >> $template > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > > >> *.* -?DynaFile_messages > > > >> $template > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > > >> *.* -?DynaFile_secure > > > >> $template > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > > >> *.* -?DynaFile_auth.log > > > >> > > > > > > > > I believe so. > > > > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > > have to > > > >> create every hosts dir upfront... ? > > > >> > > > > > > > > it will create it for you (make sure it's running with the > > > appropriate > > > > permissions, if you have rsyslog configured to drop privileges, > > > > the > > > lower > > > > privileges need the ability to create the directories) > > > > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > > somewhere > > > >> first...? > > > >> > > > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > > > David Lang > > > > > > > > > > > >> Thanks in advance :-) ! > > > >> ~maymann > > > >> > > > >> > > > >> 2012/1/14 > > > >> > > > >> http://rsyslog.com/article60/ > > > >>> > > > >>> David Lang > > > >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > > >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > > >>> > > > >>>> From: Michael Maymann > > > >>>> To: rsyslog-users , david at lang.hm, > > > >>>> Michael Maymann > > > >>>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > >>>> howto/links/examples > > > >>>> > > > >>>> Hi David, > > > >>>> > > > >>>> thanks for you kind reply...:-) ! > > > >>>> --- > > > >>>> This didn't seem to get through to the archives for some > > > reason...: > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > > >>>> > > > January/thread.html > > 2- > > > **January/thread.html> > > > >>>> > > January/thread.** > > > >>>> html > > January/thread.html> > > > >>>> > > > > >>>> > > > >>>> Hope I will not dobbel-post... > > > >>>> --- > > > >>>> I don't use syslog-relays, so this will not cause me any problems. > > > >>>> Don't actually know what version we are running - can see this > > > Monday > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 > > > >>>> if > > > not > > > >>>> already then. > > > >>>> I have to configure this into a already running live production > > > system - > > > >>>> our previous syslog-admin left...:-(. > > > >>>> Could I perhaps ask you to be so kind as to give an > > > >>>> configuration example of how this is done, if I ask really > > > >>>> nicely... :-) ? > > > >>>> > > > >>>> Thanks in advance :-) ! > > > >>>> ~maymann > > > >>>> > > > >>>> 2012/1/13 > > > >>>> > > > >>>> you need to be aware that doing the DNS queries is rather > > > expensive > > > >>>> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > > version > > > >>>>> there > > > >>>>> may now be a DNS cache that will drastically help) > > > >>>>> > > > >>>>> you would need to create a template with FROMHOST in it and > > > >>>>> use > > > that as > > > >>>>> the filename to write to (look for dynafile in the > > > >>>>> documentation) > > > >>>>> > > > >>>>> note that if you are relaying logs from one machine to > > > >>>>> another, > > > only > > > >>>>> the > > > >>>>> first machine will see the true source in FROMHOST, machines > > > after that > > > >>>>> will only see the relay box. > > > >>>>> > > > >>>>> let me know if this doesn't give you enough clues to learn how > > > >>>>> to > > > do > > > >>>>> this. > > > >>>>> > > > >>>>> David Lang > > > >>>>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > > >>>>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > > >>>>> > > > >>>>> From: Michael Maymann > > > >>>>>> Reply-To: rsyslog-users > > > >>>>>> To: rsyslog at lists.adiscon.com > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > >>>>>> howto/links/examples > > > >>>>>> > > > >>>>>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS > > > >>>>>> and > > > not > > > >>>>>> from > > > >>>>>> syslog-info hostname. > > > >>>>>> We are getting a lot of weird logfiles as some applications > > > >>>>>> are > > > not > > > >>>>>> including the hostname as the first parameter in the syslog- > > > entries, > > > >>>>>> e.g.: > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > > >>>>>> [set_host_compat_list] > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd > "SetUserData > > > >>>>>> HostCompatList text > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > > >>>>>> > > > >>>>>> Thanks in advance :-) ! > > > >>>>>> ~maymann > > > >>>>>> > > > >>>>>> > > > >>>>>> 2012/1/13 Michael Maymann > > > >>>>>> > > > >>>>>> Hi List, > > > >>>>>> > > > >>>>>> > > > >>>>>>> I'm new to rsyslog/syslog in general. > > > >>>>>>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > > FQDN-dir > > > >>>>>>> containing logs from this host if more logfiles per host are > > > best > > > >>>>>>> practice)... > > > >>>>>>> > > > >>>>>>> Can anyone give me an example of (or link to) best practice > > > >>>>>>> of > > > this > > > >>>>>>> kind > > > >>>>>>> of setup. > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> Thanks in advance :-) ! > > > >>>>>>> > > > >>>>>>> ~maymann > > > >>>>>>> > > > >>>>>>> > ______________________________******_________________ > > > >>>>>>> > > > >>>>>>> rsyslog mailing list > > > >>>>>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > >>>>>> > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > >>>>>> > > > > >>>>>> > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > >>>>>> > > > > > adi > > > scon.net/mailman/listinfo/rsyslog> > > > >>>>>> > > > > >>>>>> > > > >>>>>>> > > > >>>>>>> http://www.rsyslog.com/******professional- > > > services/ > > > >>>>>> > > services/ > > > >>>>>> > > > > >>>>>> > > **services/ > > > >>>>>> > > services/ > > > >>>>>> > > > > >>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > ______________________________******_________________ > > > >>>>>> > > > >>>>>> rsyslog mailing list > > > >>>>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > >>>>> > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > >>>>> > > > > >>>>> > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > >>>>> > > > > > adi > > > scon.net/mailman/listinfo/rsyslog> > > > >>>>> > > > > >>>>> > > > >>>>>> > > > >>>>>> http://www.rsyslog.com/******professional- > > > services/ > > > >>>>> > > services/ > > > >>>>> > > > > >>>>> > > **services/ > > > >>>>> > > services/ > > > >>>>> > > > > >>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>>> > > > >>>> > > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From david at lang.hm Mon Jan 16 11:15:58 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 16 Jan 2012 02:15:58 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: I would say that 6.2.0 is probably better. sorry I can't give you more info on this. David Lang On Mon, 16 Jan 2012, Michael Maymann wrote: > If I want DNS caching, should i use the new stable-6.2.0 or the older > devel-6.3.6... ? > Can see this feature mentioned here: > http://rsyslog.com/features/ > or here: > http://rsyslog.com/project-status/ > > > Thanks in advance :-) ! > ~maymann > > 2012/1/14 > >> On Sat, 14 Jan 2012, Michael Maymann wrote: >> >> Hi David, >>> >>> thanks for this...this is super info...:-) ! >>> If I have to create different logs per host, will this be the a valid >>> configuration: >>> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >>> *.* -?DynaFile_messages >>> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >>> *.* -?DynaFile_secure >>> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >>> *.* -?DynaFile_auth.log >>> >> >> I believe so. >> >> >> 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to >>> create every hosts dir upfront... ? >>> >> >> it will create it for you (make sure it's running with the appropriate >> permissions, if you have rsyslog configured to drop privileges, the lower >> privileges need the ability to create the directories) >> >> >> 2. Is DNS caching default enabled or do I have to enable this somewhere >>> first...? >>> >> >> I don't know, I haven't had a chance to look into that yet. >> >> David Lang >> >> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> >>> 2012/1/14 >>> >>> http://rsyslog.com/article60/ >>>> >>>> David Lang >>>> >>>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>> >>>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>>> >>>>> From: Michael Maymann >>>>> To: rsyslog-users , david at lang.hm, >>>>> Michael Maymann >>>>> >>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>> howto/links/examples >>>>> >>>>> Hi David, >>>>> >>>>> thanks for you kind reply...:-) ! >>>>> --- >>>>> This didn't seem to get through to the archives for some reason...: >>>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>>>> January/thread.html >>>>> >>>> html >>>>>> >>>>> >>>>> Hope I will not dobbel-post... >>>>> --- >>>>> I don't use syslog-relays, so this will not cause me any problems. >>>>> Don't actually know what version we are running - can see this Monday >>>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>>> already then. >>>>> I have to configure this into a already running live production system - >>>>> our previous syslog-admin left...:-(. >>>>> Could I perhaps ask you to be so kind as to give an configuration >>>>> example >>>>> of how this is done, if I ask really nicely... :-) ? >>>>> >>>>> Thanks in advance :-) ! >>>>> ~maymann >>>>> >>>>> 2012/1/13 >>>>> >>>>> you need to be aware that doing the DNS queries is rather expensive >>>>> >>>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>>> there >>>>>> may now be a DNS cache that will drastically help) >>>>>> >>>>>> you would need to create a template with FROMHOST in it and use that as >>>>>> the filename to write to (look for dynafile in the documentation) >>>>>> >>>>>> note that if you are relaying logs from one machine to another, only >>>>>> the >>>>>> first machine will see the true source in FROMHOST, machines after that >>>>>> will only see the relay box. >>>>>> >>>>>> let me know if this doesn't give you enough clues to learn how to do >>>>>> this. >>>>>> >>>>>> David Lang >>>>>> >>>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>>> >>>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>>> >>>>>> From: Michael Maymann >>>>>>> Reply-To: rsyslog-users >>>>>>> To: rsyslog at lists.adiscon.com >>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>> howto/links/examples >>>>>>> >>>>>>> >>>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>>> from >>>>>>> syslog-info hostname. >>>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>>> e.g.: >>>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>>> [set_host_compat_list] >>>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>>> HostCompatList >>>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>>>>>> >>>>>>> >>>>>>> >>>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>>> >>>>>>> Thanks in advance :-) ! >>>>>>> ~maymann >>>>>>> >>>>>>> >>>>>>> 2012/1/13 Michael Maymann >>>>>>> >>>>>>> Hi List, >>>>>>> >>>>>>> >>>>>>>> I'm new to rsyslog/syslog in general. >>>>>>>> >>>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>>>>> containing logs from this host if more logfiles per host are best >>>>>>>> practice)... >>>>>>>> >>>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>>> kind >>>>>>>> of setup. >>>>>>>> >>>>>>>> >>>>>>>> Thanks in advance :-) ! >>>>>>>> >>>>>>>> ~maymann >>>>>>>> >>>>>>>> ______________________________******_________________ >>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> http://www.rsyslog.com/******professional-services/ >>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> ______________________________******_________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>>> >>>>>>> >>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>>> http://www.rsyslog.com/******professional-services/ >>>>>> >>>>>>> >>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>> > From rgerhards at hq.adiscon.com Mon Jan 16 16:41:43 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 16:41:43 +0100 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID In-Reply-To: <4F13B68B.5070203@cn.fujitsu.com> References: <4F069932.9070408@cn.fujitsu.com> <4F13B68B.5070203@cn.fujitsu.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281752@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: Peng Haitao [mailto:penght at cn.fujitsu.com] > Sent: Monday, January 16, 2012 6:33 AM > To: Rainer Gerhards > Cc: rsyslog-users > Subject: Re: [rsyslog] [PATCH] fix error of not define HAVE_SETSID > > > Peng Haitao said the following on 2012-1-6 14:48: > > [add list to the CC list] > > > > When HAVE_SETSID is not defined, rsyslogd will use ioctl() make itself > > to daemon, but this can not make rsyslogd process become the process > > group leader of a new process group. In RHEL6.1, the status is as follows: > > # uname -a > > Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May > > 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux # /etc/init.d/rsyslog > restart > > Shutting down system logger: [ OK ] > > Starting system logger: [ OK ] > > # ps axo pgrp,ppid,pid,comm | grep rsyslog > > 6290 1 6301 rsyslogd > > > > When we send SIGTERM signal to 6290, rsyslogd will die:( So I think we > > should call setpgid() before ioctl(). > > > > hi, anyone can comment the patch? Sorry, I am still busy, it's on my todo list and looks good on first impression. Bear with me a little bit more, I hope to be able to go back to real work soon... ;) Rainer > > -- > Best Regards, > Peng > > > Signed-off-by: Peng Haitao > > --- > > tools/syslogd.c | 7 +++++++ > > 1 files changed, 7 insertions(+), 0 deletions(-) > > > > diff --git a/tools/syslogd.c b/tools/syslogd.c index d1224f2..ac954b7 > > 100644 > > --- a/tools/syslogd.c > > +++ b/tools/syslogd.c > > @@ -356,8 +356,15 @@ void untty(void) > > #else > > { > > int i; > > + pid_t pid; > > > > if(!Debug) { > > + pid = getpid(); > > + if (setpgid(pid, pid) < 0) { > > + perror("setpgid"); > > + exit(1); > > + } > > + > > i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); > > if (i >= 0) { > > # if !defined(__hpux) > From rgerhards at hq.adiscon.com Mon Jan 16 17:15:08 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 17:15:08 +0100 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281752@GRFEXC.intern.adiscon.com> References: <4F069932.9070408@cn.fujitsu.com> <4F13B68B.5070203@cn.fujitsu.com> <9B6E2A8877C38245BFB15CC491A11DA7281752@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281754@GRFEXC.intern.adiscon.com> I should have waited with my reply ;) I am currently merging things and it seems to go rather smooth. So I also merged your patch: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=167192666ba8905b83210ab7 c5f00cc6be9f7147 It goes into v5-devel. Thanks! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, January 16, 2012 4:42 PM > To: Peng Haitao > Cc: rsyslog-users > Subject: Re: [rsyslog] [PATCH] fix error of not define HAVE_SETSID > > > > > -----Original Message----- > > From: Peng Haitao [mailto:penght at cn.fujitsu.com] > > Sent: Monday, January 16, 2012 6:33 AM > > To: Rainer Gerhards > > Cc: rsyslog-users > > Subject: Re: [rsyslog] [PATCH] fix error of not define HAVE_SETSID > > > > > > Peng Haitao said the following on 2012-1-6 14:48: > > > [add list to the CC list] > > > > > > When HAVE_SETSID is not defined, rsyslogd will use ioctl() make > > > itself to daemon, but this can not make rsyslogd process become the > > > process group leader of a new process group. In RHEL6.1, the status > > > is as > follows: > > > # uname -a > > > Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue > > > May > > > 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux # > > > /etc/init.d/rsyslog > > restart > > > Shutting down system logger: [ OK ] > > > Starting system logger: [ OK ] > > > # ps axo pgrp,ppid,pid,comm | grep rsyslog > > > 6290 1 6301 rsyslogd > > > > > > When we send SIGTERM signal to 6290, rsyslogd will die:( So I think > > > we should call setpgid() before ioctl(). > > > > > > > hi, anyone can comment the patch? > > Sorry, I am still busy, it's on my todo list and looks good on first impression. > Bear with me a little bit more, I hope to be able to go back to real work > soon... ;) > > Rainer > > > > -- > > Best Regards, > > Peng > > > > > Signed-off-by: Peng Haitao > > > --- > > > tools/syslogd.c | 7 +++++++ > > > 1 files changed, 7 insertions(+), 0 deletions(-) > > > > > > diff --git a/tools/syslogd.c b/tools/syslogd.c index > > > d1224f2..ac954b7 > > > 100644 > > > --- a/tools/syslogd.c > > > +++ b/tools/syslogd.c > > > @@ -356,8 +356,15 @@ void untty(void) #else { > > > int i; > > > + pid_t pid; > > > > > > if(!Debug) { > > > + pid = getpid(); > > > + if (setpgid(pid, pid) < 0) { > > > + perror("setpgid"); > > > + exit(1); > > > + } > > > + > > > i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); > > > if (i >= 0) { > > > # if !defined(__hpux) > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Mon Jan 16 20:07:44 2012 From: michael at maymann.org (Michael Maymann) Date: Mon, 16 Jan 2012 20:07:44 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi, 6.2.0 doesn't have DNS cache, does it... I would like to use stable, but would really need the DNS caching feature - so would have to do devel (especially if this is nearly as stable - as Rainer describes): http://lists.adiscon.net/pipermail/rsyslog/2012-January/014244.html Thanks in advance :-) ! ~maymann 2012/1/16 > I would say that 6.2.0 is probably better. > > sorry I can't give you more info on this. > > David Lang > > > On Mon, 16 Jan 2012, Michael Maymann wrote: > > If I want DNS caching, should i use the new stable-6.2.0 or the older >> devel-6.3.6... ? >> Can see this feature mentioned here: >> http://rsyslog.com/features/ >> or here: >> http://rsyslog.com/project-**status/ >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/14 >> >> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> >>> Hi David, >>> >>>> >>>> thanks for this...this is super info...:-) ! >>>> If I have to create different logs per host, will this be the a valid >>>> configuration: >>>> $template DynaFile_messages,?/logfiles_****on_nfs/%HOSTNAME%/messages? >>>> >>>> *.* -?DynaFile_messages >>>> $template DynaFile_secure,?/logfiles_on_****nfs/%HOSTNAME%/secure? >>>> *.* -?DynaFile_secure >>>> $template DynaFile_auth.log,?/logfiles_****on_nfs/%HOSTNAME%/auth.log? >>>> *.* -?DynaFile_auth.log >>>> >>>> >>> I believe so. >>> >>> >>> 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have >>> to >>> >>>> create every hosts dir upfront... ? >>>> >>>> >>> it will create it for you (make sure it's running with the appropriate >>> permissions, if you have rsyslog configured to drop privileges, the lower >>> privileges need the ability to create the directories) >>> >>> >>> 2. Is DNS caching default enabled or do I have to enable this somewhere >>> >>>> first...? >>>> >>>> >>> I don't know, I haven't had a chance to look into that yet. >>> >>> David Lang >>> >>> >>> Thanks in advance :-) ! >>>> ~maymann >>>> >>>> >>>> 2012/1/14 >>>> >>>> http://rsyslog.com/article60/ >>>> >>>>> >>>>> David Lang >>>>> >>>>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>>> >>>>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>>>> >>>>> From: Michael Maymann >>>>>> To: rsyslog-users , david at lang.hm, >>>>>> Michael Maymann >>>>>> >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>> howto/links/examples >>>>>> >>>>>> Hi David, >>>>>> >>>>>> thanks for you kind reply...:-) ! >>>>>> --- >>>>>> This didn't seem to get through to the archives for some reason...: >>>>>> http://lists.adiscon.net/******pipermail/rsyslog/2012-**** >>>>>> January/thread.html>>>>> rsyslog/2012-**January/thread.**html >>>>>> > >>>>>> >>>>> January/thread.** >>>>>> >>>>>> html>>>>> January/thread.html >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>> Hope I will not dobbel-post... >>>>>> --- >>>>>> I don't use syslog-relays, so this will not cause me any problems. >>>>>> Don't actually know what version we are running - can see this Monday >>>>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>>>> already then. >>>>>> I have to configure this into a already running live production >>>>>> system - >>>>>> our previous syslog-admin left...:-(. >>>>>> Could I perhaps ask you to be so kind as to give an configuration >>>>>> example >>>>>> of how this is done, if I ask really nicely... :-) ? >>>>>> >>>>>> Thanks in advance :-) ! >>>>>> ~maymann >>>>>> >>>>>> 2012/1/13 >>>>>> >>>>>> you need to be aware that doing the DNS queries is rather expensive >>>>>> >>>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>>>> there >>>>>>> may now be a DNS cache that will drastically help) >>>>>>> >>>>>>> you would need to create a template with FROMHOST in it and use that >>>>>>> as >>>>>>> the filename to write to (look for dynafile in the documentation) >>>>>>> >>>>>>> note that if you are relaying logs from one machine to another, only >>>>>>> the >>>>>>> first machine will see the true source in FROMHOST, machines after >>>>>>> that >>>>>>> will only see the relay box. >>>>>>> >>>>>>> let me know if this doesn't give you enough clues to learn how to do >>>>>>> this. >>>>>>> >>>>>>> David Lang >>>>>>> >>>>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>>>> >>>>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>>>> >>>>>>> From: Michael Maymann >>>>>>> >>>>>>>> Reply-To: rsyslog-users >>>>>>>> To: rsyslog at lists.adiscon.com >>>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>>> howto/links/examples >>>>>>>> >>>>>>>> >>>>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>>>> from >>>>>>>> syslog-info hostname. >>>>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>>>> e.g.: >>>>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>>>> [set_host_compat_list] >>>>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>>>> HostCompatList >>>>>>>> text /tmp/hostCompatList"'#012#********01212/16/11 09:47:40 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>>>> >>>>>>>> Thanks in advance :-) ! >>>>>>>> ~maymann >>>>>>>> >>>>>>>> >>>>>>>> 2012/1/13 Michael Maymann >>>>>>>> >>>>>>>> Hi List, >>>>>>>> >>>>>>>> >>>>>>>> I'm new to rsyslog/syslog in general. >>>>>>>>> >>>>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>>>> Preferably I would like a FQDN.log file for each host (or a >>>>>>>>> FQDN-dir >>>>>>>>> containing logs from this host if more logfiles per host are best >>>>>>>>> practice)... >>>>>>>>> >>>>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>>>> kind >>>>>>>>> of setup. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks in advance :-) ! >>>>>>>>> >>>>>>>>> ~maymann >>>>>>>>> >>>>>>>>> ______________________________********_________________ >>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> >>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>>> >>>>>>>>> > >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> ______________________________********_________________ >>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>>> >>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >> From david at lang.hm Mon Jan 16 22:37:51 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 16 Jan 2012 13:37:51 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Rainer knows this stuff far better than I do, so go with what he says. David Lang On Mon, 16 Jan 2012, Michael Maymann wrote: > Date: Mon, 16 Jan 2012 20:07:44 +0100 > From: Michael Maymann > To: david at lang.hm > Cc: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi, > > 6.2.0 doesn't have DNS cache, does it... > I would like to use stable, but would really need the DNS caching feature - > so would have to do devel (especially if this is nearly as stable - as > Rainer describes): > http://lists.adiscon.net/pipermail/rsyslog/2012-January/014244.html > > Thanks in advance :-) ! > ~maymann > > 2012/1/16 > >> I would say that 6.2.0 is probably better. >> >> sorry I can't give you more info on this. >> >> David Lang >> >> >> On Mon, 16 Jan 2012, Michael Maymann wrote: >> >> If I want DNS caching, should i use the new stable-6.2.0 or the older >>> devel-6.3.6... ? >>> Can see this feature mentioned here: >>> http://rsyslog.com/features/ >>> or here: >>> http://rsyslog.com/project-**status/ >>> >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> 2012/1/14 >>> >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>> >>>> Hi David, >>>> >>>>> >>>>> thanks for this...this is super info...:-) ! >>>>> If I have to create different logs per host, will this be the a valid >>>>> configuration: >>>>> $template DynaFile_messages,?/logfiles_****on_nfs/%HOSTNAME%/messages? >>>>> >>>>> *.* -?DynaFile_messages >>>>> $template DynaFile_secure,?/logfiles_on_****nfs/%HOSTNAME%/secure? >>>>> *.* -?DynaFile_secure >>>>> $template DynaFile_auth.log,?/logfiles_****on_nfs/%HOSTNAME%/auth.log? >>>>> *.* -?DynaFile_auth.log >>>>> >>>>> >>>> I believe so. >>>> >>>> >>>> 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have >>>> to >>>> >>>>> create every hosts dir upfront... ? >>>>> >>>>> >>>> it will create it for you (make sure it's running with the appropriate >>>> permissions, if you have rsyslog configured to drop privileges, the lower >>>> privileges need the ability to create the directories) >>>> >>>> >>>> 2. Is DNS caching default enabled or do I have to enable this somewhere >>>> >>>>> first...? >>>>> >>>>> >>>> I don't know, I haven't had a chance to look into that yet. >>>> >>>> David Lang >>>> >>>> >>>> Thanks in advance :-) ! >>>>> ~maymann >>>>> >>>>> >>>>> 2012/1/14 >>>>> >>>>> http://rsyslog.com/article60/ >>>>> >>>>>> >>>>>> David Lang >>>>>> >>>>>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>>>> >>>>>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>>>>> >>>>>> From: Michael Maymann >>>>>>> To: rsyslog-users , david at lang.hm, >>>>>>> Michael Maymann >>>>>>> >>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>> howto/links/examples >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> thanks for you kind reply...:-) ! >>>>>>> --- >>>>>>> This didn't seem to get through to the archives for some reason...: >>>>>>> http://lists.adiscon.net/******pipermail/rsyslog/2012-**** >>>>>>> January/thread.html>>>>>> rsyslog/2012-**January/thread.**html >>>>>>>> >>>>>>> >>>>>> January/thread.** >>>>>>> >>>>>>> html>>>>>> January/thread.html >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Hope I will not dobbel-post... >>>>>>> --- >>>>>>> I don't use syslog-relays, so this will not cause me any problems. >>>>>>> Don't actually know what version we are running - can see this Monday >>>>>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>>>>> already then. >>>>>>> I have to configure this into a already running live production >>>>>>> system - >>>>>>> our previous syslog-admin left...:-(. >>>>>>> Could I perhaps ask you to be so kind as to give an configuration >>>>>>> example >>>>>>> of how this is done, if I ask really nicely... :-) ? >>>>>>> >>>>>>> Thanks in advance :-) ! >>>>>>> ~maymann >>>>>>> >>>>>>> 2012/1/13 >>>>>>> >>>>>>> you need to be aware that doing the DNS queries is rather expensive >>>>>>> >>>>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>>>>> there >>>>>>>> may now be a DNS cache that will drastically help) >>>>>>>> >>>>>>>> you would need to create a template with FROMHOST in it and use that >>>>>>>> as >>>>>>>> the filename to write to (look for dynafile in the documentation) >>>>>>>> >>>>>>>> note that if you are relaying logs from one machine to another, only >>>>>>>> the >>>>>>>> first machine will see the true source in FROMHOST, machines after >>>>>>>> that >>>>>>>> will only see the relay box. >>>>>>>> >>>>>>>> let me know if this doesn't give you enough clues to learn how to do >>>>>>>> this. >>>>>>>> >>>>>>>> David Lang >>>>>>>> >>>>>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>>>>> >>>>>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>>>>> >>>>>>>> From: Michael Maymann >>>>>>>> >>>>>>>>> Reply-To: rsyslog-users >>>>>>>>> To: rsyslog at lists.adiscon.com >>>>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>>>> howto/links/examples >>>>>>>>> >>>>>>>>> >>>>>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>>>>> from >>>>>>>>> syslog-info hostname. >>>>>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>>>>> e.g.: >>>>>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>>>>> [set_host_compat_list] >>>>>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>>>>> HostCompatList >>>>>>>>> text /tmp/hostCompatList"'#012#********01212/16/11 09:47:40 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>>>>> >>>>>>>>> Thanks in advance :-) ! >>>>>>>>> ~maymann >>>>>>>>> >>>>>>>>> >>>>>>>>> 2012/1/13 Michael Maymann >>>>>>>>> >>>>>>>>> Hi List, >>>>>>>>> >>>>>>>>> >>>>>>>>> I'm new to rsyslog/syslog in general. >>>>>>>>>> >>>>>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>>>>> Preferably I would like a FQDN.log file for each host (or a >>>>>>>>>> FQDN-dir >>>>>>>>>> containing logs from this host if more logfiles per host are best >>>>>>>>>> practice)... >>>>>>>>>> >>>>>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>>>>> kind >>>>>>>>>> of setup. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks in advance :-) ! >>>>>>>>>> >>>>>>>>>> ~maymann >>>>>>>>>> >>>>>>>>>> ______________________________********_________________ >>>>>>>>>> >>>>>>>>>> rsyslog mailing list >>>>>>>>>> >>>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> ______________________________********_________________ >>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> >>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > From michael at maymann.org Tue Jan 17 08:33:39 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 17 Jan 2012 08:33:39 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, do I have to enable DNS cache somewhere, or is this feature on by default ? Thanks in advance :-) ! ~maymann 2012/1/16 Rainer Gerhards > The cache is available since 6.3.1, so you need to go for the devel > version. > A good place to check those things is the ChangeLog itself, here is the > current one: > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8575d085 > a0fcf48f71339154813971 > ;hb=HEAD > > Note that v6-devel is almost as stable as v6-stable except for the config > read phase at startup. > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Monday, January 16, 2012 8:57 AM > > To: david at lang.hm > > Cc: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > howto/links/examples > > > > If I want DNS caching, should i use the new stable-6.2.0 or the older > > devel-6.3.6... ? > > Can see this feature mentioned here: > > http://rsyslog.com/features/ > > or here: > > http://rsyslog.com/project-status/ > > > > > > Thanks in advance :-) ! > > ~maymann > > > > 2012/1/14 > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > Hi David, > > >> > > >> thanks for this...this is super info...:-) ! > > >> If I have to create different logs per host, will this be the a > > valid > > >> configuration: > > >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > >> *.* -?DynaFile_messages > > >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > >> *.* -?DynaFile_secure > > >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > >> *.* -?DynaFile_auth.log > > >> > > > > > > I believe so. > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > have to > > >> create every hosts dir upfront... ? > > >> > > > > > > it will create it for you (make sure it's running with the > > appropriate > > > permissions, if you have rsyslog configured to drop privileges, the > > lower > > > privileges need the ability to create the directories) > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > somewhere > > >> first...? > > >> > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > David Lang > > > > > > > > >> Thanks in advance :-) ! > > >> ~maymann > > >> > > >> > > >> 2012/1/14 > > >> > > >> http://rsyslog.com/article60/ > > >>> > > >>> David Lang > > >>> > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > >>> > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > >>> > > >>>> From: Michael Maymann > > >>>> To: rsyslog-users , david at lang.hm, > > >>>> Michael Maymann > > >>>> > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>> howto/links/examples > > >>>> > > >>>> Hi David, > > >>>> > > >>>> thanks for you kind reply...:-) ! > > >>>> --- > > >>>> This didn't seem to get through to the archives for some > > reason...: > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > >>>> > > January/thread.html > **January/thread.html> > > >>>> > January/thread.** > > >>>> html > January/thread.html> > > >>>> > > > >>>> > > >>>> Hope I will not dobbel-post... > > >>>> --- > > >>>> I don't use syslog-relays, so this will not cause me any problems. > > >>>> Don't actually know what version we are running - can see this > > Monday > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 if > > not > > >>>> already then. > > >>>> I have to configure this into a already running live production > > system - > > >>>> our previous syslog-admin left...:-(. > > >>>> Could I perhaps ask you to be so kind as to give an configuration > > >>>> example > > >>>> of how this is done, if I ask really nicely... :-) ? > > >>>> > > >>>> Thanks in advance :-) ! > > >>>> ~maymann > > >>>> > > >>>> 2012/1/13 > > >>>> > > >>>> you need to be aware that doing the DNS queries is rather > > expensive > > >>>> > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > version > > >>>>> there > > >>>>> may now be a DNS cache that will drastically help) > > >>>>> > > >>>>> you would need to create a template with FROMHOST in it and use > > that as > > >>>>> the filename to write to (look for dynafile in the documentation) > > >>>>> > > >>>>> note that if you are relaying logs from one machine to another, > > only > > >>>>> the > > >>>>> first machine will see the true source in FROMHOST, machines > > after that > > >>>>> will only see the relay box. > > >>>>> > > >>>>> let me know if this doesn't give you enough clues to learn how to > > do > > >>>>> this. > > >>>>> > > >>>>> David Lang > > >>>>> > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > >>>>> > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > >>>>> > > >>>>> From: Michael Maymann > > >>>>>> Reply-To: rsyslog-users > > >>>>>> To: rsyslog at lists.adiscon.com > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>>>> howto/links/examples > > >>>>>> > > >>>>>> > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS and > > not > > >>>>>> from > > >>>>>> syslog-info hostname. > > >>>>>> We are getting a lot of weird logfiles as some applications are > > not > > >>>>>> including the hostname as the first parameter in the syslog- > > entries, > > >>>>>> e.g.: > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > >>>>>> [set_host_compat_list] > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData > > >>>>>> HostCompatList > > >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > >>>>>> > > >>>>>> Thanks in advance :-) ! > > >>>>>> ~maymann > > >>>>>> > > >>>>>> > > >>>>>> 2012/1/13 Michael Maymann > > >>>>>> > > >>>>>> Hi List, > > >>>>>> > > >>>>>> > > >>>>>>> I'm new to rsyslog/syslog in general. > > >>>>>>> > > >>>>>>> I would like to syslog from all my 100+ network devices. > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > FQDN-dir > > >>>>>>> containing logs from this host if more logfiles per host are > > best > > >>>>>>> practice)... > > >>>>>>> > > >>>>>>> Can anyone give me an example of (or link to) best practice of > > this > > >>>>>>> kind > > >>>>>>> of setup. > > >>>>>>> > > >>>>>>> > > >>>>>>> Thanks in advance :-) ! > > >>>>>>> > > >>>>>>> ~maymann > > >>>>>>> > > >>>>>>> ______________________________******_________________ > > >>>>>>> > > >>>>>>> rsyslog mailing list > > >>>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > **services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> ______________________________******_________________ > > >>>>>> > > >>>>>> rsyslog mailing list > > >>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > **services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From michael at maymann.org Tue Jan 17 10:36:46 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 17 Jan 2012 10:36:46 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, yes, rsyslog-6.3.6 needs libestr and libee...: I installed libestr just fine: [root at MyServer pkgconfig]# locate libestr /root/libestr-0.1.2 /root/libestr-0.1.2.tar.gz /root/libestr-0.1.2/AUTHORS /root/libestr-0.1.2/COPYING /root/libestr-0.1.2/ChangeLog /root/libestr-0.1.2/INSTALL /root/libestr-0.1.2/Makefile /root/libestr-0.1.2/Makefile.am /root/libestr-0.1.2/Makefile.in /root/libestr-0.1.2/NEWS /root/libestr-0.1.2/README /root/libestr-0.1.2/aclocal.m4 /root/libestr-0.1.2/compile /root/libestr-0.1.2/config.guess /root/libestr-0.1.2/config.h /root/libestr-0.1.2/config.h.in /root/libestr-0.1.2/config.log /root/libestr-0.1.2/config.status /root/libestr-0.1.2/config.sub /root/libestr-0.1.2/configure /root/libestr-0.1.2/configure.ac /root/libestr-0.1.2/depcomp /root/libestr-0.1.2/include /root/libestr-0.1.2/install-sh /root/libestr-0.1.2/libestr.pc /root/libestr-0.1.2/libestr.pc.in /root/libestr-0.1.2/libtool /root/libestr-0.1.2/ltmain.sh /root/libestr-0.1.2/m4 /root/libestr-0.1.2/missing /root/libestr-0.1.2/src /root/libestr-0.1.2/stamp-h1 /root/libestr-0.1.2/include/Makefile /root/libestr-0.1.2/include/Makefile.am /root/libestr-0.1.2/include/Makefile.in /root/libestr-0.1.2/include/libestr.h /root/libestr-0.1.2/m4/libtool.m4 /root/libestr-0.1.2/m4/ltoptions.m4 /root/libestr-0.1.2/m4/ltsugar.m4 /root/libestr-0.1.2/m4/ltversion.m4 /root/libestr-0.1.2/m4/lt~obsolete.m4 /root/libestr-0.1.2/src/.deps /root/libestr-0.1.2/src/.libs /root/libestr-0.1.2/src/Makefile /root/libestr-0.1.2/src/Makefile.am /root/libestr-0.1.2/src/Makefile.in /root/libestr-0.1.2/src/libestr.c /root/libestr-0.1.2/src/libestr.la /root/libestr-0.1.2/src/libestr_la-libestr.lo /root/libestr-0.1.2/src/libestr_la-libestr.o /root/libestr-0.1.2/src/libestr_la-string.lo /root/libestr-0.1.2/src/libestr_la-string.o /root/libestr-0.1.2/src/string.c /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo /root/libestr-0.1.2/src/.libs/libestr.a /root/libestr-0.1.2/src/.libs/libestr.la /root/libestr-0.1.2/src/.libs/libestr.lai /root/libestr-0.1.2/src/.libs/libestr.so /root/libestr-0.1.2/src/.libs/libestr.so.0 /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o /root/libestr-0.1.2/src/.libs/libestr_la-string.o /usr/local/include/libestr.h /usr/local/lib/libestr.a /usr/local/lib/libestr.la /usr/local/lib/libestr.so /usr/local/lib/libestr.so.0 /usr/local/lib/libestr.so.0.0.0 /usr/local/lib/pkgconfig/libestr.pc I try to install libee: [root at MyServer libee-0.3.2]# ./configure LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking whether gcc and cc understand -c and -o together... yes checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking how to print strings... printf checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1966080 checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... yes checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for ar... ar checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... no checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for LIBESTR... yes configure: creating ./config.status config.status: creating Makefile config.status: creating libee.pc config.status: creating src/Makefile config.status: creating include/Makefile config.status: creating include/libee/Makefile config.status: creating tests/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands ***************************************************** libee will be compiled with the following settings: Debug mode enabled: no Testbench enabled: yes [root at MyServer libee-0.3.2]# echo $? 0 [root at MyServer libee-0.3.2]# make make all-recursive make[1]: Entering directory `/root/libee-0.3.2' Making all in tests make[2]: Entering directory `/root/libee-0.3.2/tests' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/libee-0.3.2/tests' Making all in include make[2]: Entering directory `/root/libee-0.3.2/include' Making all in libee make[3]: Entering directory `/root/libee-0.3.2/include/libee' make[3]: Nothing to be done for `all'. make[3]: Leaving directory `/root/libee-0.3.2/include/libee' make[3]: Entering directory `/root/libee-0.3.2/include' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/root/libee-0.3.2/include' make[2]: Leaving directory `/root/libee-0.3.2/include' Making all in src make[2]: Entering directory `/root/libee-0.3.2/src' CC libee_la-ctx.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-tag.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-event.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-value.lo value.c: In function 'ee_newValue': value.c:37: warning: unused parameter 'ctx' gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-tagbucket.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-field.lo field.c: In function 'ee_getFieldValueAsStr': field.c:181: warning: 'str' may be used uninitialized in this function gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-fieldbucket.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-primitivetype.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-int_dec.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-apache_dec.lo apache_dec.c: In function 'ee_newApache': apache_dec.c:37: warning: unused parameter 'ctx' apache_dec.c: In function 'ee_apacheAddName': apache_dec.c:71: warning: unused parameter 'ctx' apache_dec.c: In function 'processLn': apache_dec.c:205: warning: unused variable 'value' gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-syslog_enc.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-json_enc.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-csv_enc.lo csv_enc.c: In function 'ee_AddName': csv_enc.c:66: warning: unused parameter 'ctx' gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-xml_enc.lo xml_enc.c: In function 'ee_addValue_XML': xml_enc.c:60: warning: unused variable 'j' xml_enc.c:59: warning: unused variable 'numbuf' xml_enc.c: At top level: xml_enc.c:40: warning: 'hexdigit' defined but not used gcc: /usr/local/include: linker input file unused because linking not done CCLD libee.la CC convert-convert.o gcc: /usr/local/include: linker input file unused because linking not done CCLD convert /usr/local/lib: file not recognized: Is a directory collect2: ld returned 1 exit status make[2]: *** [convert] Error 1 make[2]: Leaving directory `/root/libee-0.3.2/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/libee-0.3.2' make: *** [all] Error 2 [root at MyServer libee-0.3.2]# echo $? 2 It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know how to solve this ? Thanks in advance :-) ! ~maymann 2012/1/16 Rainer Gerhards > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Monday, January 16, 2012 10:48 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > howto/links/examples > > > > Hi Rainer, > > > > Thanks for your reply. > > How do I install 6.3.6 on RHEL6 easiest ? > > I have no specific instructions. Just grab the sources and compile, I'd say > ;) Note, however, that you need to install libestr and probably libee > first. > > Rainer > > > > Thanks in advance :-)! > > ~maymann > > > > 2012/1/16 Rainer Gerhards > > > > > The cache is available since 6.3.1, so you need to go for the devel > > > version. > > > A good place to check those things is the ChangeLog itself, here is > > > the current one: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 > > > 575d085 > > > a0fcf48f71339154813971 > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> > > > ;hb=HEAD > > > > > > Note that v6-devel is almost as stable as v6-stable except for the > > > config read phase at startup. > > > > > > HTH > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > > > Sent: Monday, January 16, 2012 8:57 AM > > > > To: david at lang.hm > > > > Cc: rsyslog-users > > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > > howto/links/examples > > > > > > > > If I want DNS caching, should i use the new stable-6.2.0 or the > > > > older devel-6.3.6... ? > > > > Can see this feature mentioned here: > > > > http://rsyslog.com/features/ > > > > or here: > > > > http://rsyslog.com/project-status/ > > > > > > > > > > > > Thanks in advance :-) ! > > > > ~maymann > > > > > > > > 2012/1/14 > > > > > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > > > > > Hi David, > > > > >> > > > > >> thanks for this...this is super info...:-) ! > > > > >> If I have to create different logs per host, will this be the a > > > > valid > > > > >> configuration: > > > > >> $template > > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > > > >> *.* -?DynaFile_messages > > > > >> $template > > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > > > >> *.* -?DynaFile_secure > > > > >> $template > > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > > > >> *.* -?DynaFile_auth.log > > > > >> > > > > > > > > > > I believe so. > > > > > > > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > > > have to > > > > >> create every hosts dir upfront... ? > > > > >> > > > > > > > > > > it will create it for you (make sure it's running with the > > > > appropriate > > > > > permissions, if you have rsyslog configured to drop privileges, > > > > > the > > > > lower > > > > > privileges need the ability to create the directories) > > > > > > > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > > > somewhere > > > > >> first...? > > > > >> > > > > > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > > > > > David Lang > > > > > > > > > > > > > > >> Thanks in advance :-) ! > > > > >> ~maymann > > > > >> > > > > >> > > > > >> 2012/1/14 > > > > >> > > > > >> http://rsyslog.com/article60/ > > > > >>> > > > > >>> David Lang > > > > >>> > > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > >>> > > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > > > >>> > > > > >>>> From: Michael Maymann > > > > >>>> To: rsyslog-users , david at lang.hm, > > > > >>>> Michael Maymann > > > > >>>> > > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > > >>>> howto/links/examples > > > > >>>> > > > > >>>> Hi David, > > > > >>>> > > > > >>>> thanks for you kind reply...:-) ! > > > > >>>> --- > > > > >>>> This didn't seem to get through to the archives for some > > > > reason...: > > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > > > >>>> > > > > January/thread.html > > > 2- > > > > **January/thread.html> > > > > >>>> > > > January/thread.** > > > > >>>> html > > > January/thread.html> > > > > >>>> > > > > > >>>> > > > > >>>> Hope I will not dobbel-post... > > > > >>>> --- > > > > >>>> I don't use syslog-relays, so this will not cause me any > problems. > > > > >>>> Don't actually know what version we are running - can see this > > > > Monday > > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 > > > > >>>> if > > > > not > > > > >>>> already then. > > > > >>>> I have to configure this into a already running live production > > > > system - > > > > >>>> our previous syslog-admin left...:-(. > > > > >>>> Could I perhaps ask you to be so kind as to give an > > > > >>>> configuration example of how this is done, if I ask really > > > > >>>> nicely... :-) ? > > > > >>>> > > > > >>>> Thanks in advance :-) ! > > > > >>>> ~maymann > > > > >>>> > > > > >>>> 2012/1/13 > > > > >>>> > > > > >>>> you need to be aware that doing the DNS queries is rather > > > > expensive > > > > >>>> > > > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > > > version > > > > >>>>> there > > > > >>>>> may now be a DNS cache that will drastically help) > > > > >>>>> > > > > >>>>> you would need to create a template with FROMHOST in it and > > > > >>>>> use > > > > that as > > > > >>>>> the filename to write to (look for dynafile in the > > > > >>>>> documentation) > > > > >>>>> > > > > >>>>> note that if you are relaying logs from one machine to > > > > >>>>> another, > > > > only > > > > >>>>> the > > > > >>>>> first machine will see the true source in FROMHOST, machines > > > > after that > > > > >>>>> will only see the relay box. > > > > >>>>> > > > > >>>>> let me know if this doesn't give you enough clues to learn how > > > > >>>>> to > > > > do > > > > >>>>> this. > > > > >>>>> > > > > >>>>> David Lang > > > > >>>>> > > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > > > >>>>> > > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > > > >>>>> > > > > >>>>> From: Michael Maymann > > > > >>>>>> Reply-To: rsyslog-users > > > > >>>>>> To: rsyslog at lists.adiscon.com > > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > > >>>>>> howto/links/examples > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS > > > > >>>>>> and > > > > not > > > > >>>>>> from > > > > >>>>>> syslog-info hostname. > > > > >>>>>> We are getting a lot of weird logfiles as some applications > > > > >>>>>> are > > > > not > > > > >>>>>> including the hostname as the first parameter in the syslog- > > > > entries, > > > > >>>>>> e.g.: > > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > > > >>>>>> [set_host_compat_list] > > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd > > "SetUserData > > > > >>>>>> HostCompatList text > > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > > > >>>>>> > > > > >>>>>> Thanks in advance :-) ! > > > > >>>>>> ~maymann > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> 2012/1/13 Michael Maymann > > > > >>>>>> > > > > >>>>>> Hi List, > > > > >>>>>> > > > > >>>>>> > > > > >>>>>>> I'm new to rsyslog/syslog in general. > > > > >>>>>>> > > > > >>>>>>> I would like to syslog from all my 100+ network devices. > > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > > > FQDN-dir > > > > >>>>>>> containing logs from this host if more logfiles per host are > > > > best > > > > >>>>>>> practice)... > > > > >>>>>>> > > > > >>>>>>> Can anyone give me an example of (or link to) best practice > > > > >>>>>>> of > > > > this > > > > >>>>>>> kind > > > > >>>>>>> of setup. > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> Thanks in advance :-) ! > > > > >>>>>>> > > > > >>>>>>> ~maymann > > > > >>>>>>> > > > > >>>>>>> > > ______________________________******_________________ > > > > >>>>>>> > > > > >>>>>>> rsyslog mailing list > > > > >>>>>> > > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > > >>>>>> > > > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > > >>>>>> > > > > > >>>>>> > > > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > > >>>>>> > > > > > > > adi > > > > scon.net/mailman/listinfo/rsyslog> > > > > >>>>>> > > > > > >>>>>> > > > > >>>>>>> > > > > >>>>>>> http://www.rsyslog.com/******professional- > > > > services/ > > > > >>>>>> > > > services/ > > > > >>>>>> > > > > > >>>>>> > > > **services/ > > > > >>>>>> > > > services/ > > > > >>>>>> > > > > > >>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>> > > ______________________________******_________________ > > > > >>>>>> > > > > >>>>>> rsyslog mailing list > > > > >>>>> > > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > > >>>>> > > > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > > >>>>> > > > > > >>>>> > > > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > > >>>>> > > > > > > > adi > > > > scon.net/mailman/listinfo/rsyslog> > > > > >>>>> > > > > > >>>>> > > > > >>>>>> > > > > >>>>>> http://www.rsyslog.com/******professional- > > > > services/ > > > > >>>>> > > > services/ > > > > >>>>> > > > > > >>>>> > > > **services/ > > > > >>>>> > > > services/ > > > > >>>>> > > > > > >>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>>> > > > > >>>> > > > > >> > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From michael at maymann.org Tue Jan 17 12:09:19 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 17 Jan 2012 12:09:19 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> Message-ID: Tried to delete all installed and follow this guide: http://www.liblognorm.com/help/first-steps-using-liblognorm/ same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib --includedir=/usr/include: ... checking for LIBESTR... configure: error: Package requirements (libestr >= 0.0.0) were not met: No package 'libestr' found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables LIBESTR_CFLAGS and LIBESTR_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. Thanks in advance :-) ! ~maymann 2012/1/17 Michael Maymann > Hi Rainer, > > yes, rsyslog-6.3.6 needs libestr and libee...: > I installed libestr just fine: > [root at MyServer pkgconfig]# locate libestr > /root/libestr-0.1.2 > /root/libestr-0.1.2.tar.gz > /root/libestr-0.1.2/AUTHORS > /root/libestr-0.1.2/COPYING > /root/libestr-0.1.2/ChangeLog > /root/libestr-0.1.2/INSTALL > /root/libestr-0.1.2/Makefile > /root/libestr-0.1.2/Makefile.am > /root/libestr-0.1.2/Makefile.in > /root/libestr-0.1.2/NEWS > /root/libestr-0.1.2/README > /root/libestr-0.1.2/aclocal.m4 > /root/libestr-0.1.2/compile > /root/libestr-0.1.2/config.guess > /root/libestr-0.1.2/config.h > /root/libestr-0.1.2/config.h.in > /root/libestr-0.1.2/config.log > /root/libestr-0.1.2/config.status > /root/libestr-0.1.2/config.sub > /root/libestr-0.1.2/configure > /root/libestr-0.1.2/configure.ac > /root/libestr-0.1.2/depcomp > /root/libestr-0.1.2/include > /root/libestr-0.1.2/install-sh > /root/libestr-0.1.2/libestr.pc > /root/libestr-0.1.2/libestr.pc.in > /root/libestr-0.1.2/libtool > /root/libestr-0.1.2/ltmain.sh > /root/libestr-0.1.2/m4 > /root/libestr-0.1.2/missing > /root/libestr-0.1.2/src > /root/libestr-0.1.2/stamp-h1 > /root/libestr-0.1.2/include/Makefile > /root/libestr-0.1.2/include/Makefile.am > /root/libestr-0.1.2/include/Makefile.in > /root/libestr-0.1.2/include/libestr.h > /root/libestr-0.1.2/m4/libtool.m4 > /root/libestr-0.1.2/m4/ltoptions.m4 > /root/libestr-0.1.2/m4/ltsugar.m4 > /root/libestr-0.1.2/m4/ltversion.m4 > /root/libestr-0.1.2/m4/lt~obsolete.m4 > /root/libestr-0.1.2/src/.deps > /root/libestr-0.1.2/src/.libs > /root/libestr-0.1.2/src/Makefile > /root/libestr-0.1.2/src/Makefile.am > /root/libestr-0.1.2/src/Makefile.in > /root/libestr-0.1.2/src/libestr.c > /root/libestr-0.1.2/src/libestr.la > /root/libestr-0.1.2/src/libestr_la-libestr.lo > /root/libestr-0.1.2/src/libestr_la-libestr.o > /root/libestr-0.1.2/src/libestr_la-string.lo > /root/libestr-0.1.2/src/libestr_la-string.o > /root/libestr-0.1.2/src/string.c > /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo > /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo > /root/libestr-0.1.2/src/.libs/libestr.a > /root/libestr-0.1.2/src/.libs/libestr.la > /root/libestr-0.1.2/src/.libs/libestr.lai > /root/libestr-0.1.2/src/.libs/libestr.so > /root/libestr-0.1.2/src/.libs/libestr.so.0 > /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 > /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o > /root/libestr-0.1.2/src/.libs/libestr_la-string.o > /usr/local/include/libestr.h > /usr/local/lib/libestr.a > /usr/local/lib/libestr.la > /usr/local/lib/libestr.so > /usr/local/lib/libestr.so.0 > /usr/local/lib/libestr.so.0.0.0 > /usr/local/lib/pkgconfig/libestr.pc > > I try to install libee: > [root at MyServer libee-0.3.2]# ./configure > LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib > checking for a BSD-compatible install... /usr/bin/install -c > checking whether build environment is sane... yes > checking for a thread-safe mkdir -p... /bin/mkdir -p > checking for gawk... gawk > checking whether make sets $(MAKE)... yes > checking for gcc... gcc > checking whether the C compiler works... yes > checking for C compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ISO C89... none needed > checking for style of include used by make... GNU > checking dependency style of gcc... gcc3 > checking whether gcc and cc understand -c and -o together... yes > checking build system type... x86_64-unknown-linux-gnu > checking host system type... x86_64-unknown-linux-gnu > checking how to print strings... printf > checking for a sed that does not truncate output... /bin/sed > checking for grep that handles long lines and -e... /bin/grep > checking for egrep... /bin/grep -E > checking for fgrep... /bin/grep -F > checking for ld used by gcc... /usr/bin/ld > checking if the linker (/usr/bin/ld) is GNU ld... yes > checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B > checking the name lister (/usr/bin/nm -B) interface... BSD nm > checking whether ln -s works... yes > checking the maximum length of command line arguments... 1966080 > checking whether the shell understands some XSI constructs... yes > checking whether the shell understands "+="... yes > checking for /usr/bin/ld option to reload object files... -r > checking for objdump... objdump > checking how to recognize dependent libraries... pass_all > checking for ar... ar > checking for strip... strip > checking for ranlib... ranlib > checking command to parse /usr/bin/nm -B output from gcc object... ok > checking how to run the C preprocessor... gcc -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking for dlfcn.h... yes > checking for objdir... .libs > checking if gcc supports -fno-rtti -fno-exceptions... no > checking for gcc option to produce PIC... -fPIC -DPIC > checking if gcc PIC flag -fPIC -DPIC works... yes > checking if gcc static flag -static works... no > checking if gcc supports -c -o file.o... yes > checking if gcc supports -c -o file.o... (cached) yes > checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports > shared libraries... yes > checking whether -lc should be explicitly linked in... no > checking dynamic linker characteristics... GNU/Linux ld.so > checking how to hardcode library paths into programs... immediate > checking whether stripping libraries is possible... yes > checking if libtool supports shared libraries... yes > checking whether to build shared libraries... yes > checking whether to build static libraries... yes > checking for stdlib.h... (cached) yes > checking for GNU libc compatible malloc... yes > checking for pkg-config... /usr/bin/pkg-config > checking pkg-config is at least version 0.9.0... yes > checking for LIBESTR... yes > configure: creating ./config.status > config.status: creating Makefile > config.status: creating libee.pc > config.status: creating src/Makefile > config.status: creating include/Makefile > config.status: creating include/libee/Makefile > config.status: creating tests/Makefile > config.status: creating config.h > config.status: config.h is unchanged > config.status: executing depfiles commands > config.status: executing libtool commands > ***************************************************** > libee will be compiled with the following settings: > > Debug mode enabled: no > Testbench enabled: yes > [root at MyServer libee-0.3.2]# echo $? > 0 > [root at MyServer libee-0.3.2]# make > make all-recursive > make[1]: Entering directory `/root/libee-0.3.2' > Making all in tests > make[2]: Entering directory `/root/libee-0.3.2/tests' > make[2]: Nothing to be done for `all'. > make[2]: Leaving directory `/root/libee-0.3.2/tests' > Making all in include > make[2]: Entering directory `/root/libee-0.3.2/include' > Making all in libee > make[3]: Entering directory `/root/libee-0.3.2/include/libee' > make[3]: Nothing to be done for `all'. > make[3]: Leaving directory `/root/libee-0.3.2/include/libee' > make[3]: Entering directory `/root/libee-0.3.2/include' > make[3]: Nothing to be done for `all-am'. > make[3]: Leaving directory `/root/libee-0.3.2/include' > make[2]: Leaving directory `/root/libee-0.3.2/include' > Making all in src > make[2]: Entering directory `/root/libee-0.3.2/src' > CC libee_la-ctx.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-tag.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-event.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-value.lo > value.c: In function 'ee_newValue': > value.c:37: warning: unused parameter 'ctx' > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-tagbucket.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-field.lo > field.c: In function 'ee_getFieldValueAsStr': > field.c:181: warning: 'str' may be used uninitialized in this function > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-fieldbucket.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-primitivetype.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-int_dec.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-apache_dec.lo > apache_dec.c: In function 'ee_newApache': > apache_dec.c:37: warning: unused parameter 'ctx' > apache_dec.c: In function 'ee_apacheAddName': > apache_dec.c:71: warning: unused parameter 'ctx' > apache_dec.c: In function 'processLn': > apache_dec.c:205: warning: unused variable 'value' > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-syslog_enc.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-json_enc.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-csv_enc.lo > csv_enc.c: In function 'ee_AddName': > csv_enc.c:66: warning: unused parameter 'ctx' > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-xml_enc.lo > xml_enc.c: In function 'ee_addValue_XML': > xml_enc.c:60: warning: unused variable 'j' > xml_enc.c:59: warning: unused variable 'numbuf' > xml_enc.c: At top level: > xml_enc.c:40: warning: 'hexdigit' defined but not used > gcc: /usr/local/include: linker input file unused because linking not done > CCLD libee.la > CC convert-convert.o > gcc: /usr/local/include: linker input file unused because linking not done > CCLD convert > /usr/local/lib: file not recognized: Is a directory > collect2: ld returned 1 exit status > make[2]: *** [convert] Error 1 > make[2]: Leaving directory `/root/libee-0.3.2/src' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/libee-0.3.2' > make: *** [all] Error 2 > [root at MyServer libee-0.3.2]# echo $? > 2 > > It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know > how to solve this ? > > > > Thanks in advance :-) ! > ~maymann > > 2012/1/16 Rainer Gerhards > >> > -----Original Message----- >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >> > Sent: Monday, January 16, 2012 10:48 AM >> > To: rsyslog-users >> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> howto/links/examples >> > >> > Hi Rainer, >> > >> > Thanks for your reply. >> > How do I install 6.3.6 on RHEL6 easiest ? >> >> I have no specific instructions. Just grab the sources and compile, I'd >> say >> ;) Note, however, that you need to install libestr and probably libee >> first. >> >> Rainer >> > >> > Thanks in advance :-)! >> > ~maymann >> > >> > 2012/1/16 Rainer Gerhards >> > >> > > The cache is available since 6.3.1, so you need to go for the devel >> > > version. >> > > A good place to check those things is the ChangeLog itself, here is >> > > the current one: >> > > >> > > >> > > >> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 >> > > 575d085 >> > > a0fcf48f71339154813971< >> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= >> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> >> > > ;hb=HEAD >> > > >> > > Note that v6-devel is almost as stable as v6-stable except for the >> > > config read phase at startup. >> > > >> > > HTH >> > > Rainer >> > > >> > > > -----Original Message----- >> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >> > > > Sent: Monday, January 16, 2012 8:57 AM >> > > > To: david at lang.hm >> > > > Cc: rsyslog-users >> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> > > > howto/links/examples >> > > > >> > > > If I want DNS caching, should i use the new stable-6.2.0 or the >> > > > older devel-6.3.6... ? >> > > > Can see this feature mentioned here: >> > > > http://rsyslog.com/features/ >> > > > or here: >> > > > http://rsyslog.com/project-status/ >> > > > >> > > > >> > > > Thanks in advance :-) ! >> > > > ~maymann >> > > > >> > > > 2012/1/14 >> > > > >> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: >> > > > > >> > > > > Hi David, >> > > > >> >> > > > >> thanks for this...this is super info...:-) ! >> > > > >> If I have to create different logs per host, will this be the a >> > > > valid >> > > > >> configuration: >> > > > >> $template >> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >> > > > >> *.* -?DynaFile_messages >> > > > >> $template >> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >> > > > >> *.* -?DynaFile_secure >> > > > >> $template >> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >> > > > >> *.* -?DynaFile_auth.log >> > > > >> >> > > > > >> > > > > I believe so. >> > > > > >> > > > > >> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I >> > > > have to >> > > > >> create every hosts dir upfront... ? >> > > > >> >> > > > > >> > > > > it will create it for you (make sure it's running with the >> > > > appropriate >> > > > > permissions, if you have rsyslog configured to drop privileges, >> > > > > the >> > > > lower >> > > > > privileges need the ability to create the directories) >> > > > > >> > > > > >> > > > > 2. Is DNS caching default enabled or do I have to enable this >> > > > somewhere >> > > > >> first...? >> > > > >> >> > > > > >> > > > > I don't know, I haven't had a chance to look into that yet. >> > > > > >> > > > > David Lang >> > > > > >> > > > > >> > > > >> Thanks in advance :-) ! >> > > > >> ~maymann >> > > > >> >> > > > >> >> > > > >> 2012/1/14 >> > > > >> >> > > > >> http://rsyslog.com/article60/ >> > > > >>> >> > > > >>> David Lang >> > > > >>> >> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >> > > > >>> >> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >> > > > >>> >> > > > >>>> From: Michael Maymann >> > > > >>>> To: rsyslog-users , david at lang.hm, >> > > > >>>> Michael Maymann >> > > > >>>> >> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> > > > >>>> howto/links/examples >> > > > >>>> >> > > > >>>> Hi David, >> > > > >>>> >> > > > >>>> thanks for you kind reply...:-) ! >> > > > >>>> --- >> > > > >>>> This didn't seem to get through to the archives for some >> > > > reason...: >> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >> > > > >>>> >> > > > January/thread.html< >> http://lists.adiscon.net/**pipermail/rsyslog/201 >> > > > 2- >> > > > **January/thread.html> >> > > > >>>> > > > > January/thread.** >> > > > >>>> html> > > > January/thread.html> >> > > > >>>> > >> > > > >>>> >> > > > >>>> Hope I will not dobbel-post... >> > > > >>>> --- >> > > > >>>> I don't use syslog-relays, so this will not cause me any >> problems. >> > > > >>>> Don't actually know what version we are running - can see this >> > > > Monday >> > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 >> > > > >>>> if >> > > > not >> > > > >>>> already then. >> > > > >>>> I have to configure this into a already running live production >> > > > system - >> > > > >>>> our previous syslog-admin left...:-(. >> > > > >>>> Could I perhaps ask you to be so kind as to give an >> > > > >>>> configuration example of how this is done, if I ask really >> > > > >>>> nicely... :-) ? >> > > > >>>> >> > > > >>>> Thanks in advance :-) ! >> > > > >>>> ~maymann >> > > > >>>> >> > > > >>>> 2012/1/13 >> > > > >>>> >> > > > >>>> you need to be aware that doing the DNS queries is rather >> > > > expensive >> > > > >>>> >> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 >> > > > version >> > > > >>>>> there >> > > > >>>>> may now be a DNS cache that will drastically help) >> > > > >>>>> >> > > > >>>>> you would need to create a template with FROMHOST in it and >> > > > >>>>> use >> > > > that as >> > > > >>>>> the filename to write to (look for dynafile in the >> > > > >>>>> documentation) >> > > > >>>>> >> > > > >>>>> note that if you are relaying logs from one machine to >> > > > >>>>> another, >> > > > only >> > > > >>>>> the >> > > > >>>>> first machine will see the true source in FROMHOST, machines >> > > > after that >> > > > >>>>> will only see the relay box. >> > > > >>>>> >> > > > >>>>> let me know if this doesn't give you enough clues to learn how >> > > > >>>>> to >> > > > do >> > > > >>>>> this. >> > > > >>>>> >> > > > >>>>> David Lang >> > > > >>>>> >> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >> > > > >>>>> >> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >> > > > >>>>> >> > > > >>>>> From: Michael Maymann >> > > > >>>>>> Reply-To: rsyslog-users >> > > > >>>>>> To: rsyslog at lists.adiscon.com >> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> > > > >>>>>> howto/links/examples >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS >> > > > >>>>>> and >> > > > not >> > > > >>>>>> from >> > > > >>>>>> syslog-info hostname. >> > > > >>>>>> We are getting a lot of weird logfiles as some applications >> > > > >>>>>> are >> > > > not >> > > > >>>>>> including the hostname as the first parameter in the syslog- >> > > > entries, >> > > > >>>>>> e.g.: >> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >> > > > >>>>>> [set_host_compat_list] >> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd >> > "SetUserData >> > > > >>>>>> HostCompatList text >> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... >> > > > >>>>>> >> > > > >>>>>> Thanks in advance :-) ! >> > > > >>>>>> ~maymann >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> 2012/1/13 Michael Maymann >> > > > >>>>>> >> > > > >>>>>> Hi List, >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>>> I'm new to rsyslog/syslog in general. >> > > > >>>>>>> >> > > > >>>>>>> I would like to syslog from all my 100+ network devices. >> > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a >> > > > FQDN-dir >> > > > >>>>>>> containing logs from this host if more logfiles per host are >> > > > best >> > > > >>>>>>> practice)... >> > > > >>>>>>> >> > > > >>>>>>> Can anyone give me an example of (or link to) best practice >> > > > >>>>>>> of >> > > > this >> > > > >>>>>>> kind >> > > > >>>>>>> of setup. >> > > > >>>>>>> >> > > > >>>>>>> >> > > > >>>>>>> Thanks in advance :-) ! >> > > > >>>>>>> >> > > > >>>>>>> ~maymann >> > > > >>>>>>> >> > > > >>>>>>> >> > ______________________________******_________________ >> > > > >>>>>>> >> > > > >>>>>>> rsyslog mailing list >> > > > >>>>>> >> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >> http://lists >> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >> > > > >>>>>> >> > > > > http://list >> > > > s.a discon.net/**mailman/listinfo/rsyslog> >> > > > >>>>>> > >> > > > >>>>>> >> > > > > http://list >> > > > s.a discon.net/mailman/**listinfo/rsyslog> >> > > > >>>>>> >> > > > > . >> > > > adi >> > > > scon.net/mailman/listinfo/rsyslog> >> > > > >>>>>> > >> > > > >>>>>> >> > > > >>>>>>> >> > > > >>>>>>> http://www.rsyslog.com/******professional- >> > > > services/ >> > > > >>>>>> > > > > services/ >> > > > >>>>>> > >> > > > >>>>>> > > > > **services/ >> > > > >>>>>> > > > > services/ >> > > > >>>>>> > >> > > > >>>>>> >> > > > >>>>>>> >> > > > >>>>>>> >> > > > >>>>>> >> > ______________________________******_________________ >> > > > >>>>>> >> > > > >>>>>> rsyslog mailing list >> > > > >>>>> >> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >> http://lists >> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >> > > > >>>>> >> > > > > http://list >> > > > s.a discon.net/**mailman/listinfo/rsyslog> >> > > > >>>>> > >> > > > >>>>> >> > > > > http://list >> > > > s.a discon.net/mailman/**listinfo/rsyslog> >> > > > >>>>> >> > > > > . >> > > > adi >> > > > scon.net/mailman/listinfo/rsyslog> >> > > > >>>>> > >> > > > >>>>> >> > > > >>>>>> >> > > > >>>>>> http://www.rsyslog.com/******professional- >> > > > services/ >> > > > >>>>> > > > > services/ >> > > > >>>>> > >> > > > >>>>> > > > > **services/ >> > > > >>>>> > > > > services/ >> > > > >>>>> > >> > > > >>>>> >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>> >> > > > >>>>> >> > > > >>>> >> > > > >> >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> > > From michael at maymann.org Wed Jan 18 20:36:07 2012 From: michael at maymann.org (Michael Maymann) Date: Wed, 18 Jan 2012 20:36:07 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> Message-ID: The --libdir=/usr/lib64 option did the trick...:-): cd libestr ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd libee/ ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd rsyslog-6.3.6 ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr [root at MyServer rsyslog-6.3.6]# echo $? 0 [root at MyServer rsyslog-6.3.6]# make make all-recursive make[1]: Entering directory `/root/rsyslog-6.3.6' Making all in doc make[2]: Entering directory `/root/rsyslog-6.3.6/doc' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/rsyslog-6.3.6/doc' Making all in runtime make[2]: Entering directory `/root/rsyslog-6.3.6/runtime' CC librsyslog_la-rsyslog.lo In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:28:18: error: zlib.h: No such file or directory In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:32: error: expected ')' before 'strm' zlibw.h:33: error: expected ';' before 'int' In file included from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: stream.h:123: error: expected specifier-qualifier-list before 'Bytef' make[2]: *** [librsyslog_la-rsyslog.lo] Error 1 make[2]: Leaving directory `/root/rsyslog-6.3.6/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog-6.3.6' make: *** [all] Error 2 So both libestr and linee in installed and ./configure in rsyslog dir ran without errors, but now make is causing me problems..., anyone who know how to solve this ? Thanks in advance :-) ! ~maymann 2012/1/17 Michael Maymann > Tried to delete all installed and follow this guide: > http://www.liblognorm.com/help/first-steps-using-liblognorm/ > same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib > --includedir=/usr/include: > ... > checking for LIBESTR... configure: error: Package requirements (libestr >= > 0.0.0) were not met: > > No package 'libestr' found > > Consider adjusting the PKG_CONFIG_PATH environment variable if you > installed software in a non-standard prefix. > > Alternatively, you may set the environment variables LIBESTR_CFLAGS > and LIBESTR_LIBS to avoid the need to call pkg-config. > See the pkg-config man page for more details. > > > Thanks in advance :-) ! > ~maymann > > > 2012/1/17 Michael Maymann > >> Hi Rainer, >> >> yes, rsyslog-6.3.6 needs libestr and libee...: >> I installed libestr just fine: >> [root at MyServer pkgconfig]# locate libestr >> /root/libestr-0.1.2 >> /root/libestr-0.1.2.tar.gz >> /root/libestr-0.1.2/AUTHORS >> /root/libestr-0.1.2/COPYING >> /root/libestr-0.1.2/ChangeLog >> /root/libestr-0.1.2/INSTALL >> /root/libestr-0.1.2/Makefile >> /root/libestr-0.1.2/Makefile.am >> /root/libestr-0.1.2/Makefile.in >> /root/libestr-0.1.2/NEWS >> /root/libestr-0.1.2/README >> /root/libestr-0.1.2/aclocal.m4 >> /root/libestr-0.1.2/compile >> /root/libestr-0.1.2/config.guess >> /root/libestr-0.1.2/config.h >> /root/libestr-0.1.2/config.h.in >> /root/libestr-0.1.2/config.log >> /root/libestr-0.1.2/config.status >> /root/libestr-0.1.2/config.sub >> /root/libestr-0.1.2/configure >> /root/libestr-0.1.2/configure.ac >> /root/libestr-0.1.2/depcomp >> /root/libestr-0.1.2/include >> /root/libestr-0.1.2/install-sh >> /root/libestr-0.1.2/libestr.pc >> /root/libestr-0.1.2/libestr.pc.in >> /root/libestr-0.1.2/libtool >> /root/libestr-0.1.2/ltmain.sh >> /root/libestr-0.1.2/m4 >> /root/libestr-0.1.2/missing >> /root/libestr-0.1.2/src >> /root/libestr-0.1.2/stamp-h1 >> /root/libestr-0.1.2/include/Makefile >> /root/libestr-0.1.2/include/Makefile.am >> /root/libestr-0.1.2/include/Makefile.in >> /root/libestr-0.1.2/include/libestr.h >> /root/libestr-0.1.2/m4/libtool.m4 >> /root/libestr-0.1.2/m4/ltoptions.m4 >> /root/libestr-0.1.2/m4/ltsugar.m4 >> /root/libestr-0.1.2/m4/ltversion.m4 >> /root/libestr-0.1.2/m4/lt~obsolete.m4 >> /root/libestr-0.1.2/src/.deps >> /root/libestr-0.1.2/src/.libs >> /root/libestr-0.1.2/src/Makefile >> /root/libestr-0.1.2/src/Makefile.am >> /root/libestr-0.1.2/src/Makefile.in >> /root/libestr-0.1.2/src/libestr.c >> /root/libestr-0.1.2/src/libestr.la >> /root/libestr-0.1.2/src/libestr_la-libestr.lo >> /root/libestr-0.1.2/src/libestr_la-libestr.o >> /root/libestr-0.1.2/src/libestr_la-string.lo >> /root/libestr-0.1.2/src/libestr_la-string.o >> /root/libestr-0.1.2/src/string.c >> /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo >> /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo >> /root/libestr-0.1.2/src/.libs/libestr.a >> /root/libestr-0.1.2/src/.libs/libestr.la >> /root/libestr-0.1.2/src/.libs/libestr.lai >> /root/libestr-0.1.2/src/.libs/libestr.so >> /root/libestr-0.1.2/src/.libs/libestr.so.0 >> /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 >> /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o >> /root/libestr-0.1.2/src/.libs/libestr_la-string.o >> /usr/local/include/libestr.h >> /usr/local/lib/libestr.a >> /usr/local/lib/libestr.la >> /usr/local/lib/libestr.so >> /usr/local/lib/libestr.so.0 >> /usr/local/lib/libestr.so.0.0.0 >> /usr/local/lib/pkgconfig/libestr.pc >> >> I try to install libee: >> [root at MyServer libee-0.3.2]# ./configure >> LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib >> checking for a BSD-compatible install... /usr/bin/install -c >> checking whether build environment is sane... yes >> checking for a thread-safe mkdir -p... /bin/mkdir -p >> checking for gawk... gawk >> checking whether make sets $(MAKE)... yes >> checking for gcc... gcc >> checking whether the C compiler works... yes >> checking for C compiler default output file name... a.out >> checking for suffix of executables... >> checking whether we are cross compiling... no >> checking for suffix of object files... o >> checking whether we are using the GNU C compiler... yes >> checking whether gcc accepts -g... yes >> checking for gcc option to accept ISO C89... none needed >> checking for style of include used by make... GNU >> checking dependency style of gcc... gcc3 >> checking whether gcc and cc understand -c and -o together... yes >> checking build system type... x86_64-unknown-linux-gnu >> checking host system type... x86_64-unknown-linux-gnu >> checking how to print strings... printf >> checking for a sed that does not truncate output... /bin/sed >> checking for grep that handles long lines and -e... /bin/grep >> checking for egrep... /bin/grep -E >> checking for fgrep... /bin/grep -F >> checking for ld used by gcc... /usr/bin/ld >> checking if the linker (/usr/bin/ld) is GNU ld... yes >> checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B >> checking the name lister (/usr/bin/nm -B) interface... BSD nm >> checking whether ln -s works... yes >> checking the maximum length of command line arguments... 1966080 >> checking whether the shell understands some XSI constructs... yes >> checking whether the shell understands "+="... yes >> checking for /usr/bin/ld option to reload object files... -r >> checking for objdump... objdump >> checking how to recognize dependent libraries... pass_all >> checking for ar... ar >> checking for strip... strip >> checking for ranlib... ranlib >> checking command to parse /usr/bin/nm -B output from gcc object... ok >> checking how to run the C preprocessor... gcc -E >> checking for ANSI C header files... yes >> checking for sys/types.h... yes >> checking for sys/stat.h... yes >> checking for stdlib.h... yes >> checking for string.h... yes >> checking for memory.h... yes >> checking for strings.h... yes >> checking for inttypes.h... yes >> checking for stdint.h... yes >> checking for unistd.h... yes >> checking for dlfcn.h... yes >> checking for objdir... .libs >> checking if gcc supports -fno-rtti -fno-exceptions... no >> checking for gcc option to produce PIC... -fPIC -DPIC >> checking if gcc PIC flag -fPIC -DPIC works... yes >> checking if gcc static flag -static works... no >> checking if gcc supports -c -o file.o... yes >> checking if gcc supports -c -o file.o... (cached) yes >> checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports >> shared libraries... yes >> checking whether -lc should be explicitly linked in... no >> checking dynamic linker characteristics... GNU/Linux ld.so >> checking how to hardcode library paths into programs... immediate >> checking whether stripping libraries is possible... yes >> checking if libtool supports shared libraries... yes >> checking whether to build shared libraries... yes >> checking whether to build static libraries... yes >> checking for stdlib.h... (cached) yes >> checking for GNU libc compatible malloc... yes >> checking for pkg-config... /usr/bin/pkg-config >> checking pkg-config is at least version 0.9.0... yes >> checking for LIBESTR... yes >> configure: creating ./config.status >> config.status: creating Makefile >> config.status: creating libee.pc >> config.status: creating src/Makefile >> config.status: creating include/Makefile >> config.status: creating include/libee/Makefile >> config.status: creating tests/Makefile >> config.status: creating config.h >> config.status: config.h is unchanged >> config.status: executing depfiles commands >> config.status: executing libtool commands >> ***************************************************** >> libee will be compiled with the following settings: >> >> Debug mode enabled: no >> Testbench enabled: yes >> [root at MyServer libee-0.3.2]# echo $? >> 0 >> [root at MyServer libee-0.3.2]# make >> make all-recursive >> make[1]: Entering directory `/root/libee-0.3.2' >> Making all in tests >> make[2]: Entering directory `/root/libee-0.3.2/tests' >> make[2]: Nothing to be done for `all'. >> make[2]: Leaving directory `/root/libee-0.3.2/tests' >> Making all in include >> make[2]: Entering directory `/root/libee-0.3.2/include' >> Making all in libee >> make[3]: Entering directory `/root/libee-0.3.2/include/libee' >> make[3]: Nothing to be done for `all'. >> make[3]: Leaving directory `/root/libee-0.3.2/include/libee' >> make[3]: Entering directory `/root/libee-0.3.2/include' >> make[3]: Nothing to be done for `all-am'. >> make[3]: Leaving directory `/root/libee-0.3.2/include' >> make[2]: Leaving directory `/root/libee-0.3.2/include' >> Making all in src >> make[2]: Entering directory `/root/libee-0.3.2/src' >> CC libee_la-ctx.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tag.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-event.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-value.lo >> value.c: In function 'ee_newValue': >> value.c:37: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tagbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-field.lo >> field.c: In function 'ee_getFieldValueAsStr': >> field.c:181: warning: 'str' may be used uninitialized in this function >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-fieldbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-primitivetype.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-int_dec.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-apache_dec.lo >> apache_dec.c: In function 'ee_newApache': >> apache_dec.c:37: warning: unused parameter 'ctx' >> apache_dec.c: In function 'ee_apacheAddName': >> apache_dec.c:71: warning: unused parameter 'ctx' >> apache_dec.c: In function 'processLn': >> apache_dec.c:205: warning: unused variable 'value' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-syslog_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-json_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-csv_enc.lo >> csv_enc.c: In function 'ee_AddName': >> csv_enc.c:66: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-xml_enc.lo >> xml_enc.c: In function 'ee_addValue_XML': >> xml_enc.c:60: warning: unused variable 'j' >> xml_enc.c:59: warning: unused variable 'numbuf' >> xml_enc.c: At top level: >> xml_enc.c:40: warning: 'hexdigit' defined but not used >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD libee.la >> CC convert-convert.o >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD convert >> /usr/local/lib: file not recognized: Is a directory >> collect2: ld returned 1 exit status >> make[2]: *** [convert] Error 1 >> make[2]: Leaving directory `/root/libee-0.3.2/src' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/root/libee-0.3.2' >> make: *** [all] Error 2 >> [root at MyServer libee-0.3.2]# echo $? >> 2 >> >> It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know >> how to solve this ? >> >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/16 Rainer Gerhards >> >>> > -----Original Message----- >>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > Sent: Monday, January 16, 2012 10:48 AM >>> > To: rsyslog-users >>> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> howto/links/examples >>> > >>> > Hi Rainer, >>> > >>> > Thanks for your reply. >>> > How do I install 6.3.6 on RHEL6 easiest ? >>> >>> I have no specific instructions. Just grab the sources and compile, I'd >>> say >>> ;) Note, however, that you need to install libestr and probably libee >>> first. >>> >>> Rainer >>> > >>> > Thanks in advance :-)! >>> > ~maymann >>> > >>> > 2012/1/16 Rainer Gerhards >>> > >>> > > The cache is available since 6.3.1, so you need to go for the devel >>> > > version. >>> > > A good place to check those things is the ChangeLog itself, here is >>> > > the current one: >>> > > >>> > > >>> > > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 >>> > > 575d085 >>> > > a0fcf48f71339154813971< >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= >>> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> >>> > > ;hb=HEAD >>> > > >>> > > Note that v6-devel is almost as stable as v6-stable except for the >>> > > config read phase at startup. >>> > > >>> > > HTH >>> > > Rainer >>> > > >>> > > > -----Original Message----- >>> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > > > Sent: Monday, January 16, 2012 8:57 AM >>> > > > To: david at lang.hm >>> > > > Cc: rsyslog-users >>> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > howto/links/examples >>> > > > >>> > > > If I want DNS caching, should i use the new stable-6.2.0 or the >>> > > > older devel-6.3.6... ? >>> > > > Can see this feature mentioned here: >>> > > > http://rsyslog.com/features/ >>> > > > or here: >>> > > > http://rsyslog.com/project-status/ >>> > > > >>> > > > >>> > > > Thanks in advance :-) ! >>> > > > ~maymann >>> > > > >>> > > > 2012/1/14 >>> > > > >>> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > > >>> > > > > Hi David, >>> > > > >> >>> > > > >> thanks for this...this is super info...:-) ! >>> > > > >> If I have to create different logs per host, will this be the a >>> > > > valid >>> > > > >> configuration: >>> > > > >> $template >>> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >>> > > > >> *.* -?DynaFile_messages >>> > > > >> $template >>> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >>> > > > >> *.* -?DynaFile_secure >>> > > > >> $template >>> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >>> > > > >> *.* -?DynaFile_auth.log >>> > > > >> >>> > > > > >>> > > > > I believe so. >>> > > > > >>> > > > > >>> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do >>> I >>> > > > have to >>> > > > >> create every hosts dir upfront... ? >>> > > > >> >>> > > > > >>> > > > > it will create it for you (make sure it's running with the >>> > > > appropriate >>> > > > > permissions, if you have rsyslog configured to drop privileges, >>> > > > > the >>> > > > lower >>> > > > > privileges need the ability to create the directories) >>> > > > > >>> > > > > >>> > > > > 2. Is DNS caching default enabled or do I have to enable this >>> > > > somewhere >>> > > > >> first...? >>> > > > >> >>> > > > > >>> > > > > I don't know, I haven't had a chance to look into that yet. >>> > > > > >>> > > > > David Lang >>> > > > > >>> > > > > >>> > > > >> Thanks in advance :-) ! >>> > > > >> ~maymann >>> > > > >> >>> > > > >> >>> > > > >> 2012/1/14 >>> > > > >> >>> > > > >> http://rsyslog.com/article60/ >>> > > > >>> >>> > > > >>> David Lang >>> > > > >>> >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > >>> >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> > > > >>> >>> > > > >>>> From: Michael Maymann >>> > > > >>>> To: rsyslog-users , david at lang.hm, >>> > > > >>>> Michael Maymann >>> > > > >>>> >>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>> howto/links/examples >>> > > > >>>> >>> > > > >>>> Hi David, >>> > > > >>>> >>> > > > >>>> thanks for you kind reply...:-) ! >>> > > > >>>> --- >>> > > > >>>> This didn't seem to get through to the archives for some >>> > > > reason...: >>> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>> > > > >>>> >>> > > > January/thread.html< >>> http://lists.adiscon.net/**pipermail/rsyslog/201 >>> > > > 2- >>> > > > **January/thread.html> >>> > > > >>>> >> > > > January/thread.** >>> > > > >>>> html>> > > > January/thread.html> >>> > > > >>>> > >>> > > > >>>> >>> > > > >>>> Hope I will not dobbel-post... >>> > > > >>>> --- >>> > > > >>>> I don't use syslog-relays, so this will not cause me any >>> problems. >>> > > > >>>> Don't actually know what version we are running - can see this >>> > > > Monday >>> > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 >>> > > > >>>> if >>> > > > not >>> > > > >>>> already then. >>> > > > >>>> I have to configure this into a already running live >>> production >>> > > > system - >>> > > > >>>> our previous syslog-admin left...:-(. >>> > > > >>>> Could I perhaps ask you to be so kind as to give an >>> > > > >>>> configuration example of how this is done, if I ask really >>> > > > >>>> nicely... :-) ? >>> > > > >>>> >>> > > > >>>> Thanks in advance :-) ! >>> > > > >>>> ~maymann >>> > > > >>>> >>> > > > >>>> 2012/1/13 >>> > > > >>>> >>> > > > >>>> you need to be aware that doing the DNS queries is rather >>> > > > expensive >>> > > > >>>> >>> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 >>> > > > version >>> > > > >>>>> there >>> > > > >>>>> may now be a DNS cache that will drastically help) >>> > > > >>>>> >>> > > > >>>>> you would need to create a template with FROMHOST in it and >>> > > > >>>>> use >>> > > > that as >>> > > > >>>>> the filename to write to (look for dynafile in the >>> > > > >>>>> documentation) >>> > > > >>>>> >>> > > > >>>>> note that if you are relaying logs from one machine to >>> > > > >>>>> another, >>> > > > only >>> > > > >>>>> the >>> > > > >>>>> first machine will see the true source in FROMHOST, machines >>> > > > after that >>> > > > >>>>> will only see the relay box. >>> > > > >>>>> >>> > > > >>>>> let me know if this doesn't give you enough clues to learn >>> how >>> > > > >>>>> to >>> > > > do >>> > > > >>>>> this. >>> > > > >>>>> >>> > > > >>>>> David Lang >>> > > > >>>>> >>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>> > > > >>>>> >>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> > > > >>>>> >>> > > > >>>>> From: Michael Maymann >>> > > > >>>>>> Reply-To: rsyslog-users >>> > > > >>>>>> To: rsyslog at lists.adiscon.com >>> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>>>> howto/links/examples >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS >>> > > > >>>>>> and >>> > > > not >>> > > > >>>>>> from >>> > > > >>>>>> syslog-info hostname. >>> > > > >>>>>> We are getting a lot of weird logfiles as some applications >>> > > > >>>>>> are >>> > > > not >>> > > > >>>>>> including the hostname as the first parameter in the syslog- >>> > > > entries, >>> > > > >>>>>> e.g.: >>> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>> > > > >>>>>> [set_host_compat_list] >>> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd >>> > "SetUserData >>> > > > >>>>>> HostCompatList text >>> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... >>> > > > >>>>>> >>> > > > >>>>>> Thanks in advance :-) ! >>> > > > >>>>>> ~maymann >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> 2012/1/13 Michael Maymann >>> > > > >>>>>> >>> > > > >>>>>> Hi List, >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>>> I'm new to rsyslog/syslog in general. >>> > > > >>>>>>> >>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. >>> > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a >>> > > > FQDN-dir >>> > > > >>>>>>> containing logs from this host if more logfiles per host >>> are >>> > > > best >>> > > > >>>>>>> practice)... >>> > > > >>>>>>> >>> > > > >>>>>>> Can anyone give me an example of (or link to) best practice >>> > > > >>>>>>> of >>> > > > this >>> > > > >>>>>>> kind >>> > > > >>>>>>> of setup. >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> Thanks in advance :-) ! >>> > > > >>>>>>> >>> > > > >>>>>>> ~maymann >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>>> >>> > > > >>>>>>> rsyslog mailing list >>> > > > >>>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >> > > > **services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>> >>> > > > >>>>>> rsyslog mailing list >>> > > > >>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >> > > > **services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>> >>> > > > >>>>> >>> > > > >>>> >>> > > > >> >>> > > > _______________________________________________ >>> > > > rsyslog mailing list >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > > http://www.rsyslog.com/professional-services/ >>> > > _______________________________________________ >>> > > rsyslog mailing list >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > http://www.rsyslog.com/professional-services/ >>> > > >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com/professional-services/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> >> >> > From rgerhards at hq.adiscon.com Wed Jan 18 20:38:52 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 18 Jan 2012 20:38:52 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples Message-ID: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Zlib-dev is missing, looks like configure did not catch that. Rainer Michael Maymann hat geschrieben:The --libdir=/usr/lib64 option did the trick...:-): cd libestr ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd libee/ ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd rsyslog-6.3.6 ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr [root at MyServer rsyslog-6.3.6]# echo $? 0 [root at MyServer rsyslog-6.3.6]# make make all-recursive make[1]: Entering directory `/root/rsyslog-6.3.6' Making all in doc make[2]: Entering directory `/root/rsyslog-6.3.6/doc' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/rsyslog-6.3.6/doc' Making all in runtime make[2]: Entering directory `/root/rsyslog-6.3.6/runtime' CC librsyslog_la-rsyslog.lo In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:28:18: error: zlib.h: No such file or directory In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:32: error: expected ')' before 'strm' zlibw.h:33: error: expected ';' before 'int' In file included from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: stream.h:123: error: expected specifier-qualifier-list before 'Bytef' make[2]: *** [librsyslog_la-rsyslog.lo] Error 1 make[2]: Leaving directory `/root/rsyslog-6.3.6/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog-6.3.6' make: *** [all] Error 2 So both libestr and linee in installed and ./configure in rsyslog dir ran without errors, but now make is causing me problems..., anyone who know how to solve this ? Thanks in advance :-) ! ~maymann 2012/1/17 Michael Maymann > Tried to delete all installed and follow this guide: > http://www.liblognorm.com/help/first-steps-using-liblognorm/ > same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib > --includedir=/usr/include: > ... > checking for LIBESTR... configure: error: Package requirements (libestr >= > 0.0.0) were not met: > > No package 'libestr' found > > Consider adjusting the PKG_CONFIG_PATH environment variable if you > installed software in a non-standard prefix. > > Alternatively, you may set the environment variables LIBESTR_CFLAGS > and LIBESTR_LIBS to avoid the need to call pkg-config. > See the pkg-config man page for more details. > > > Thanks in advance :-) ! > ~maymann > > > 2012/1/17 Michael Maymann > >> Hi Rainer, >> >> yes, rsyslog-6.3.6 needs libestr and libee...: >> I installed libestr just fine: >> [root at MyServer pkgconfig]# locate libestr >> /root/libestr-0.1.2 >> /root/libestr-0.1.2.tar.gz >> /root/libestr-0.1.2/AUTHORS >> /root/libestr-0.1.2/COPYING >> /root/libestr-0.1.2/ChangeLog >> /root/libestr-0.1.2/INSTALL >> /root/libestr-0.1.2/Makefile >> /root/libestr-0.1.2/Makefile.am >> /root/libestr-0.1.2/Makefile.in >> /root/libestr-0.1.2/NEWS >> /root/libestr-0.1.2/README >> /root/libestr-0.1.2/aclocal.m4 >> /root/libestr-0.1.2/compile >> /root/libestr-0.1.2/config.guess >> /root/libestr-0.1.2/config.h >> /root/libestr-0.1.2/config.h.in >> /root/libestr-0.1.2/config.log >> /root/libestr-0.1.2/config.status >> /root/libestr-0.1.2/config.sub >> /root/libestr-0.1.2/configure >> /root/libestr-0.1.2/configure.ac >> /root/libestr-0.1.2/depcomp >> /root/libestr-0.1.2/include >> /root/libestr-0.1.2/install-sh >> /root/libestr-0.1.2/libestr.pc >> /root/libestr-0.1.2/libestr.pc.in >> /root/libestr-0.1.2/libtool >> /root/libestr-0.1.2/ltmain.sh >> /root/libestr-0.1.2/m4 >> /root/libestr-0.1.2/missing >> /root/libestr-0.1.2/src >> /root/libestr-0.1.2/stamp-h1 >> /root/libestr-0.1.2/include/Makefile >> /root/libestr-0.1.2/include/Makefile.am >> /root/libestr-0.1.2/include/Makefile.in >> /root/libestr-0.1.2/include/libestr.h >> /root/libestr-0.1.2/m4/libtool.m4 >> /root/libestr-0.1.2/m4/ltoptions.m4 >> /root/libestr-0.1.2/m4/ltsugar.m4 >> /root/libestr-0.1.2/m4/ltversion.m4 >> /root/libestr-0.1.2/m4/lt~obsolete.m4 >> /root/libestr-0.1.2/src/.deps >> /root/libestr-0.1.2/src/.libs >> /root/libestr-0.1.2/src/Makefile >> /root/libestr-0.1.2/src/Makefile.am >> /root/libestr-0.1.2/src/Makefile.in >> /root/libestr-0.1.2/src/libestr.c >> /root/libestr-0.1.2/src/libestr.la >> /root/libestr-0.1.2/src/libestr_la-libestr.lo >> /root/libestr-0.1.2/src/libestr_la-libestr.o >> /root/libestr-0.1.2/src/libestr_la-string.lo >> /root/libestr-0.1.2/src/libestr_la-string.o >> /root/libestr-0.1.2/src/string.c >> /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo >> /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo >> /root/libestr-0.1.2/src/.libs/libestr.a >> /root/libestr-0.1.2/src/.libs/libestr.la >> /root/libestr-0.1.2/src/.libs/libestr.lai >> /root/libestr-0.1.2/src/.libs/libestr.so >> /root/libestr-0.1.2/src/.libs/libestr.so.0 >> /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 >> /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o >> /root/libestr-0.1.2/src/.libs/libestr_la-string.o >> /usr/local/include/libestr.h >> /usr/local/lib/libestr.a >> /usr/local/lib/libestr.la >> /usr/local/lib/libestr.so >> /usr/local/lib/libestr.so.0 >> /usr/local/lib/libestr.so.0.0.0 >> /usr/local/lib/pkgconfig/libestr.pc >> >> I try to install libee: >> [root at MyServer libee-0.3.2]# ./configure >> LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib >> checking for a BSD-compatible install... /usr/bin/install -c >> checking whether build environment is sane... yes >> checking for a thread-safe mkdir -p... /bin/mkdir -p >> checking for gawk... gawk >> checking whether make sets $(MAKE)... yes >> checking for gcc... gcc >> checking whether the C compiler works... yes >> checking for C compiler default output file name... a.out >> checking for suffix of executables... >> checking whether we are cross compiling... no >> checking for suffix of object files... o >> checking whether we are using the GNU C compiler... yes >> checking whether gcc accepts -g... yes >> checking for gcc option to accept ISO C89... none needed >> checking for style of include used by make... GNU >> checking dependency style of gcc... gcc3 >> checking whether gcc and cc understand -c and -o together... yes >> checking build system type... x86_64-unknown-linux-gnu >> checking host system type... x86_64-unknown-linux-gnu >> checking how to print strings... printf >> checking for a sed that does not truncate output... /bin/sed >> checking for grep that handles long lines and -e... /bin/grep >> checking for egrep... /bin/grep -E >> checking for fgrep... /bin/grep -F >> checking for ld used by gcc... /usr/bin/ld >> checking if the linker (/usr/bin/ld) is GNU ld... yes >> checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B >> checking the name lister (/usr/bin/nm -B) interface... BSD nm >> checking whether ln -s works... yes >> checking the maximum length of command line arguments... 1966080 >> checking whether the shell understands some XSI constructs... yes >> checking whether the shell understands "+="... yes >> checking for /usr/bin/ld option to reload object files... -r >> checking for objdump... objdump >> checking how to recognize dependent libraries... pass_all >> checking for ar... ar >> checking for strip... strip >> checking for ranlib... ranlib >> checking command to parse /usr/bin/nm -B output from gcc object... ok >> checking how to run the C preprocessor... gcc -E >> checking for ANSI C header files... yes >> checking for sys/types.h... yes >> checking for sys/stat.h... yes >> checking for stdlib.h... yes >> checking for string.h... yes >> checking for memory.h... yes >> checking for strings.h... yes >> checking for inttypes.h... yes >> checking for stdint.h... yes >> checking for unistd.h... yes >> checking for dlfcn.h... yes >> checking for objdir... .libs >> checking if gcc supports -fno-rtti -fno-exceptions... no >> checking for gcc option to produce PIC... -fPIC -DPIC >> checking if gcc PIC flag -fPIC -DPIC works... yes >> checking if gcc static flag -static works... no >> checking if gcc supports -c -o file.o... yes >> checking if gcc supports -c -o file.o... (cached) yes >> checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports >> shared libraries... yes >> checking whether -lc should be explicitly linked in... no >> checking dynamic linker characteristics... GNU/Linux ld.so >> checking how to hardcode library paths into programs... immediate >> checking whether stripping libraries is possible... yes >> checking if libtool supports shared libraries... yes >> checking whether to build shared libraries... yes >> checking whether to build static libraries... yes >> checking for stdlib.h... (cached) yes >> checking for GNU libc compatible malloc... yes >> checking for pkg-config... /usr/bin/pkg-config >> checking pkg-config is at least version 0.9.0... yes >> checking for LIBESTR... yes >> configure: creating ./config.status >> config.status: creating Makefile >> config.status: creating libee.pc >> config.status: creating src/Makefile >> config.status: creating include/Makefile >> config.status: creating include/libee/Makefile >> config.status: creating tests/Makefile >> config.status: creating config.h >> config.status: config.h is unchanged >> config.status: executing depfiles commands >> config.status: executing libtool commands >> ***************************************************** >> libee will be compiled with the following settings: >> >> Debug mode enabled: no >> Testbench enabled: yes >> [root at MyServer libee-0.3.2]# echo $? >> 0 >> [root at MyServer libee-0.3.2]# make >> make all-recursive >> make[1]: Entering directory `/root/libee-0.3.2' >> Making all in tests >> make[2]: Entering directory `/root/libee-0.3.2/tests' >> make[2]: Nothing to be done for `all'. >> make[2]: Leaving directory `/root/libee-0.3.2/tests' >> Making all in include >> make[2]: Entering directory `/root/libee-0.3.2/include' >> Making all in libee >> make[3]: Entering directory `/root/libee-0.3.2/include/libee' >> make[3]: Nothing to be done for `all'. >> make[3]: Leaving directory `/root/libee-0.3.2/include/libee' >> make[3]: Entering directory `/root/libee-0.3.2/include' >> make[3]: Nothing to be done for `all-am'. >> make[3]: Leaving directory `/root/libee-0.3.2/include' >> make[2]: Leaving directory `/root/libee-0.3.2/include' >> Making all in src >> make[2]: Entering directory `/root/libee-0.3.2/src' >> CC libee_la-ctx.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tag.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-event.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-value.lo >> value.c: In function 'ee_newValue': >> value.c:37: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tagbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-field.lo >> field.c: In function 'ee_getFieldValueAsStr': >> field.c:181: warning: 'str' may be used uninitialized in this function >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-fieldbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-primitivetype.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-int_dec.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-apache_dec.lo >> apache_dec.c: In function 'ee_newApache': >> apache_dec.c:37: warning: unused parameter 'ctx' >> apache_dec.c: In function 'ee_apacheAddName': >> apache_dec.c:71: warning: unused parameter 'ctx' >> apache_dec.c: In function 'processLn': >> apache_dec.c:205: warning: unused variable 'value' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-syslog_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-json_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-csv_enc.lo >> csv_enc.c: In function 'ee_AddName': >> csv_enc.c:66: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-xml_enc.lo >> xml_enc.c: In function 'ee_addValue_XML': >> xml_enc.c:60: warning: unused variable 'j' >> xml_enc.c:59: warning: unused variable 'numbuf' >> xml_enc.c: At top level: >> xml_enc.c:40: warning: 'hexdigit' defined but not used >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD libee.la >> CC convert-convert.o >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD convert >> /usr/local/lib: file not recognized: Is a directory >> collect2: ld returned 1 exit status >> make[2]: *** [convert] Error 1 >> make[2]: Leaving directory `/root/libee-0.3.2/src' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/root/libee-0.3.2' >> make: *** [all] Error 2 >> [root at MyServer libee-0.3.2]# echo $? >> 2 >> >> It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know >> how to solve this ? >> >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/16 Rainer Gerhards >> >>> > -----Original Message----- >>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > Sent: Monday, January 16, 2012 10:48 AM >>> > To: rsyslog-users >>> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> howto/links/examples >>> > >>> > Hi Rainer, >>> > >>> > Thanks for your reply. >>> > How do I install 6.3.6 on RHEL6 easiest ? >>> >>> I have no specific instructions. Just grab the sources and compile, I'd >>> say >>> ;) Note, however, that you need to install libestr and probably libee >>> first. >>> >>> Rainer >>> > >>> > Thanks in advance :-)! >>> > ~maymann >>> > >>> > 2012/1/16 Rainer Gerhards >>> > >>> > > The cache is available since 6.3.1, so you need to go for the devel >>> > > version. >>> > > A good place to check those things is the ChangeLog itself, here is >>> > > the current one: >>> > > >>> > > >>> > > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 >>> > > 575d085 >>> > > a0fcf48f71339154813971< >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= >>> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> >>> > > ;hb=HEAD >>> > > >>> > > Note that v6-devel is almost as stable as v6-stable except for the >>> > > config read phase at startup. >>> > > >>> > > HTH >>> > > Rainer >>> > > >>> > > > -----Original Message----- >>> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > > > Sent: Monday, January 16, 2012 8:57 AM >>> > > > To: david at lang.hm >>> > > > Cc: rsyslog-users >>> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > howto/links/examples >>> > > > >>> > > > If I want DNS caching, should i use the new stable-6.2.0 or the >>> > > > older devel-6.3.6... ? >>> > > > Can see this feature mentioned here: >>> > > > http://rsyslog.com/features/ >>> > > > or here: >>> > > > http://rsyslog.com/project-status/ >>> > > > >>> > > > >>> > > > Thanks in advance :-) ! >>> > > > ~maymann >>> > > > >>> > > > 2012/1/14 >>> > > > >>> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > > >>> > > > > Hi David, >>> > > > >> >>> > > > >> thanks for this...this is super info...:-) ! >>> > > > >> If I have to create different logs per host, will this be the a >>> > > > valid >>> > > > >> configuration: >>> > > > >> $template >>> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >>> > > > >> *.* -?DynaFile_messages >>> > > > >> $template >>> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >>> > > > >> *.* -?DynaFile_secure >>> > > > >> $template >>> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >>> > > > >> *.* -?DynaFile_auth.log >>> > > > >> >>> > > > > >>> > > > > I believe so. >>> > > > > >>> > > > > >>> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do >>> I >>> > > > have to >>> > > > >> create every hosts dir upfront... ? >>> > > > >> >>> > > > > >>> > > > > it will create it for you (make sure it's running with the >>> > > > appropriate >>> > > > > permissions, if you have rsyslog configured to drop privileges, >>> > > > > the >>> > > > lower >>> > > > > privileges need the ability to create the directories) >>> > > > > >>> > > > > >>> > > > > 2. Is DNS caching default enabled or do I have to enable this >>> > > > somewhere >>> > > > >> first...? >>> > > > >> >>> > > > > >>> > > > > I don't know, I haven't had a chance to look into that yet. >>> > > > > >>> > > > > David Lang >>> > > > > >>> > > > > >>> > > > >> Thanks in advance :-) ! >>> > > > >> ~maymann >>> > > > >> >>> > > > >> >>> > > > >> 2012/1/14 >>> > > > >> >>> > > > >> http://rsyslog.com/article60/ >>> > > > >>> >>> > > > >>> David Lang >>> > > > >>> >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > >>> >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> > > > >>> >>> > > > >>>> From: Michael Maymann >>> > > > >>>> To: rsyslog-users , david at lang.hm, >>> > > > >>>> Michael Maymann >>> > > > >>>> >>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>> howto/links/examples >>> > > > >>>> >>> > > > >>>> Hi David, >>> > > > >>>> >>> > > > >>>> thanks for you kind reply...:-) ! >>> > > > >>>> --- >>> > > > >>>> This didn't seem to get through to the archives for some >>> > > > reason...: >>> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>> > > > >>>> >>> > > > January/thread.html< >>> http://lists.adiscon.net/**pipermail/rsyslog/201 >>> > > > 2- >>> > > > **January/thread.html> >>> > > > >>>> >> > > > January/thread.** >>> > > > >>>> html>> > > > January/thread.html> >>> > > > >>>> > >>> > > > >>>> >>> > > > >>>> Hope I will not dobbel-post... >>> > > > >>>> --- >>> > > > >>>> I don't use syslog-relays, so this will not cause me any >>> problems. >>> > > > >>>> Don't actually know what version we are running - can see this >>> > > > Monday >>> > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 >>> > > > >>>> if >>> > > > not >>> > > > >>>> already then. >>> > > > >>>> I have to configure this into a already running live >>> production >>> > > > system - >>> > > > >>>> our previous syslog-admin left...:-(. >>> > > > >>>> Could I perhaps ask you to be so kind as to give an >>> > > > >>>> configuration example of how this is done, if I ask really >>> > > > >>>> nicely... :-) ? >>> > > > >>>> >>> > > > >>>> Thanks in advance :-) ! >>> > > > >>>> ~maymann >>> > > > >>>> >>> > > > >>>> 2012/1/13 >>> > > > >>>> >>> > > > >>>> you need to be aware that doing the DNS queries is rather >>> > > > expensive >>> > > > >>>> >>> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 >>> > > > version >>> > > > >>>>> there >>> > > > >>>>> may now be a DNS cache that will drastically help) >>> > > > >>>>> >>> > > > >>>>> you would need to create a template with FROMHOST in it and >>> > > > >>>>> use >>> > > > that as >>> > > > >>>>> the filename to write to (look for dynafile in the >>> > > > >>>>> documentation) >>> > > > >>>>> >>> > > > >>>>> note that if you are relaying logs from one machine to >>> > > > >>>>> another, >>> > > > only >>> > > > >>>>> the >>> > > > >>>>> first machine will see the true source in FROMHOST, machines >>> > > > after that >>> > > > >>>>> will only see the relay box. >>> > > > >>>>> >>> > > > >>>>> let me know if this doesn't give you enough clues to learn >>> how >>> > > > >>>>> to >>> > > > do >>> > > > >>>>> this. >>> > > > >>>>> >>> > > > >>>>> David Lang >>> > > > >>>>> >>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>> > > > >>>>> >>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> > > > >>>>> >>> > > > >>>>> From: Michael Maymann >>> > > > >>>>>> Reply-To: rsyslog-users >>> > > > >>>>>> To: rsyslog at lists.adiscon.com >>> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>>>> howto/links/examples >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS >>> > > > >>>>>> and >>> > > > not >>> > > > >>>>>> from >>> > > > >>>>>> syslog-info hostname. >>> > > > >>>>>> We are getting a lot of weird logfiles as some applications >>> > > > >>>>>> are >>> > > > not >>> > > > >>>>>> including the hostname as the first parameter in the syslog- >>> > > > entries, >>> > > > >>>>>> e.g.: >>> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>> > > > >>>>>> [set_host_compat_list] >>> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd >>> > "SetUserData >>> > > > >>>>>> HostCompatList text >>> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... >>> > > > >>>>>> >>> > > > >>>>>> Thanks in advance :-) ! >>> > > > >>>>>> ~maymann >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> 2012/1/13 Michael Maymann >>> > > > >>>>>> >>> > > > >>>>>> Hi List, >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>>> I'm new to rsyslog/syslog in general. >>> > > > >>>>>>> >>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. >>> > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a >>> > > > FQDN-dir >>> > > > >>>>>>> containing logs from this host if more logfiles per host >>> are >>> > > > best >>> > > > >>>>>>> practice)... >>> > > > >>>>>>> >>> > > > >>>>>>> Can anyone give me an example of (or link to) best practice >>> > > > >>>>>>> of >>> > > > this >>> > > > >>>>>>> kind >>> > > > >>>>>>> of setup. >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> Thanks in advance :-) ! >>> > > > >>>>>>> >>> > > > >>>>>>> ~maymann >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>>> >>> > > > >>>>>>> rsyslog mailing list >>> > > > >>>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >> > > > **services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>> >>> > > > >>>>>> rsyslog mailing list >>> > > > >>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >> > > > **services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>> >>> > > > >>>>> >>> > > > >>>> >>> > > > >> >>> > > > _______________________________________________ >>> > > > rsyslog mailing list >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > > http://www.rsyslog.com/professional-services/ >>> > > _______________________________________________ >>> > > rsyslog mailing list >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > http://www.rsyslog.com/professional-services/ >>> > > >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com/professional-services/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> >> >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ From michael at maymann.org Thu Jan 19 11:24:13 2012 From: michael at maymann.org (Michael Maymann) Date: Thu, 19 Jan 2012 11:24:13 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: Hi, Got it installed...: yum install zlib-devel.x86_64 yum install byacc.x86_64 did the trick for me... Thanks for your help Rainer...:-) ! Trying to look through the man-page and googling a bit... 1. do I have to enable DNS cache somewhere, or is this feature on by default ? 2. Is there some v6.3.6 version config examples somewhere. I would need to: a. generally have logfiles in a /NFS_logdir/FQDN/equial_to_local_logfile_name (e.g. messages) b. have maillog from mailcluster=host001 (host001a+host001b) being merged in a single /NFS_logdir/host001/maillog file Thanks in advance :-) ! ~maymann 2012/1/18 Rainer Gerhards > Zlib-dev is missing, looks like configure did not catch that. > > Rainer > > Michael Maymann hat geschrieben:The > --libdir=/usr/lib64 option did the trick...:-): > > cd libestr > > ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr > > make > > make install > > cd .. > > cd libee/ > > ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr > > make > > make install > > cd .. > > cd rsyslog-6.3.6 > > ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr > > [root at MyServer rsyslog-6.3.6]# echo $? > > 0 > > [root at MyServer rsyslog-6.3.6]# make > > make all-recursive > > make[1]: Entering directory `/root/rsyslog-6.3.6' > > Making all in doc > > make[2]: Entering directory `/root/rsyslog-6.3.6/doc' > > make[2]: Nothing to be done for `all'. > > make[2]: Leaving directory `/root/rsyslog-6.3.6/doc' > > Making all in runtime > > make[2]: Entering directory `/root/rsyslog-6.3.6/runtime' > > CC librsyslog_la-rsyslog.lo > > In file included from stream.h:72, > > from obj.h:50, > > from rsyslog.h:474, > > from rsyslog.c:63: > > zlibw.h:28:18: error: zlib.h: No such file or directory > > In file included from stream.h:72, > > from obj.h:50, > > from rsyslog.h:474, > > from rsyslog.c:63: > > zlibw.h:32: error: expected ')' before 'strm' > > zlibw.h:33: error: expected ';' before 'int' > > In file included from obj.h:50, > > from rsyslog.h:474, > > from rsyslog.c:63: > > stream.h:123: error: expected specifier-qualifier-list before 'Bytef' > > make[2]: *** [librsyslog_la-rsyslog.lo] Error 1 > > make[2]: Leaving directory `/root/rsyslog-6.3.6/runtime' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/root/rsyslog-6.3.6' > make: *** [all] Error 2 > > So both libestr and linee in installed and ./configure in rsyslog dir ran > without errors, but now make is causing me problems..., anyone who know how > to solve this ? > > > Thanks in advance :-) ! > ~maymann > > 2012/1/17 Michael Maymann > > > Tried to delete all installed and follow this guide: > > http://www.liblognorm.com/help/first-steps-using-liblognorm/ > > same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib > > --includedir=/usr/include: > > ... > > checking for LIBESTR... configure: error: Package requirements (libestr > >= > > 0.0.0) were not met: > > > > No package 'libestr' found > > > > Consider adjusting the PKG_CONFIG_PATH environment variable if you > > installed software in a non-standard prefix. > > > > Alternatively, you may set the environment variables LIBESTR_CFLAGS > > and LIBESTR_LIBS to avoid the need to call pkg-config. > > See the pkg-config man page for more details. > > > > > > Thanks in advance :-) ! > > ~maymann > > > > > > 2012/1/17 Michael Maymann > > > >> Hi Rainer, > >> > >> yes, rsyslog-6.3.6 needs libestr and libee...: > >> I installed libestr just fine: > >> [root at MyServer pkgconfig]# locate libestr > >> /root/libestr-0.1.2 > >> /root/libestr-0.1.2.tar.gz > >> /root/libestr-0.1.2/AUTHORS > >> /root/libestr-0.1.2/COPYING > >> /root/libestr-0.1.2/ChangeLog > >> /root/libestr-0.1.2/INSTALL > >> /root/libestr-0.1.2/Makefile > >> /root/libestr-0.1.2/Makefile.am > >> /root/libestr-0.1.2/Makefile.in > >> /root/libestr-0.1.2/NEWS > >> /root/libestr-0.1.2/README > >> /root/libestr-0.1.2/aclocal.m4 > >> /root/libestr-0.1.2/compile > >> /root/libestr-0.1.2/config.guess > >> /root/libestr-0.1.2/config.h > >> /root/libestr-0.1.2/config.h.in > >> /root/libestr-0.1.2/config.log > >> /root/libestr-0.1.2/config.status > >> /root/libestr-0.1.2/config.sub > >> /root/libestr-0.1.2/configure > >> /root/libestr-0.1.2/configure.ac > >> /root/libestr-0.1.2/depcomp > >> /root/libestr-0.1.2/include > >> /root/libestr-0.1.2/install-sh > >> /root/libestr-0.1.2/libestr.pc > >> /root/libestr-0.1.2/libestr.pc.in > >> /root/libestr-0.1.2/libtool > >> /root/libestr-0.1.2/ltmain.sh > >> /root/libestr-0.1.2/m4 > >> /root/libestr-0.1.2/missing > >> /root/libestr-0.1.2/src > >> /root/libestr-0.1.2/stamp-h1 > >> /root/libestr-0.1.2/include/Makefile > >> /root/libestr-0.1.2/include/Makefile.am > >> /root/libestr-0.1.2/include/Makefile.in > >> /root/libestr-0.1.2/include/libestr.h > >> /root/libestr-0.1.2/m4/libtool.m4 > >> /root/libestr-0.1.2/m4/ltoptions.m4 > >> /root/libestr-0.1.2/m4/ltsugar.m4 > >> /root/libestr-0.1.2/m4/ltversion.m4 > >> /root/libestr-0.1.2/m4/lt~obsolete.m4 > >> /root/libestr-0.1.2/src/.deps > >> /root/libestr-0.1.2/src/.libs > >> /root/libestr-0.1.2/src/Makefile > >> /root/libestr-0.1.2/src/Makefile.am > >> /root/libestr-0.1.2/src/Makefile.in > >> /root/libestr-0.1.2/src/libestr.c > >> /root/libestr-0.1.2/src/libestr.la > >> /root/libestr-0.1.2/src/libestr_la-libestr.lo > >> /root/libestr-0.1.2/src/libestr_la-libestr.o > >> /root/libestr-0.1.2/src/libestr_la-string.lo > >> /root/libestr-0.1.2/src/libestr_la-string.o > >> /root/libestr-0.1.2/src/string.c > >> /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo > >> /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo > >> /root/libestr-0.1.2/src/.libs/libestr.a > >> /root/libestr-0.1.2/src/.libs/libestr.la > >> /root/libestr-0.1.2/src/.libs/libestr.lai > >> /root/libestr-0.1.2/src/.libs/libestr.so > >> /root/libestr-0.1.2/src/.libs/libestr.so.0 > >> /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 > >> /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o > >> /root/libestr-0.1.2/src/.libs/libestr_la-string.o > >> /usr/local/include/libestr.h > >> /usr/local/lib/libestr.a > >> /usr/local/lib/libestr.la > >> /usr/local/lib/libestr.so > >> /usr/local/lib/libestr.so.0 > >> /usr/local/lib/libestr.so.0.0.0 > >> /usr/local/lib/pkgconfig/libestr.pc > >> > >> I try to install libee: > >> [root at MyServer libee-0.3.2]# ./configure > >> LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib > >> checking for a BSD-compatible install... /usr/bin/install -c > >> checking whether build environment is sane... yes > >> checking for a thread-safe mkdir -p... /bin/mkdir -p > >> checking for gawk... gawk > >> checking whether make sets $(MAKE)... yes > >> checking for gcc... gcc > >> checking whether the C compiler works... yes > >> checking for C compiler default output file name... a.out > >> checking for suffix of executables... > >> checking whether we are cross compiling... no > >> checking for suffix of object files... o > >> checking whether we are using the GNU C compiler... yes > >> checking whether gcc accepts -g... yes > >> checking for gcc option to accept ISO C89... none needed > >> checking for style of include used by make... GNU > >> checking dependency style of gcc... gcc3 > >> checking whether gcc and cc understand -c and -o together... yes > >> checking build system type... x86_64-unknown-linux-gnu > >> checking host system type... x86_64-unknown-linux-gnu > >> checking how to print strings... printf > >> checking for a sed that does not truncate output... /bin/sed > >> checking for grep that handles long lines and -e... /bin/grep > >> checking for egrep... /bin/grep -E > >> checking for fgrep... /bin/grep -F > >> checking for ld used by gcc... /usr/bin/ld > >> checking if the linker (/usr/bin/ld) is GNU ld... yes > >> checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B > >> checking the name lister (/usr/bin/nm -B) interface... BSD nm > >> checking whether ln -s works... yes > >> checking the maximum length of command line arguments... 1966080 > >> checking whether the shell understands some XSI constructs... yes > >> checking whether the shell understands "+="... yes > >> checking for /usr/bin/ld option to reload object files... -r > >> checking for objdump... objdump > >> checking how to recognize dependent libraries... pass_all > >> checking for ar... ar > >> checking for strip... strip > >> checking for ranlib... ranlib > >> checking command to parse /usr/bin/nm -B output from gcc object... ok > >> checking how to run the C preprocessor... gcc -E > >> checking for ANSI C header files... yes > >> checking for sys/types.h... yes > >> checking for sys/stat.h... yes > >> checking for stdlib.h... yes > >> checking for string.h... yes > >> checking for memory.h... yes > >> checking for strings.h... yes > >> checking for inttypes.h... yes > >> checking for stdint.h... yes > >> checking for unistd.h... yes > >> checking for dlfcn.h... yes > >> checking for objdir... .libs > >> checking if gcc supports -fno-rtti -fno-exceptions... no > >> checking for gcc option to produce PIC... -fPIC -DPIC > >> checking if gcc PIC flag -fPIC -DPIC works... yes > >> checking if gcc static flag -static works... no > >> checking if gcc supports -c -o file.o... yes > >> checking if gcc supports -c -o file.o... (cached) yes > >> checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports > >> shared libraries... yes > >> checking whether -lc should be explicitly linked in... no > >> checking dynamic linker characteristics... GNU/Linux ld.so > >> checking how to hardcode library paths into programs... immediate > >> checking whether stripping libraries is possible... yes > >> checking if libtool supports shared libraries... yes > >> checking whether to build shared libraries... yes > >> checking whether to build static libraries... yes > >> checking for stdlib.h... (cached) yes > >> checking for GNU libc compatible malloc... yes > >> checking for pkg-config... /usr/bin/pkg-config > >> checking pkg-config is at least version 0.9.0... yes > >> checking for LIBESTR... yes > >> configure: creating ./config.status > >> config.status: creating Makefile > >> config.status: creating libee.pc > >> config.status: creating src/Makefile > >> config.status: creating include/Makefile > >> config.status: creating include/libee/Makefile > >> config.status: creating tests/Makefile > >> config.status: creating config.h > >> config.status: config.h is unchanged > >> config.status: executing depfiles commands > >> config.status: executing libtool commands > >> ***************************************************** > >> libee will be compiled with the following settings: > >> > >> Debug mode enabled: no > >> Testbench enabled: yes > >> [root at MyServer libee-0.3.2]# echo $? > >> 0 > >> [root at MyServer libee-0.3.2]# make > >> make all-recursive > >> make[1]: Entering directory `/root/libee-0.3.2' > >> Making all in tests > >> make[2]: Entering directory `/root/libee-0.3.2/tests' > >> make[2]: Nothing to be done for `all'. > >> make[2]: Leaving directory `/root/libee-0.3.2/tests' > >> Making all in include > >> make[2]: Entering directory `/root/libee-0.3.2/include' > >> Making all in libee > >> make[3]: Entering directory `/root/libee-0.3.2/include/libee' > >> make[3]: Nothing to be done for `all'. > >> make[3]: Leaving directory `/root/libee-0.3.2/include/libee' > >> make[3]: Entering directory `/root/libee-0.3.2/include' > >> make[3]: Nothing to be done for `all-am'. > >> make[3]: Leaving directory `/root/libee-0.3.2/include' > >> make[2]: Leaving directory `/root/libee-0.3.2/include' > >> Making all in src > >> make[2]: Entering directory `/root/libee-0.3.2/src' > >> CC libee_la-ctx.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-tag.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-event.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-value.lo > >> value.c: In function 'ee_newValue': > >> value.c:37: warning: unused parameter 'ctx' > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-tagbucket.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-field.lo > >> field.c: In function 'ee_getFieldValueAsStr': > >> field.c:181: warning: 'str' may be used uninitialized in this function > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-fieldbucket.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-primitivetype.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-int_dec.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-apache_dec.lo > >> apache_dec.c: In function 'ee_newApache': > >> apache_dec.c:37: warning: unused parameter 'ctx' > >> apache_dec.c: In function 'ee_apacheAddName': > >> apache_dec.c:71: warning: unused parameter 'ctx' > >> apache_dec.c: In function 'processLn': > >> apache_dec.c:205: warning: unused variable 'value' > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-syslog_enc.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-json_enc.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-csv_enc.lo > >> csv_enc.c: In function 'ee_AddName': > >> csv_enc.c:66: warning: unused parameter 'ctx' > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-xml_enc.lo > >> xml_enc.c: In function 'ee_addValue_XML': > >> xml_enc.c:60: warning: unused variable 'j' > >> xml_enc.c:59: warning: unused variable 'numbuf' > >> xml_enc.c: At top level: > >> xml_enc.c:40: warning: 'hexdigit' defined but not used > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CCLD libee.la > >> CC convert-convert.o > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CCLD convert > >> /usr/local/lib: file not recognized: Is a directory > >> collect2: ld returned 1 exit status > >> make[2]: *** [convert] Error 1 > >> make[2]: Leaving directory `/root/libee-0.3.2/src' > >> make[1]: *** [all-recursive] Error 1 > >> make[1]: Leaving directory `/root/libee-0.3.2' > >> make: *** [all] Error 2 > >> [root at MyServer libee-0.3.2]# echo $? > >> 2 > >> > >> It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know > >> how to solve this ? > >> > >> > >> > >> Thanks in advance :-) ! > >> ~maymann > >> > >> 2012/1/16 Rainer Gerhards > >> > >>> > -----Original Message----- > >>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > >>> > Sent: Monday, January 16, 2012 10:48 AM > >>> > To: rsyslog-users > >>> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> howto/links/examples > >>> > > >>> > Hi Rainer, > >>> > > >>> > Thanks for your reply. > >>> > How do I install 6.3.6 on RHEL6 easiest ? > >>> > >>> I have no specific instructions. Just grab the sources and compile, I'd > >>> say > >>> ;) Note, however, that you need to install libestr and probably libee > >>> first. > >>> > >>> Rainer > >>> > > >>> > Thanks in advance :-)! > >>> > ~maymann > >>> > > >>> > 2012/1/16 Rainer Gerhards > >>> > > >>> > > The cache is available since 6.3.1, so you need to go for the devel > >>> > > version. > >>> > > A good place to check those things is the ChangeLog itself, here is > >>> > > the current one: > >>> > > > >>> > > > >>> > > > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 > >>> > > 575d085 > >>> > > a0fcf48f71339154813971< > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= > >>> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> > >>> > > ;hb=HEAD > >>> > > > >>> > > Note that v6-devel is almost as stable as v6-stable except for the > >>> > > config read phase at startup. > >>> > > > >>> > > HTH > >>> > > Rainer > >>> > > > >>> > > > -----Original Message----- > >>> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > >>> > > > Sent: Monday, January 16, 2012 8:57 AM > >>> > > > To: david at lang.hm > >>> > > > Cc: rsyslog-users > >>> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> > > > howto/links/examples > >>> > > > > >>> > > > If I want DNS caching, should i use the new stable-6.2.0 or the > >>> > > > older devel-6.3.6... ? > >>> > > > Can see this feature mentioned here: > >>> > > > http://rsyslog.com/features/ > >>> > > > or here: > >>> > > > http://rsyslog.com/project-status/ > >>> > > > > >>> > > > > >>> > > > Thanks in advance :-) ! > >>> > > > ~maymann > >>> > > > > >>> > > > 2012/1/14 > >>> > > > > >>> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > >>> > > > > > >>> > > > > Hi David, > >>> > > > >> > >>> > > > >> thanks for this...this is super info...:-) ! > >>> > > > >> If I have to create different logs per host, will this be the > a > >>> > > > valid > >>> > > > >> configuration: > >>> > > > >> $template > >>> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > >>> > > > >> *.* -?DynaFile_messages > >>> > > > >> $template > >>> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > >>> > > > >> *.* -?DynaFile_secure > >>> > > > >> $template > >>> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > >>> > > > >> *.* -?DynaFile_auth.log > >>> > > > >> > >>> > > > > > >>> > > > > I believe so. > >>> > > > > > >>> > > > > > >>> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or > do > >>> I > >>> > > > have to > >>> > > > >> create every hosts dir upfront... ? > >>> > > > >> > >>> > > > > > >>> > > > > it will create it for you (make sure it's running with the > >>> > > > appropriate > >>> > > > > permissions, if you have rsyslog configured to drop privileges, > >>> > > > > the > >>> > > > lower > >>> > > > > privileges need the ability to create the directories) > >>> > > > > > >>> > > > > > >>> > > > > 2. Is DNS caching default enabled or do I have to enable this > >>> > > > somewhere > >>> > > > >> first...? > >>> > > > >> > >>> > > > > > >>> > > > > I don't know, I haven't had a chance to look into that yet. > >>> > > > > > >>> > > > > David Lang > >>> > > > > > >>> > > > > > >>> > > > >> Thanks in advance :-) ! > >>> > > > >> ~maymann > >>> > > > >> > >>> > > > >> > >>> > > > >> 2012/1/14 > >>> > > > >> > >>> > > > >> http://rsyslog.com/article60/ > >>> > > > >>> > >>> > > > >>> David Lang > >>> > > > >>> > >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > >>> > > > >>> > >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > >>> > > > >>> > >>> > > > >>>> From: Michael Maymann > >>> > > > >>>> To: rsyslog-users , > david at lang.hm, > >>> > > > >>>> Michael Maymann > >>> > > > >>>> > >>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> > > > >>>> howto/links/examples > >>> > > > >>>> > >>> > > > >>>> Hi David, > >>> > > > >>>> > >>> > > > >>>> thanks for you kind reply...:-) ! > >>> > > > >>>> --- > >>> > > > >>>> This didn't seem to get through to the archives for some > >>> > > > reason...: > >>> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > >>> > > > >>>> > >>> > > > January/thread.html< > >>> http://lists.adiscon.net/**pipermail/rsyslog/201 > >>> > > > 2- > >>> > > > **January/thread.html> > >>> > > > >>>> >>> > > > January/thread.** > >>> > > > >>>> html >>> > > > January/thread.html> > >>> > > > >>>> > > >>> > > > >>>> > >>> > > > >>>> Hope I will not dobbel-post... > >>> > > > >>>> --- > >>> > > > >>>> I don't use syslog-relays, so this will not cause me any > >>> problems. > >>> > > > >>>> Don't actually know what version we are running - can see > this > >>> > > > Monday > >>> > > > >>>> morning though... Thanks for this hint... will upgrade to > 6.2 > >>> > > > >>>> if > >>> > > > not > >>> > > > >>>> already then. > >>> > > > >>>> I have to configure this into a already running live > >>> production > >>> > > > system - > >>> > > > >>>> our previous syslog-admin left...:-(. > >>> > > > >>>> Could I perhaps ask you to be so kind as to give an > >>> > > > >>>> configuration example of how this is done, if I ask really > >>> > > > >>>> nicely... :-) ? > >>> > > > >>>> > >>> > > > >>>> Thanks in advance :-) ! > >>> > > > >>>> ~maymann > >>> > > > >>>> > >>> > > > >>>> 2012/1/13 > >>> > > > >>>> > >>> > > > >>>> you need to be aware that doing the DNS queries is rather > >>> > > > expensive > >>> > > > >>>> > >>> > > > >>>>> (although I think I saw a comment that in the very latest > 6.2 > >>> > > > version > >>> > > > >>>>> there > >>> > > > >>>>> may now be a DNS cache that will drastically help) > >>> > > > >>>>> > >>> > > > >>>>> you would need to create a template with FROMHOST in it and > >>> > > > >>>>> use > >>> > > > that as > >>> > > > >>>>> the filename to write to (look for dynafile in the > >>> > > > >>>>> documentation) > >>> > > > >>>>> > >>> > > > >>>>> note that if you are relaying logs from one machine to > >>> > > > >>>>> another, > >>> > > > only > >>> > > > >>>>> the > >>> > > > >>>>> first machine will see the true source in FROMHOST, > machines > >>> > > > after that > >>> > > > >>>>> will only see the relay box. > >>> > > > >>>>> > >>> > > > >>>>> let me know if this doesn't give you enough clues to learn > >>> how > >>> > > > >>>>> to > >>> > > > do > >>> > > > >>>>> this. > >>> > > > >>>>> > >>> > > > >>>>> David Lang > >>> > > > >>>>> > >>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > >>> > > > >>>>> > >>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > >>> > > > >>>>> > >>> > > > >>>>> From: Michael Maymann > >>> > > > >>>>>> Reply-To: rsyslog-users > >>> > > > >>>>>> To: rsyslog at lists.adiscon.com > >>> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> > > > >>>>>> howto/links/examples > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from > DNS > >>> > > > >>>>>> and > >>> > > > not > >>> > > > >>>>>> from > >>> > > > >>>>>> syslog-info hostname. > >>> > > > >>>>>> We are getting a lot of weird logfiles as some > applications > >>> > > > >>>>>> are > >>> > > > not > >>> > > > >>>>>> including the hostname as the first parameter in the > syslog- > >>> > > > entries, > >>> > > > >>>>>> e.g.: > >>> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > >>> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > >>> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > >>> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > >>> > > > >>>>>> [set_host_compat_list] > >>> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command > is > >>> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd > >>> > "SetUserData > >>> > > > >>>>>> HostCompatList text > >>> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... > >>> > > > >>>>>> > >>> > > > >>>>>> Thanks in advance :-) ! > >>> > > > >>>>>> ~maymann > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> 2012/1/13 Michael Maymann > >>> > > > >>>>>> > >>> > > > >>>>>> Hi List, > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>>> I'm new to rsyslog/syslog in general. > >>> > > > >>>>>>> > >>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. > >>> > > > >>>>>>> Preferably I would like a FQDN.log file for each host > (or a > >>> > > > FQDN-dir > >>> > > > >>>>>>> containing logs from this host if more logfiles per host > >>> are > >>> > > > best > >>> > > > >>>>>>> practice)... > >>> > > > >>>>>>> > >>> > > > >>>>>>> Can anyone give me an example of (or link to) best > practice > >>> > > > >>>>>>> of > >>> > > > this > >>> > > > >>>>>>> kind > >>> > > > >>>>>>> of setup. > >>> > > > >>>>>>> > >>> > > > >>>>>>> > >>> > > > >>>>>>> Thanks in advance :-) ! > >>> > > > >>>>>>> > >>> > > > >>>>>>> ~maymann > >>> > > > >>>>>>> > >>> > > > >>>>>>> > >>> > ______________________________******_________________ > >>> > > > >>>>>>> > >>> > > > >>>>>>> rsyslog mailing list > >>> > > > >>>>>> > >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< > >>> http://lists > >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> > >>> > > > >>>>>> > >>> > > > >>> http://list > >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> > >>> > > > >>>>>> > > >>> > > > >>>>>> > >>> > > > >>> http://list > >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> > >>> > > > >>>>>> > >>> > > > >>> http://lists. > >>> > > > adi > >>> > > > scon.net/mailman/listinfo/rsyslog> > >>> > > > >>>>>> > > >>> > > > >>>>>> > >>> > > > >>>>>>> > >>> > > > >>>>>>> http://www.rsyslog.com/******professional- > >>> > > > services/ > >>> > > > >>>>>> >>> > > > services/ > >>> > > > >>>>>> > > >>> > > > >>>>>> >>> > > > **services/ > >>> > > > >>>>>> >>> > > > services/ > >>> > > > >>>>>> > > >>> > > > >>>>>> > >>> > > > >>>>>>> > >>> > > > >>>>>>> > >>> > > > >>>>>> > >>> > ______________________________******_________________ > >>> > > > >>>>>> > >>> > > > >>>>>> rsyslog mailing list > >>> > > > >>>>> > >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< > >>> http://lists > >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> > >>> > > > >>>>> > >>> > > > >>> http://list > >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> > >>> > > > >>>>> > > >>> > > > >>>>> > >>> > > > >>> http://list > >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> > >>> > > > >>>>> > >>> > > > >>> http://lists. > >>> > > > adi > >>> > > > scon.net/mailman/listinfo/rsyslog> > >>> > > > >>>>> > > >>> > > > >>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> http://www.rsyslog.com/******professional- > >>> > > > services/ > >>> > > > >>>>> >>> > > > services/ > >>> > > > >>>>> > > >>> > > > >>>>> >>> > > > **services/ > >>> > > > >>>>> >>> > > > services/ > >>> > > > >>>>> > > >>> > > > >>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>> > >>> > > > >>>>> > >>> > > > >>>> > >>> > > > >> > >>> > > > _______________________________________________ > >>> > > > rsyslog mailing list > >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > > > http://www.rsyslog.com/professional-services/ > >>> > > _______________________________________________ > >>> > > rsyslog mailing list > >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > > http://www.rsyslog.com/professional-services/ > >>> > > > >>> > _______________________________________________ > >>> > rsyslog mailing list > >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> > http://www.rsyslog.com/professional-services/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >>> > >> > >> > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From david at lang.hm Thu Jan 19 11:32:33 2012 From: david at lang.hm (david at lang.hm) Date: Thu, 19 Jan 2012 02:32:33 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: On Thu, 19 Jan 2012, Michael Maymann wrote: > Hi, > > Got it installed...: > yum install zlib-devel.x86_64 > yum install byacc.x86_64 > did the trick for me... > Thanks for your help Rainer...:-) ! > > Trying to look through the man-page and googling a bit... > 1. do I have to enable DNS cache somewhere, or is this feature on by > default ? I don't know > 2. Is there some v6.3.6 version config examples somewhere. I would need to: > a. generally have logfiles in a > /NFS_logdir/FQDN/equial_to_local_logfile_name (e.g. messages) > b. have maillog from mailcluster=host001 (host001a+host001b) being merged > in a single /NFS_logdir/host001/maillog file unless you are needing some 6.x specific feature, the configs from earlier versions will still work. David Lang From michael at maymann.org Thu Jan 19 12:12:50 2012 From: michael at maymann.org (Michael Maymann) Date: Thu, 19 Jan 2012 12:12:50 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: Hi David, Thanks for your reply...:-) ! 2012/1/19 > On Thu, 19 Jan 2012, Michael Maymann wrote: > > Hi, >> >> Got it installed...: >> yum install zlib-devel.x86_64 >> yum install byacc.x86_64 >> did the trick for me... >> Thanks for your help Rainer...:-) ! >> >> Trying to look through the man-page and googling a bit... >> 1. do I have to enable DNS cache somewhere, or is this feature on by >> default ? >> > > I don't know Ok, anyone who knows this ? > > > 2. Is there some v6.3.6 version config examples somewhere. I would need >> to: >> a. generally have logfiles in a >> /NFS_logdir/FQDN/equial_to_**local_logfile_name (e.g. messages) >> b. have maillog from mailcluster=host001 (host001a+host001b) being merged >> in a single /NFS_logdir/host001/maillog file >> > > unless you are needing some 6.x specific feature, the configs from earlier > versions will still work. > I'm don't think I do...:-) - besides the DNS cache thing. But I'm new to syslog/rsyslog and wondering if someone has a link/example to something similar to what I describe above. > > David Lang > > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From f.sordillo at cineca.it Thu Jan 19 14:59:31 2012 From: f.sordillo at cineca.it (Francesco Sordillo) Date: Thu, 19 Jan 2012 14:59:31 +0100 Subject: [rsyslog] rsyslog and apache Message-ID: <4F1821C3.8090200@cineca.it> Hi to all. I configured Apache to log with logger and rsyslog. In Apache I have -------- ErrorLog "|/bin/logger -p local5.info -t http_myservice-error_log" CustomLog "|/bin/logger -p local5.info -t http_myservice-access_log" common ------- and my rsyslog.conf ------- #Log apache $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%.log" :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog :syslogtag,startswith,"http" ~ ------- So apache log are created inside /vag/log/httpd/ directory and it works fine! But _all_ the files for each service are created within the directory /var/log/httpd/. So now, I need to create a specific directory inside /var/log/http/ for each service configured inside apache. For example: - /var/log/httpd/service1/ (error and access log) - /var/log/httpd/service2/ (error and access log) - /var/log/httpd/service3/ (error and access log) and so on... I'm a new with this tool, so how can I configure rsyslog and apache to do that? Note that the number of services is not predictable. Anyone can help me? Thanks a lot! Francesco. From david at lang.hm Fri Jan 20 08:47:24 2012 From: david at lang.hm (david at lang.hm) Date: Thu, 19 Jan 2012 23:47:24 -0800 (PST) Subject: [rsyslog] rsyslog and apache In-Reply-To: <4F1821C3.8090200@cineca.it> References: <4F1821C3.8090200@cineca.it> Message-ID: On Thu, 19 Jan 2012, Francesco Sordillo wrote: > Hi to all. > > I configured Apache to log with logger and rsyslog. In Apache I have > > -------- > ErrorLog "|/bin/logger -p local5.info -t http_myservice-error_log" > CustomLog "|/bin/logger -p local5.info -t http_myservice-access_log" common > ------- > > and my rsyslog.conf > > ------- > #Log apache > $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" > $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%.log" > > :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog > :syslogtag,startswith,"http" ~ > ------- > > So apache log are created inside /vag/log/httpd/ directory and it works fine! > But _all_ the files for each service are created within the directory > /var/log/httpd/. > So now, I need to create a specific directory inside /var/log/http/ for each > service configured inside apache. For example: > - /var/log/httpd/service1/ (error and access log) > - /var/log/httpd/service2/ (error and access log) > - /var/log/httpd/service3/ (error and access log) > and so on... > > I'm a new with this tool, so how can I configure rsyslog and apache to do > that? Note that the number of services is not predictable. > Anyone can help me? change your template to have a variable portion in the path, not just in the file. for example $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%/file.log" would create a directory for each thing that was previously a file and in that directory would be a file called "file.log" if you want both the directory and filename to be variable, just defien the appropriate variables in each. David Lang From f.sordillo at cineca.it Fri Jan 20 10:28:23 2012 From: f.sordillo at cineca.it (Francesco Sordillo) Date: Fri, 20 Jan 2012 10:28:23 +0100 Subject: [rsyslog] rsyslog and apache In-Reply-To: References: <4F1821C3.8090200@cineca.it> Message-ID: <4F1933B7.10802@cineca.it> Il 20/01/2012 08:47, david at lang.hm ha scritto: > On Thu, 19 Jan 2012, Francesco Sordillo wrote: > >> Hi to all. >> >> I configured Apache to log with logger and rsyslog. In Apache I have >> >> -------- >> ErrorLog "|/bin/logger -p local5.info -t http_myservice-error_log" >> CustomLog "|/bin/logger -p local5.info -t http_myservice-access_log" >> common >> ------- >> >> and my rsyslog.conf >> >> ------- >> #Log apache >> $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" >> $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%.log" >> >> :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog >> :syslogtag,startswith,"http" ~ >> ------- >> >> So apache log are created inside /vag/log/httpd/ directory and it >> works fine! But _all_ the files for each service are created within >> the directory /var/log/httpd/. >> So now, I need to create a specific directory inside /var/log/http/ >> for each service configured inside apache. For example: >> - /var/log/httpd/service1/ (error and access log) >> - /var/log/httpd/service2/ (error and access log) >> - /var/log/httpd/service3/ (error and access log) >> and so on... >> >> I'm a new with this tool, so how can I configure rsyslog and apache to >> do that? Note that the number of services is not predictable. >> Anyone can help me? > > change your template to have a variable portion in the path, not just in > the file. > > for example > > $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%/file.log" > > would create a directory for each thing that was previously a file and > in that directory would be a file called "file.log" > > if you want both the directory and filename to be variable, just defien > the appropriate variables in each. Hi David. Thanks a lot for your reply. Could you please explain the last suggestion? How can I have error.log and access.log defining two variable? Thanks. Francesco. From f.sordillo at cineca.it Fri Jan 20 15:28:41 2012 From: f.sordillo at cineca.it (Francesco Sordillo) Date: Fri, 20 Jan 2012 15:28:41 +0100 Subject: [rsyslog] rsyslog and apache In-Reply-To: References: <4F1821C3.8090200@cineca.it> Message-ID: <4F197A19.6040107@cineca.it> Il 20/01/2012 08:47, david at lang.hm ha scritto: > On Thu, 19 Jan 2012, Francesco Sordillo wrote: > >> Hi to all. >> >> I configured Apache to log with logger and rsyslog. In Apache I have >> >> -------- >> ErrorLog "|/bin/logger -p local5.info -t http_myservice-error_log" >> CustomLog "|/bin/logger -p local5.info -t http_myservice-access_log" >> common >> ------- >> >> and my rsyslog.conf >> >> ------- >> #Log apache >> $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" >> $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%.log" >> >> :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog >> :syslogtag,startswith,"http" ~ >> ------- >> >> So apache log are created inside /vag/log/httpd/ directory and it >> works fine! But _all_ the files for each service are created within >> the directory /var/log/httpd/. >> So now, I need to create a specific directory inside /var/log/http/ >> for each service configured inside apache. For example: >> - /var/log/httpd/service1/ (error and access log) >> - /var/log/httpd/service2/ (error and access log) >> - /var/log/httpd/service3/ (error and access log) >> and so on... >> >> I'm a new with this tool, so how can I configure rsyslog and apache to >> do that? Note that the number of services is not predictable. >> Anyone can help me? > > change your template to have a variable portion in the path, not just in > the file. > > for example > > $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%/file.log" > > would create a directory for each thing that was previously a file and > in that directory would be a file called "file.log" > > if you want both the directory and filename to be variable, just defien > the appropriate variables in each. > > David Lang I think I got a way out... using parsing of syslog tag. I changed my apache configuration as ---------- ErrorLog "|/bin/logger -p local5.info -t service1_http_www.service1.it_error_log" CustomLog "|/bin/logger -p local5.info -t service1_http_www.service1.it_access_log" common ---------- and in rsyslog.conf I used properties replacer to create service directory inside /var/log/httpd/ and error.log and access.log --------- $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" /%syslogtag:F,95:2+%" $template ArchiveApache,"/var/log/httpd/%syslogtag:F,95:1%/%syslogtag:R,ERE,0,FIELD:http(.*)[^\:]--end%" :syslogtag,contains,"http" -?ArchiveApache;ApacheLog :syslogtag,contains,"http" ~ ---------- Looks at: - "%syslogtag:F,95:1%": it splits the syslog tag extracting the "service1" name (to create /var/log/https/service1 directory) - "%syslogtag:R,ERE,0,FIELD:http(.*)[^\:]--end%": it splits the syslog tag removing "service1" from the begin of the string and ":" character from the end (to create http_www.service1.it_error.log or access.log). What do yuo think about? Is there a more elegant solution? Thanks a lot. Francesco From dj.turkmaster at gmail.com Fri Jan 20 16:22:15 2012 From: dj.turkmaster at gmail.com (=?ISO-8859-9?Q?alptugay_de=F0irmencio=F0lu?=) Date: Fri, 20 Jan 2012 17:22:15 +0200 Subject: [rsyslog] How to accept tls messages from all peers Message-ID: Hello all, I have one server and multiple clients sending messages to the server using tls. I dont know the clients' hostnames and they are not on my network, they are across the internet. So I want my rsyslog server to accept all incoming tls messages. What should I enter to the server's InputTCPServerStreamDriverPermittedPeer parameter? And also do I have to create a private key and certificate for each client. I am asking this because in this link it says I have to http://rsyslog.com/doc/tls_cert_machine.html But in this link http://rsyslog.com/doc/rsyslog_tls.html it says that: The client setup is equally simple. You need less certificates, just the CA cert. So it means that I just need to copy ca.cert to each client but not create any certificate for them, which I think is the right approach. But the first link confuses me. If the first link is true and I need to create a certificate and private key for each clients what do I have to enter Common name: Enter the dnsName of the subject of the certificate: To these questions. As I said I don't and can't know their hostnames. Thanks for reading From david at lang.hm Fri Jan 20 20:22:10 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 20 Jan 2012 11:22:10 -0800 (PST) Subject: [rsyslog] rsyslog and apache In-Reply-To: <4F197A19.6040107@cineca.it> References: <4F1821C3.8090200@cineca.it> <4F197A19.6040107@cineca.it> Message-ID: On Fri, 20 Jan 2012, Francesco Sordillo wrote: > Il 20/01/2012 08:47, david at lang.hm ha scritto: >> On Thu, 19 Jan 2012, Francesco Sordillo wrote: >> >>> Hi to all. >>> >>> I configured Apache to log with logger and rsyslog. In Apache I have >>> >>> -------- >>> ErrorLog "|/bin/logger -p local5.info -t http_myservice-error_log" >>> CustomLog "|/bin/logger -p local5.info -t http_myservice-access_log" >>> common >>> ------- >>> >>> and my rsyslog.conf >>> >>> ------- >>> #Log apache >>> $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" >>> $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%.log" >>> >>> :syslogtag,startswith,"http" -?ArchiveApache;ApacheLog >>> :syslogtag,startswith,"http" ~ >>> ------- >>> >>> So apache log are created inside /vag/log/httpd/ directory and it >>> works fine! But _all_ the files for each service are created within >>> the directory /var/log/httpd/. >>> So now, I need to create a specific directory inside /var/log/http/ >>> for each service configured inside apache. For example: >>> - /var/log/httpd/service1/ (error and access log) >>> - /var/log/httpd/service2/ (error and access log) >>> - /var/log/httpd/service3/ (error and access log) >>> and so on... >>> >>> I'm a new with this tool, so how can I configure rsyslog and apache to >>> do that? Note that the number of services is not predictable. >>> Anyone can help me? >> >> change your template to have a variable portion in the path, not just in >> the file. >> >> for example >> >> $template ArchiveApache,"/var/log/httpd/%syslogtag:F,58:1%/file.log" >> >> would create a directory for each thing that was previously a file and >> in that directory would be a file called "file.log" >> >> if you want both the directory and filename to be variable, just defien >> the appropriate variables in each. >> >> David Lang > > I think I got a way out... using parsing of syslog tag. I changed my apache > configuration as > > ---------- > ErrorLog "|/bin/logger -p local5.info -t > service1_http_www.service1.it_error_log" > CustomLog "|/bin/logger -p local5.info -t > service1_http_www.service1.it_access_log" common > ---------- > > and in rsyslog.conf I used properties replacer to create service directory > inside /var/log/httpd/ and error.log and access.log > > --------- > $template ApacheLog,"%msg:2:$:drop-last-lf%\r\n" > /%syslogtag:F,95:2+%" > $template > ArchiveApache,"/var/log/httpd/%syslogtag:F,95:1%/%syslogtag:R,ERE,0,FIELD:http(.*)[^\:]--end%" > > :syslogtag,contains,"http" -?ArchiveApache;ApacheLog > :syslogtag,contains,"http" ~ > ---------- > > Looks at: > - "%syslogtag:F,95:1%": it splits the syslog tag extracting the "service1" > name (to create /var/log/https/service1 directory) > - "%syslogtag:R,ERE,0,FIELD:http(.*)[^\:]--end%": it splits the syslog tag > removing "service1" from the begin of the string and ":" character from the > end (to create http_www.service1.it_error.log or access.log). > > What do yuo think about? Is there a more elegant solution? > Thanks a lot. this is the right approach. David Lang From dj.turkmaster at gmail.com Sat Jan 21 09:55:16 2012 From: dj.turkmaster at gmail.com (=?ISO-8859-9?Q?alptugay_de=F0irmencio=F0lu?=) Date: Sat, 21 Jan 2012 10:55:16 +0200 Subject: [rsyslog] How to accept tls messages from all peers In-Reply-To: References: Message-ID: Okay I have made some progress but I still need some help, I have created the ssl certificates and my rsyslog.conf is like this: *Client rsyslog.conf:* $DefaultNetstreamDriver gtls $DefaultNetstreamDriverCAFile /root/ssl/ca.pem $DefaultNetstreamDriverCertFile /root/ssl/cert.pem $DefaultNetstreamDriverKeyFile /root/ssl/key.pem $ActionSendStreamDriverAuthMode anon $ActionSendStreamDriverMode 1 :rawmsg, contains , "dansguardian" @@server.myhostname.com:7890;Alpformat And to the client's /etc/hosts file I have added the server's ip adress and hostname entry 192.168.2.207 server.myhostname.com (this is something I can do, but I cannot add the client's ip to the server's hostfile) *Server rsyslog.conf* # make gtls driver the default $DefaultNetstreamDriver gtls # certificate files $DefaultNetstreamDriverCAFile /root/ssl/ca.pem $DefaultNetstreamDriverCertFile /root/ssl/cert.pem $DefaultNetstreamDriverKeyFile /root/ssl/key.pem $InputTCPServerStreamDriverAuthMode anon # client is NOT authenticated $InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode $InputTCPServerRun 7890 $template FileFormat,"%rawmsg%\n" $template webfile,"/var/log/webfilter-%fromhost-ip%.log" :rawmsg, contains , "dansguardian" -?webfile;FileFormat And I get an error message on the clients rsyslog debug log: 5402.926211647:b4da8b90: -------- calling select, active fds (max 6): 5 6 5406.181947207:b4da8b90: hasRcvInBuffer on nsd 0x942eb50: pszRcvBuf (nil), lenRcvBuf 0 5406.181989083:b4da8b90: hasRcvInBuffer on nsd 0x9430330: pszRcvBuf (nil), lenRcvBuf 0 5406.182000181:b4da8b90: tcpsrv: ready to process 1 event entries 5406.182009712:b4da8b90: tcpsrv: processing item 1, pUsr 0x942d398, bAbortConn 5406.182018873:b4da8b90: New connect on NSD 0x942dc10. 5406.182056336:b4da8b90: dnscache: entry (nil) found *5406.246851507:b4da8b90: Host name for your address (192.168.0.156) unknown 5406.246911619:b4da8b90: tcpsrv: error -2020 during accept* 5406.246936472:b4da8b90: hasRcvInBuffer on nsd 0x942eb50: pszRcvBuf (nil), lenRcvBuf 0 5406.246947385:b4da8b90: hasRcvInBuffer on nsd 0x9430330: pszRcvBuf (nil), lenRcvBuf 0 And also in the server debug log: 5444.008547267:b6120b90: GTLS CA file: '/root/ssl/ca.pem' 5444.009915890:b6120b90: source file nsdsel_gtls.c requested reference for module 'lmnsd_ptcp', reference count now 2 5444.009927732:b6120b90: module lmnsd_gtls of type 2 being loaded. 5444.009933381:b6120b90: entry point 'isCompatibleWithFeature' not present in module 5444.009938112:b6120b90: entry point 'getModCnfName' not present in module 5444.009942681:b6120b90: entry point 'beginCnfLoad' not present in module 5444.009948825:b6120b90: source file netstrms.c requested reference for module 'lmnsd_gtls', reference count now 1 *5444.079660215:b6120b90: unexpected GnuTLS error -9 in nsd_gtls.c:1624: A TLS packet with unexpected length was received.* 5444.079774753:b6120b90: file netstrms.c released module 'lmnsd_gtls', reference count now 0 5444.079781678:b6120b90: module 'lmnsd_gtls' has zero reference count, unloading... 5444.079786859:b6120b90: Unloading module lmnsd_gtls But when I add the client's hostname and ip to the server's hosts file (wihch normally I cannot but for testing purposes I can) 192.168.0.156 machine.example.net I get no errors and messages are delivered succesfully. As I said normally I will not be able to know the clients' dns names they may not even have a dns record. It is strange rsyslog looks for a hostname in tls connection. Why would that be needed? Are there any solutions any workarounds? Am I missing something? 2012/1/20 alptugay de?irmencio?lu > Hello all, > I have one server and multiple clients sending messages to the server > using tls. I dont know the clients' hostnames and they are not on my > network, they are across the internet. So I want my rsyslog server to > accept all incoming tls messages. What should I enter to the server's > InputTCPServerStreamDriverPermittedPeer > parameter? > > And also do I have to create a private key and certificate for each > client. I am asking this because in this link it says I have to > http://rsyslog.com/doc/tls_cert_machine.html > > But in this link > http://rsyslog.com/doc/rsyslog_tls.html > it says that: > The client setup is equally simple. You need less certificates, just the > CA cert. > > So it means that I just need to copy ca.cert to each client but not create > any certificate for them, which I think is the right approach. But the > first link confuses me. > > If the first link is true and I need to create a certificate and private > key for each clients what do I have to enter > Common name: > > Enter the dnsName of the subject of the certificate: > > To these questions. As I said I don't and can't know their hostnames. > > > Thanks for reading > From alexc223 at googlemail.com Sat Jan 21 18:56:37 2012 From: alexc223 at googlemail.com (Alex Cartwright) Date: Sat, 21 Jan 2012 17:56:37 +0000 Subject: [rsyslog] How to stop 'user' facility printing to console Message-ID: Hello, I have a few web applications that are logging to rsyslog/syslog using the 'user' facility on Debian 6, which by default is configured to log these to the file /var/log/user.log This is working just fine, however these messages are also being printed to the console of every user that is currently logged in (via SSH). I'm really struggling to figure out how I can stop this behaviour. Is it anything to do with the "kernel.printk" level? This is the current value of it: $ sysctl kernel.printk kernel.printk = 7 4 1 7 Kind regards Alex From ar at xlrs.de Sat Jan 21 20:19:01 2012 From: ar at xlrs.de (Axel) Date: Sat, 21 Jan 2012 20:19:01 +0100 Subject: [rsyslog] rsyslog and apache In-Reply-To: <4F1821C3.8090200@cineca.it> References: <4F1821C3.8090200@cineca.it> Message-ID: <201201212019.01971.ar@xlrs.de> Am Donnerstag 19 Januar 2012, 14:59:31 schrieb Francesco Sordillo: > Hi to all. > > I configured Apache to log with logger and rsyslog. In Apache I have > > -------- > ErrorLog "|/bin/logger -p local5.info -t http_myservice-error_log" > CustomLog "|/bin/logger -p local5.info -t http_myservice-access_log" common > ------- When you use different facilities there's no need to parse and split the syslogtag. Just a suggestion... Regards, Axel -- Never argue with an idiot; people watching may not tell the difference From david at lang.hm Sun Jan 22 03:02:16 2012 From: david at lang.hm (david at lang.hm) Date: Sat, 21 Jan 2012 18:02:16 -0800 (PST) Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: References: Message-ID: On Sat, 21 Jan 2012, Alex Cartwright wrote: > Hello, > > I have a few web applications that are logging to rsyslog/syslog using > the 'user' facility on Debian 6, which by default is configured to log > these to the file /var/log/user.log > > This is working just fine, however these messages are also being > printed to the console of every user that is currently logged in (via > SSH). I'm really struggling to figure out how I can stop this > behaviour. Is it anything to do with the "kernel.printk" level? This > is the current value of it: > > $ sysctl kernel.printk > kernel.printk = 7 4 1 7 no, this has nothing to do with the kernel.printk config. probably what is happening is the severity is set to critical or something like that. We would need to see your rsyslog.conf (and whatever files it includes) to find what's configured to be sent to the logged in users. David Lang From alexc223 at googlemail.com Sun Jan 22 11:24:33 2012 From: alexc223 at googlemail.com (Alex Cartwright) Date: Sun, 22 Jan 2012 10:24:33 +0000 Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: References: Message-ID: Here is my configuration for rsyslog, https://gist.github.com/1656491 I can see that emergencies are being sent to all users, however that priority is never used. Regards, Alex On 22 January 2012 02:02, wrote: > On Sat, 21 Jan 2012, Alex Cartwright wrote: > >> Hello, >> >> I have a few web applications that are logging to rsyslog/syslog using >> the 'user' facility on Debian 6, which by default is configured to log >> these to the file /var/log/user.log >> >> This is working just fine, however these messages are also being >> printed to the console of every user that is currently logged in (via >> SSH). I'm really struggling to figure out how I can stop this >> behaviour. Is it anything to do with the "kernel.printk" level? This >> is the current value of it: >> >> $ sysctl kernel.printk >> kernel.printk = 7 ? ? ? 4 ? ? ? 1 ? ? ? 7 > > > no, this has nothing to do with the kernel.printk config. probably what is > happening is the severity is set to critical or something like that. We > would need to see your rsyslog.conf (and whatever files it includes) to find > what's configured to be sent to the logged in users. > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From david at lang.hm Sun Jan 22 22:16:31 2012 From: david at lang.hm (david at lang.hm) Date: Sun, 22 Jan 2012 13:16:31 -0800 (PST) Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: References: Message-ID: what priority is being set on th logs that you are seeing for these 'user' facility logs? one thing that you can do is to setup a filter to log the types of messages that you are seeing with the format RSYSLOG_DEBUG so that you can see the facility, severity, etc that the log messages have. David Lang On Sun, 22 Jan 2012, Alex Cartwright wrote: > Here is my configuration for rsyslog, https://gist.github.com/1656491 > > I can see that emergencies are being sent to all users, however that > priority is never used. > > Regards, > Alex > > On 22 January 2012 02:02, wrote: >> On Sat, 21 Jan 2012, Alex Cartwright wrote: >> >>> Hello, >>> >>> I have a few web applications that are logging to rsyslog/syslog using >>> the 'user' facility on Debian 6, which by default is configured to log >>> these to the file /var/log/user.log >>> >>> This is working just fine, however these messages are also being >>> printed to the console of every user that is currently logged in (via >>> SSH). I'm really struggling to figure out how I can stop this >>> behaviour. Is it anything to do with the "kernel.printk" level? This >>> is the current value of it: >>> >>> $ sysctl kernel.printk >>> kernel.printk = 7 ? ? ? 4 ? ? ? 1 ? ? ? 7 >> >> >> no, this has nothing to do with the kernel.printk config. probably what is >> happening is the severity is set to critical or something like that. We >> would need to see your rsyslog.conf (and whatever files it includes) to find >> what's configured to be sent to the logged in users. >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From xuan_zhong at hotmail.com Tue Jan 24 01:58:08 2012 From: xuan_zhong at hotmail.com (Xuan Zhong) Date: Tue, 24 Jan 2012 00:58:08 +0000 Subject: [rsyslog] multiple snmp trap destinations? Message-ID: Hi, Does anyone know how to configure multiple snmp trap destinations in rsyslog.conf? $actionsnmptarget seems only accepting one IP address. Thanks Xuan From david at lang.hm Tue Jan 24 02:17:02 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 23 Jan 2012 17:17:02 -0800 (PST) Subject: [rsyslog] multiple snmp trap destinations? In-Reply-To: References: Message-ID: I haven't tried it, but knowing how rsyslog tends to work, the way to do this would be to have multiple sections $actionsnmptarget= *.* $actionsnmptarget = *.* David Lang On Tue, 24 Jan 2012, Xuan Zhong wrote: > Hi, > > Does anyone know how to configure multiple snmp trap destinations in rsyslog.conf? > $actionsnmptarget seems only accepting one IP address. From rgerhards at hq.adiscon.com Tue Jan 24 08:47:32 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 24 Jan 2012 08:47:32 +0100 Subject: [rsyslog] multiple snmp trap destinations? In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817B2@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Tuesday, January 24, 2012 2:17 AM > To: rsyslog-users > Subject: Re: [rsyslog] multiple snmp trap destinations? > > I haven't tried it, but knowing how rsyslog tends to work, the way to > do > this would be to have multiple sections > $actionsnmptarget= > *.* > $actionsnmptarget = > *.* Yup, that should work (if not, its probably a bug). rainer > > David Lang > > > On Tue, 24 Jan 2012, Xuan Zhong wrote: > > > Hi, > > > > Does anyone know how to configure multiple snmp trap destinations in > rsyslog.conf? > > $actionsnmptarget seems only accepting one IP address. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From alexc223 at gmail.com Tue Jan 24 17:00:38 2012 From: alexc223 at gmail.com (Alex Cartwright) Date: Tue, 24 Jan 2012 16:00:38 +0000 Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: References: Message-ID: This is the output from rsyslog debug: https://gist.github.com/1670748 I think I have found the problem, and it appears to be a bug in the web app. It is logging with a priority/level of 8, which unless I am mistaken simply does not exist. However I still can't see where in the rsyslog.conf that it says to print to the console - can you? I'll get this fixed up in the app, but at least I've found the problem! Many thanks, Alex On 22 January 2012 21:16, wrote: > what priority is being set on th logs that you are seeing for these 'user' > facility logs? > > one thing that you can do is to setup a filter to log the types of messages > that you are seeing with the format RSYSLOG_DEBUG so that you can see the > facility, severity, etc that the log messages have. > > David Lang > > > On Sun, 22 Jan 2012, Alex Cartwright wrote: > >> Here is my configuration for rsyslog, https://gist.github.com/1656491 >> >> I can see that emergencies are being sent to all users, however that >> priority is never used. >> >> Regards, >> Alex >> >> On 22 January 2012 02:02, ? wrote: >>> >>> On Sat, 21 Jan 2012, Alex Cartwright wrote: >>> >>>> Hello, >>>> >>>> I have a few web applications that are logging to rsyslog/syslog using >>>> the 'user' facility on Debian 6, which by default is configured to log >>>> these to the file /var/log/user.log >>>> >>>> This is working just fine, however these messages are also being >>>> printed to the console of every user that is currently logged in (via >>>> SSH). I'm really struggling to figure out how I can stop this >>>> behaviour. Is it anything to do with the "kernel.printk" level? This >>>> is the current value of it: >>>> >>>> $ sysctl kernel.printk >>>> kernel.printk = 7 ? ? ? 4 ? ? ? 1 ? ? ? 7 >>> >>> >>> >>> no, this has nothing to do with the kernel.printk config. probably what >>> is >>> happening is the severity is set to critical or something like that. We >>> would need to see your rsyslog.conf (and whatever files it includes) to >>> find >>> what's configured to be sent to the logged in users. >>> >>> David Lang >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >> >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Tue Jan 24 17:03:43 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 24 Jan 2012 17:03:43 +0100 Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817C6@GRFEXC.intern.adiscon.com> Not looked at the debug log, but... 8 is user/emergency, so you have an emergency message ;) rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Alex Cartwright > Sent: Tuesday, January 24, 2012 5:01 PM > To: rsyslog-users > Subject: Re: [rsyslog] How to stop 'user' facility printing to console > > This is the output from rsyslog debug: https://gist.github.com/1670748 > > I think I have found the problem, and it appears to be a bug in the > web app. It is logging with a priority/level of 8, which unless I am > mistaken simply does not exist. However I still can't see where in the > rsyslog.conf that it says to print to the console - can you? > > I'll get this fixed up in the app, but at least I've found the problem! > > Many thanks, > Alex > > On 22 January 2012 21:16, wrote: > > what priority is being set on th logs that you are seeing for these > 'user' > > facility logs? > > > > one thing that you can do is to setup a filter to log the types of > messages > > that you are seeing with the format RSYSLOG_DEBUG so that you can see > the > > facility, severity, etc that the log messages have. > > > > David Lang > > > > > > On Sun, 22 Jan 2012, Alex Cartwright wrote: > > > >> Here is my configuration for rsyslog, > https://gist.github.com/1656491 > >> > >> I can see that emergencies are being sent to all users, however that > >> priority is never used. > >> > >> Regards, > >> Alex > >> > >> On 22 January 2012 02:02, ? wrote: > >>> > >>> On Sat, 21 Jan 2012, Alex Cartwright wrote: > >>> > >>>> Hello, > >>>> > >>>> I have a few web applications that are logging to rsyslog/syslog > using > >>>> the 'user' facility on Debian 6, which by default is configured to > log > >>>> these to the file /var/log/user.log > >>>> > >>>> This is working just fine, however these messages are also being > >>>> printed to the console of every user that is currently logged in > (via > >>>> SSH). I'm really struggling to figure out how I can stop this > >>>> behaviour. Is it anything to do with the "kernel.printk" level? > This > >>>> is the current value of it: > >>>> > >>>> $ sysctl kernel.printk > >>>> kernel.printk = 7 ? ? ? 4 ? ? ? 1 ? ? ? 7 > >>> > >>> > >>> > >>> no, this has nothing to do with the kernel.printk config. probably > what > >>> is > >>> happening is the severity is set to critical or something like > that. We > >>> would need to see your rsyslog.conf (and whatever files it > includes) to > >>> find > >>> what's configured to be sent to the logged in users. > >>> > >>> David Lang > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From alexc223 at gmail.com Tue Jan 24 17:05:29 2012 From: alexc223 at gmail.com (Alex Cartwright) Date: Tue, 24 Jan 2012 16:05:29 +0000 Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72817C6@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72817C6@GRFEXC.intern.adiscon.com> Message-ID: Excellent, so "*.emerg *" is doing the magic here. Time to fix the app. Thank you. Alex On 24 January 2012 16:03, Rainer Gerhards wrote: > Not looked at the debug log, but... 8 is user/emergency, so you have an > emergency message ;) > > rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Alex Cartwright >> Sent: Tuesday, January 24, 2012 5:01 PM >> To: rsyslog-users >> Subject: Re: [rsyslog] How to stop 'user' facility printing to console >> >> This is the output from rsyslog debug: https://gist.github.com/1670748 >> >> I think I have found the problem, and it appears to be a bug in the >> web app. It is logging with a priority/level of 8, which unless I am >> mistaken simply does not exist. However I still can't see where in the >> rsyslog.conf that it says to print to the console - can you? >> >> I'll get this fixed up in the app, but at least I've found the problem! >> >> Many thanks, >> Alex >> >> On 22 January 2012 21:16, ? wrote: >> > what priority is being set on th logs that you are seeing for these >> 'user' >> > facility logs? >> > >> > one thing that you can do is to setup a filter to log the types of >> messages >> > that you are seeing with the format RSYSLOG_DEBUG so that you can see >> the >> > facility, severity, etc that the log messages have. >> > >> > David Lang >> > >> > >> > On Sun, 22 Jan 2012, Alex Cartwright wrote: >> > >> >> Here is my configuration for rsyslog, >> https://gist.github.com/1656491 >> >> >> >> I can see that emergencies are being sent to all users, however that >> >> priority is never used. >> >> >> >> Regards, >> >> Alex >> >> >> >> On 22 January 2012 02:02, ? wrote: >> >>> >> >>> On Sat, 21 Jan 2012, Alex Cartwright wrote: >> >>> >> >>>> Hello, >> >>>> >> >>>> I have a few web applications that are logging to rsyslog/syslog >> using >> >>>> the 'user' facility on Debian 6, which by default is configured to >> log >> >>>> these to the file /var/log/user.log >> >>>> >> >>>> This is working just fine, however these messages are also being >> >>>> printed to the console of every user that is currently logged in >> (via >> >>>> SSH). I'm really struggling to figure out how I can stop this >> >>>> behaviour. Is it anything to do with the "kernel.printk" level? >> This >> >>>> is the current value of it: >> >>>> >> >>>> $ sysctl kernel.printk >> >>>> kernel.printk = 7 ? ? ? 4 ? ? ? 1 ? ? ? 7 >> >>> >> >>> >> >>> >> >>> no, this has nothing to do with the kernel.printk config. probably >> what >> >>> is >> >>> happening is the severity is set to critical or something like >> that. We >> >>> would need to see your rsyslog.conf (and whatever files it >> includes) to >> >>> find >> >>> what's configured to be sent to the logged in users. >> >>> >> >>> David Lang >> >>> _______________________________________________ >> >>> rsyslog mailing list >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>> http://www.rsyslog.com/professional-services/ >> >> >> >> _______________________________________________ >> >> rsyslog mailing list >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> > >> > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From rgerhards at hq.adiscon.com Tue Jan 24 17:06:45 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 24 Jan 2012 17:06:45 +0100 Subject: [rsyslog] How to stop 'user' facility printing to console In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA72817C6@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817C7@GRFEXC.intern.adiscon.com> Yup, think so. rg > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Alex Cartwright > Sent: Tuesday, January 24, 2012 5:05 PM > To: rsyslog-users > Subject: Re: [rsyslog] How to stop 'user' facility printing to console > > Excellent, so "*.emerg *" is doing the magic > here. > > Time to fix the app. Thank you. > > Alex > > On 24 January 2012 16:03, Rainer Gerhards > wrote: > > Not looked at the debug log, but... 8 is user/emergency, so you have > an > > emergency message ;) > > > > rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Alex Cartwright > >> Sent: Tuesday, January 24, 2012 5:01 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] How to stop 'user' facility printing to > console > >> > >> This is the output from rsyslog debug: > https://gist.github.com/1670748 > >> > >> I think I have found the problem, and it appears to be a bug in the > >> web app. It is logging with a priority/level of 8, which unless I am > >> mistaken simply does not exist. However I still can't see where in > the > >> rsyslog.conf that it says to print to the console - can you? > >> > >> I'll get this fixed up in the app, but at least I've found the > problem! > >> > >> Many thanks, > >> Alex > >> > >> On 22 January 2012 21:16, ? wrote: > >> > what priority is being set on th logs that you are seeing for > these > >> 'user' > >> > facility logs? > >> > > >> > one thing that you can do is to setup a filter to log the types of > >> messages > >> > that you are seeing with the format RSYSLOG_DEBUG so that you can > see > >> the > >> > facility, severity, etc that the log messages have. > >> > > >> > David Lang > >> > > >> > > >> > On Sun, 22 Jan 2012, Alex Cartwright wrote: > >> > > >> >> Here is my configuration for rsyslog, > >> https://gist.github.com/1656491 > >> >> > >> >> I can see that emergencies are being sent to all users, however > that > >> >> priority is never used. > >> >> > >> >> Regards, > >> >> Alex > >> >> > >> >> On 22 January 2012 02:02, ? wrote: > >> >>> > >> >>> On Sat, 21 Jan 2012, Alex Cartwright wrote: > >> >>> > >> >>>> Hello, > >> >>>> > >> >>>> I have a few web applications that are logging to > rsyslog/syslog > >> using > >> >>>> the 'user' facility on Debian 6, which by default is configured > to > >> log > >> >>>> these to the file /var/log/user.log > >> >>>> > >> >>>> This is working just fine, however these messages are also > being > >> >>>> printed to the console of every user that is currently logged > in > >> (via > >> >>>> SSH). I'm really struggling to figure out how I can stop this > >> >>>> behaviour. Is it anything to do with the "kernel.printk" level? > >> This > >> >>>> is the current value of it: > >> >>>> > >> >>>> $ sysctl kernel.printk > >> >>>> kernel.printk = 7 ? ? ? 4 ? ? ? 1 ? ? ? 7 > >> >>> > >> >>> > >> >>> > >> >>> no, this has nothing to do with the kernel.printk config. > probably > >> what > >> >>> is > >> >>> happening is the severity is set to critical or something like > >> that. We > >> >>> would need to see your rsyslog.conf (and whatever files it > >> includes) to > >> >>> find > >> >>> what's configured to be sent to the logged in users. > >> >>> > >> >>> David Lang > >> >>> _______________________________________________ > >> >>> rsyslog mailing list > >> >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >>> http://www.rsyslog.com/professional-services/ > >> >> > >> >> _______________________________________________ > >> >> rsyslog mailing list > >> >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> >> http://www.rsyslog.com/professional-services/ > >> > > >> > > >> > _______________________________________________ > >> > rsyslog mailing list > >> > http://lists.adiscon.net/mailman/listinfo/rsyslog > >> > http://www.rsyslog.com/professional-services/ > >> > > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From pgillard at thoughtworks.com Tue Jan 24 18:40:40 2012 From: pgillard at thoughtworks.com (Peter Gillard-Moss) Date: Tue, 24 Jan 2012 17:40:40 +0000 Subject: [rsyslog] Conditional forwarding on imfiles Message-ID: Hello, I am using loggly to forward messages to. I need to send logs from a custom application which uses a bespoke logging format to a different location. I have an imfile config which looks like this: $ModLoad imfile $InputFileName /opt/myapp/log/my_app.log $InputFileTag myapp: $InputFileStateFile stat-myapp $InputFileSeverity info $InputRunFileMonitor $InputFilePollInterval 10 *.* @@logs.loggly.com:[port] I only want to forward entries from my_app. How would I go about doing this? Thanks in advance. Peter From rgerhards at hq.adiscon.com Tue Jan 24 18:50:15 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 24 Jan 2012 18:50:15 +0100 Subject: [rsyslog] Conditional forwarding on imfiles In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817CE@GRFEXC.intern.adiscon.com> You could check for the tag you have configured; make sure that it is sufficiently unique. rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Peter Gillard-Moss > Sent: Tuesday, January 24, 2012 6:41 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Conditional forwarding on imfiles > > Hello, > > I am using loggly to forward messages to. I need to send logs from a > custom application which uses a bespoke logging format to a different > location. > > I have an imfile config which looks like this: > > $ModLoad imfile > $InputFileName /opt/myapp/log/my_app.log > $InputFileTag myapp: > $InputFileStateFile stat-myapp > $InputFileSeverity info > $InputRunFileMonitor > > $InputFilePollInterval 10 > > *.* @@logs.loggly.com:[port] > > I only want to forward entries from my_app. How would I go about doing > this? > > Thanks in advance. > Peter > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From xuan_zhong at hotmail.com Tue Jan 24 21:39:37 2012 From: xuan_zhong at hotmail.com (Xuan Zhong) Date: Tue, 24 Jan 2012 20:39:37 +0000 Subject: [rsyslog] Timestamp formatting In-Reply-To: References: Message-ID: Hi, I am using rfc3339 to format the timestamp and get the following in the log: 2012-01-24T21:36:11.860808+01:00 Does anyone know if there is a way to change the template to display only part of the Timestamp: 2012-01-24T21:36:11.8608 which is fixed length string and replace T with a white space? Thanks Xuan From rgerhards at hq.adiscon.com Tue Jan 24 21:59:50 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 24 Jan 2012 21:59:50 +0100 Subject: [rsyslog] Timestamp formatting In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817D2@GRFEXC.intern.adiscon.com> Search for the property replacer, you can set the start and endposition oft he string. Yoou need to extract the date, then put a SP into the template, than extract the time part rg > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Xuan Zhong > Sent: Tuesday, January 24, 2012 9:40 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Timestamp formatting > > > > Hi, > I am using rfc3339 to format the timestamp and get the following in the > log: > 2012-01-24T21:36:11.860808+01:00 > > Does anyone know if there is a way to change the template to display > only part of the Timestamp: 2012-01-24T21:36:11.8608 which is fixed > length string and replace T with a white space? > > Thanks > Xuan > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From pgillard at thoughtworks.com Wed Jan 25 10:19:53 2012 From: pgillard at thoughtworks.com (Peter Gillard-Moss) Date: Wed, 25 Jan 2012 09:19:53 +0000 Subject: [rsyslog] Conditional forwarding on imfiles In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72817CE@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72817CE@GRFEXC.intern.adiscon.com> Message-ID: Thanks for the reply Rainer. I have searched and searched for this (so perhaps it is a RTFM but I'm struggling to find what part). The nearest thing I could find was this: :msg, startswith, "my_app:" @@logs.loggly.com:[port]', Does this look correct? On 24 January 2012 17:50, Rainer Gerhards wrote: > You could check for the tag you have configured; make sure that it is > sufficiently unique. > > rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Peter Gillard-Moss > > Sent: Tuesday, January 24, 2012 6:41 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] Conditional forwarding on imfiles > > > > Hello, > > > > I am using loggly to forward messages to. I need to send logs from a > > custom application which uses a bespoke logging format to a different > > location. > > > > I have an imfile config which looks like this: > > > > $ModLoad imfile > > $InputFileName /opt/myapp/log/my_app.log > > $InputFileTag myapp: > > $InputFileStateFile stat-myapp > > $InputFileSeverity info > > $InputRunFileMonitor > > > > $InputFilePollInterval 10 > > > > *.* @@logs.loggly.com:[port] > > > > I only want to forward entries from my_app. How would I go about doing > > this? > > > > Thanks in advance. > > Peter > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Wed Jan 25 10:24:58 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 25 Jan 2012 10:24:58 +0100 Subject: [rsyslog] Conditional forwarding on imfiles In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA72817CE@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817D5@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Peter Gillard-Moss > Sent: Wednesday, January 25, 2012 10:20 AM > To: rsyslog-users > Subject: Re: [rsyslog] Conditional forwarding on imfiles > > Thanks for the reply Rainer. > > I have searched and searched for this (so perhaps it is a RTFM but I'm > struggling to find what part). > > The nearest thing I could find was this: > :msg, startswith, "my_app:" @@logs.loggly.com:[port]', almost ;) It's not msg but rather syslogtag (or tag, don't know out of my head). Quick way to see what is in message: add *.* /var/log/debuglog;RSYSLOG_DebugFormat The file will tell you all properties used and actual content. Hth rg > > Does this look correct? > > On 24 January 2012 17:50, Rainer Gerhards > wrote: > > > You could check for the tag you have configured; make sure that it is > > sufficiently unique. > > > > rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Peter Gillard-Moss > > > Sent: Tuesday, January 24, 2012 6:41 PM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] Conditional forwarding on imfiles > > > > > > Hello, > > > > > > I am using loggly to forward messages to. I need to send logs from > > > a custom application which uses a bespoke logging format to a > > > different location. > > > > > > I have an imfile config which looks like this: > > > > > > $ModLoad imfile > > > $InputFileName /opt/myapp/log/my_app.log $InputFileTag myapp: > > > $InputFileStateFile stat-myapp > > > $InputFileSeverity info > > > $InputRunFileMonitor > > > > > > $InputFilePollInterval 10 > > > > > > *.* @@logs.loggly.com:[port] > > > > > > I only want to forward entries from my_app. How would I go about > > > doing this? > > > > > > Thanks in advance. > > > Peter > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Wed Jan 25 10:53:06 2012 From: michael at maymann.org (Michael Maymann) Date: Wed, 25 Jan 2012 10:53:06 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: Hi, 1. how do I test if dns-cache is enabled in v6.3.6-devel (strace rsyslog -c6... what to look for...?) ? 2. This is my current rsyslog.conf - comments are very welcome (any thing that looks odd to you...?): --- #LOAD MODULES $ModLoad imtcp $InputTCPServerRun 514 #SET DESTINATION FOR LOGS $template DYNmessages,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_messages" $template DYNsecure,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_secure" $template DYNmaillog,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_maillog" $template DYNcron,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_cron" $template DYNspooler,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_spooler" $template DYNboot,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_boot.log" $template DYNtraps,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_traps" #SET LOGGING CONDITIONS if $syslogseverity <= '6' then ?DYNmessages if $syslogfacility-text == 'authpriv' then ?DYNsecure if $syslogfacility-text == 'mail' then ?DYNmaillog if $syslogfacility-text == 'cron' then ?DYNcron if $syslogseverity-text == 'crit' then ?DYNspooler if $syslogfacility-text == 'local7' then ?DYNboot if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' then ?DYNtraps --- 3. A little off-topic - I have setup monthly logrotate: /DIR_TO_LOGS/*/* { rotate 36 monthly missingok notifempty # delaycompress # compress create 2750 _rsyslog rme_it sharedscripts postrotate service rsyslog restart >/dev/null 2>&1 || true endscript } 3.a: is this looking at each logfiles fileage (and rotate when the fileage is more than 1 month old) or does it rotate the files every 1st of the month ? 3.b: we use SIS and think of dedublication (from NetAPP) instead of compressing data during logrotate - any experience, recommendations ? Br. ~maymann 2012/1/19 Michael Maymann > Hi David, > > Thanks for your reply...:-) ! > > 2012/1/19 > > On Thu, 19 Jan 2012, Michael Maymann wrote: >> >> Hi, >>> >>> Got it installed...: >>> yum install zlib-devel.x86_64 >>> yum install byacc.x86_64 >>> did the trick for me... >>> Thanks for your help Rainer...:-) ! >>> >>> Trying to look through the man-page and googling a bit... >>> 1. do I have to enable DNS cache somewhere, or is this feature on by >>> default ? >>> >> >> I don't know > > > Ok, anyone who knows this ? > >> >> >> 2. Is there some v6.3.6 version config examples somewhere. I would need >>> to: >>> a. generally have logfiles in a >>> /NFS_logdir/FQDN/equial_to_**local_logfile_name (e.g. messages) >>> b. have maillog from mailcluster=host001 (host001a+host001b) being merged >>> in a single /NFS_logdir/host001/maillog file >>> >> >> unless you are needing some 6.x specific feature, the configs from >> earlier versions will still work. >> > > I'm don't think I do...:-) - besides the DNS cache thing. > But I'm new to syslog/rsyslog and wondering if someone has a link/example > to something similar to what I describe above. > >> >> David Lang >> >> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> > > From pgillard at thoughtworks.com Wed Jan 25 11:13:11 2012 From: pgillard at thoughtworks.com (Peter Gillard-Moss) Date: Wed, 25 Jan 2012 10:13:11 +0000 Subject: [rsyslog] Conditional forwarding on imfiles In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72817D5@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72817CE@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA72817D5@GRFEXC.intern.adiscon.com> Message-ID: Thanks for the pointers. I used the debug and found that both programname and APP-NAME did the job. I also found this thanks to your pointers: http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/ch-Viewing_and_Managing_Log_Files.html That had a useful guide to property based and expression based filters. In the end I went with the following expression based filter: if $programname == 'my_app' then @@logs.loggly.com:[port] On 25 January 2012 09:24, Rainer Gerhards wrote: > > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Peter Gillard-Moss > > Sent: Wednesday, January 25, 2012 10:20 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] Conditional forwarding on imfiles > > > > Thanks for the reply Rainer. > > > > I have searched and searched for this (so perhaps it is a RTFM but I'm > > struggling to find what part). > > > > The nearest thing I could find was this: > > :msg, startswith, "my_app:" @@logs.loggly.com:[port]', > > almost ;) It's not msg but rather syslogtag (or tag, don't know out of my > head). > > Quick way to see what is in message: add > *.* /var/log/debuglog;RSYSLOG_DebugFormat > The file will tell you all properties used and actual content. > > Hth rg > > > > Does this look correct? > > > > On 24 January 2012 17:50, Rainer Gerhards > > wrote: > > > > > You could check for the tag you have configured; make sure that it is > > > sufficiently unique. > > > > > > rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Peter Gillard-Moss > > > > Sent: Tuesday, January 24, 2012 6:41 PM > > > > To: rsyslog at lists.adiscon.com > > > > Subject: [rsyslog] Conditional forwarding on imfiles > > > > > > > > Hello, > > > > > > > > I am using loggly to forward messages to. I need to send logs from > > > > a custom application which uses a bespoke logging format to a > > > > different location. > > > > > > > > I have an imfile config which looks like this: > > > > > > > > $ModLoad imfile > > > > $InputFileName /opt/myapp/log/my_app.log $InputFileTag myapp: > > > > $InputFileStateFile stat-myapp > > > > $InputFileSeverity info > > > > $InputRunFileMonitor > > > > > > > > $InputFilePollInterval 10 > > > > > > > > *.* @@logs.loggly.com:[port] > > > > > > > > I only want to forward entries from my_app. How would I go about > > > > doing this? > > > > > > > > Thanks in advance. > > > > Peter > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From xuan_zhong at hotmail.com Thu Jan 26 03:37:35 2012 From: xuan_zhong at hotmail.com (Xuan Zhong) Date: Thu, 26 Jan 2012 02:37:35 +0000 Subject: [rsyslog] Timestamp formatting In-Reply-To: References: , Message-ID: Hi, I subscribed to the mailer list but didn't receive the email. I saw other people responded my previous email in the http archives link. I tried to extract the date part using the start and end position of the string, it didn't work. Here is my timestamp template: %TIMESTAMP:::date-rfc3339% I tried to extract the substring position from 1 to 24, %TIMESTAMP:::date-rfc3339:1:24%, which gave me a totally different formatted timestamp. Any idea which part is wrong? I tried to use that to extract part of msg text, it did work, like %msg:1:24% gave me first 24 characters. Thanks Xuan Search for the property replacer, you can set the start and endposition oft he string. Yoou need to extract the date, then put a SP into the template, than extract the time part rg From: xuan_zhong at hotmail.com To: rsyslog at lists.adiscon.com Subject: Timestamp formatting Date: Tue, 24 Jan 2012 20:39:37 +0000 Hi, I am using rfc3339 to format the timestamp and get the following in the log: 2012-01-24T21:36:11.860808+01:00 Does anyone know if there is a way to change the template to display only part of the Timestamp: 2012-01-24T21:36:11.8608 which is fixed length string and replace T with a white space? Thanks Xuan From rgerhards at hq.adiscon.com Thu Jan 26 07:21:59 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 26 Jan 2012 07:21:59 +0100 Subject: [rsyslog] Timestamp formatting In-Reply-To: References: , Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72817EA@GRFEXC.intern.adiscon.com> %TIMESTAMP:1:24:date-rfc3339%, > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Xuan Zhong > Sent: Thursday, January 26, 2012 3:38 AM > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Timestamp formatting > > > Hi, > > I subscribed to the mailer list but didn't receive the email. I saw > other people responded my previous email in the http archives link. > I tried to extract the date part using the start and end position of > the string, it didn't work. > Here is my timestamp template: %TIMESTAMP:::date-rfc3339% > I tried to extract the substring position from 1 to 24, > %TIMESTAMP:::date-rfc3339:1:24%, which gave me a totally different > formatted timestamp. > Any idea which part is wrong? I tried to use that to extract part of > msg text, it did work, like %msg:1:24% gave me first 24 characters. > > Thanks > Xuan > > > Search for the property replacer, you can set the start and endposition > oft > he string. Yoou need to extract the date, then put a SP into the > template, > than extract the time part > > rg > > From: xuan_zhong at hotmail.com > To: rsyslog at lists.adiscon.com > Subject: Timestamp formatting > Date: Tue, 24 Jan 2012 20:39:37 +0000 > > > > > > > > > Hi, > I am using rfc3339 to format the timestamp and get the following in the > log: > 2012-01-24T21:36:11.860808+01:00 > > Does anyone know if there is a way to change the template to display > only part of the Timestamp: 2012-01-24T21:36:11.8608 which is fixed > length string and replace T with a white space? > > Thanks > Xuan > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From Lars.Peterson at cision.com Thu Jan 26 17:53:17 2012 From: Lars.Peterson at cision.com (Lars Peterson) Date: Thu, 26 Jan 2012 16:53:17 +0000 Subject: [rsyslog] Date formatting property options incompatible with SOLR DateField Message-ID: Hello, I'm trying to get syslog data into a SOLR index. I'm using James Keating's omsolr output module (thanks James!). I'm using a binary I built from rsyslog 5.8.7 and its outputting things as specified in my rsyslog.conf template: $Template SolrLog, "%fromhost%%syslogfacility-text%%hostname%%syslogtag%%programname%%syslogseverity-text%%syslogpriority-text%%timegenerated:::date-rfc3339%%timereported:::date-rfc3339%" The documents are added to the SOLR index just fine so long as each field is defined as a "text" type in the SOLR schema. I'd like to define the "generated" and "timestamp" fields with type "DateField" or "TrieDateField" so that the index is searchable by date (vs string pattern matching). The problem I'm having is when I try to add documents to the SOLR index using fields defined as native solr.DateField or solr.TrieDateField types in the SOLR schema. According to the documentation, http://lucene.apache.org/solr/api/org/apache/solr/schema/DateField.html : A date field shall be of the form 1995-12-31T23:59:59Z The trailing "Z" designates UTC time and is mandatory (See below for an explanation of UTC). Optional fractional seconds are allowed, as long as they do not end in a trailing 0 (but any precision beyond milliseconds will be ignored). All other parts are mandatory. I sniffed the wire to see what rsyslog is sending to SOLR and I see that the output (RFC-3339) is formatted like so: 2012-01-25T21:46:13.102571+00:00 When I attempt to insert the document using this format I get an error: The request sent by the client was syntactically incorrect (ERROR: [doc=null] Error adding field 'generated'='2012-01-25T21:46:13.102571+00:00'). I see 3 possible workarounds: 1) Add another property option to format the timestamp so that its compliant with ISO-8601 / Java DateField 2) Format the timestamp using a [regex] property replacer in rsyslog.conf. The time zone suffix would be stripped off and replaced with a trailing 'Z'. Not sure if this is possible and if it is it seems ugly and perhaps costly from a performance perspective. 3) Ditch the omsolr plugin altogether and use an external script to pipe the output to so that the field can be properly formatted. Yuck - I'd much rather use the compiled module. Any suggestions? And thanks to Rainer & James for sharing these tools! Regards, Lars Peterson From david at lang.hm Thu Jan 26 18:02:36 2012 From: david at lang.hm (david at lang.hm) Date: Thu, 26 Jan 2012 09:02:36 -0800 (PST) Subject: [rsyslog] Date formatting property options incompatible with SOLR DateField In-Reply-To: References: Message-ID: On Thu, 26 Jan 2012, Lars Peterson wrote: > A date field shall be of the form 1995-12-31T23:59:59Z The trailing "Z" designates UTC time and is mandatory (See below for an explanation of UTC). Optional fractional seconds are allowed, as long as they do not end in a trailing 0 (but any precision beyond milliseconds will be ignored). All other parts are mandatory. > > I sniffed the wire to see what rsyslog is sending to SOLR and I see that the output (RFC-3339) is formatted like so: > > 2012-01-25T21:46:13.102571+00:00 > > When I attempt to insert the document using this format I get an error: > > The request sent by the client was syntactically incorrect (ERROR: [doc=null] Error adding field 'generated'='2012-01-25T21:46:13.102571+00:00'). > > I see 3 possible workarounds: > > > 1) Add another property option to format the timestamp so that its compliant with ISO-8601 / Java DateField this is the right option to take. currently you are doing %timegenerated:::date-rfc3339% change it to %timegenerated:1:19:date-rfc3339%Z this will grab the first 19 characters of the timestamp (up to seconds) and then the Z will be added (as it's outside the %) David Lang From Lars.Peterson at cision.com Thu Jan 26 18:39:59 2012 From: Lars.Peterson at cision.com (Lars Peterson) Date: Thu, 26 Jan 2012 17:39:59 +0000 Subject: [rsyslog] Date formatting property options incompatible with SOLR DateField In-Reply-To: References: Message-ID: David, Love those 4 character fixes. Works perfectly now. Thank you so much!!! -Lars -----Original Message----- From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm Sent: Thursday, January 26, 2012 11:03 AM To: rsyslog-users Cc: jamesk at okeating.net Subject: Re: [rsyslog] Date formatting property options incompatible with SOLR DateField On Thu, 26 Jan 2012, Lars Peterson wrote: > A date field shall be of the form 1995-12-31T23:59:59Z The trailing "Z" designates UTC time and is mandatory (See below for an explanation of UTC). Optional fractional seconds are allowed, as long as they do not end in a trailing 0 (but any precision beyond milliseconds will be ignored). All other parts are mandatory. > > I sniffed the wire to see what rsyslog is sending to SOLR and I see that the output (RFC-3339) is formatted like so: > > 2012-01-25T21:46:13.102571+00:00 > > When I attempt to insert the document using this format I get an error: > > The request sent by the client was syntactically incorrect (ERROR: [doc=null] Error adding field 'generated'='2012-01-25T21:46:13.102571+00:00'). > > I see 3 possible workarounds: > > > 1) Add another property option to format the timestamp so that its compliant with ISO-8601 / Java DateField this is the right option to take. currently you are doing %timegenerated:::date-rfc3339% change it to %timegenerated:1:19:date-rfc3339%Z this will grab the first 19 characters of the timestamp (up to seconds) and then the Z will be added (as it's outside the %) David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ From michael at maymann.org Fri Jan 27 15:53:30 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 27 Jan 2012 15:53:30 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: Hi List, Please - anyone who can help with this ? Thanks in advance :-) ! ~maymann 2012/1/25 Michael Maymann > Hi, > > 1. how do I test if dns-cache is enabled in v6.3.6-devel (strace rsyslog > -c6... what to look for...?) ? > 2. This is my current rsyslog.conf - comments are very welcome (any thing > that looks odd to you...?): > --- > #LOAD MODULES > $ModLoad imtcp > $InputTCPServerRun 514 > > #SET DESTINATION FOR LOGS > $template > DYNmessages,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_messages" > $template > DYNsecure,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_secure" > $template > DYNmaillog,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_maillog" > $template DYNcron,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_cron" > $template > DYNspooler,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_spooler" > $template > DYNboot,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_boot.log" > $template > DYNtraps,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_traps" > > #SET LOGGING CONDITIONS > if $syslogseverity <= '6' then ?DYNmessages > if $syslogfacility-text == 'authpriv' then ?DYNsecure > if $syslogfacility-text == 'mail' then ?DYNmaillog > if $syslogfacility-text == 'cron' then ?DYNcron > if $syslogseverity-text == 'crit' then ?DYNspooler > if $syslogfacility-text == 'local7' then ?DYNboot > if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' > then ?DYNtraps > --- > > 3. A little off-topic - I have setup monthly logrotate: > /DIR_TO_LOGS/*/* > { > rotate 36 > monthly > missingok > notifempty > # delaycompress > # compress > create 2750 _rsyslog rme_it > sharedscripts > postrotate > service rsyslog restart >/dev/null 2>&1 || true > endscript > } > 3.a: is this looking at each logfiles fileage (and rotate when the fileage > is more than 1 month old) or does it rotate the files every 1st of the > month ? > 3.b: we use SIS and think of dedublication (from NetAPP) instead of > compressing data during logrotate - any experience, recommendations ? > > > Br. > ~maymann > > > > 2012/1/19 Michael Maymann > >> Hi David, >> >> Thanks for your reply...:-) ! >> >> 2012/1/19 >> >> On Thu, 19 Jan 2012, Michael Maymann wrote: >>> >>> Hi, >>>> >>>> Got it installed...: >>>> yum install zlib-devel.x86_64 >>>> yum install byacc.x86_64 >>>> did the trick for me... >>>> Thanks for your help Rainer...:-) ! >>>> >>>> Trying to look through the man-page and googling a bit... >>>> 1. do I have to enable DNS cache somewhere, or is this feature on by >>>> default ? >>>> >>> >>> I don't know >> >> >> Ok, anyone who knows this ? >> >>> >>> >>> 2. Is there some v6.3.6 version config examples somewhere. I would need >>>> to: >>>> a. generally have logfiles in a >>>> /NFS_logdir/FQDN/equial_to_**local_logfile_name (e.g. messages) >>>> b. have maillog from mailcluster=host001 (host001a+host001b) being >>>> merged >>>> in a single /NFS_logdir/host001/maillog file >>>> >>> >>> unless you are needing some 6.x specific feature, the configs from >>> earlier versions will still work. >>> >> >> I'm don't think I do...:-) - besides the DNS cache thing. >> But I'm new to syslog/rsyslog and wondering if someone has a link/example >> to something similar to what I describe above. >> >>> >>> David Lang >>> >>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>> http://www.rsyslog.com/**professional-services/ >>> >> >> > From teifler at adiscon.com Fri Jan 27 16:51:54 2012 From: teifler at adiscon.com (Tim Eifler) Date: Fri, 27 Jan 2012 16:51:54 +0100 Subject: [rsyslog] rsyslog 5.9.5 (devel) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728180A@GRFEXC.intern.adiscon.com> Hi all, This release brings many additional statistics counters and a couple of bug fixes. Note that the default setting of $IMUxSockRateLimitInterval was changed to 0, as rate limiting seems to have caused more trouble than it was worth. To enable it, simply set it to 200, the previous default. For more detailed information, please read the ChangeLog. ChangeLog: http://rsyslog.com/changelog-for-5-9-5-v5-devel/ Download: http://rsyslog.com/rsyslog-5-9-5-v5-devel/ As always, feedback is appreciated. Best regards, Tim Eifler From david at lang.hm Fri Jan 27 19:35:27 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 27 Jan 2012 10:35:27 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: the surest way to tell that the DNS cache is working is to look at traffic to your DNS server (or more precisely the lack of traffic to that host) any way of checking in rsyslog itself is something that is brand new thing. There may be something in the rsyslog documentation, but I don't know anything about it. David Lang On Fri, 27 Jan 2012, Michael Maymann wrote: > Date: Fri, 27 Jan 2012 15:53:30 +0100 > From: Michael Maymann > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi List, > > Please - anyone who can help with this ? > > > Thanks in advance :-) ! > ~maymann > > 2012/1/25 Michael Maymann > >> Hi, >> >> 1. how do I test if dns-cache is enabled in v6.3.6-devel (strace rsyslog >> -c6... what to look for...?) ? >> 2. This is my current rsyslog.conf - comments are very welcome (any thing >> that looks odd to you...?): >> --- >> #LOAD MODULES >> $ModLoad imtcp >> $InputTCPServerRun 514 >> >> #SET DESTINATION FOR LOGS >> $template >> DYNmessages,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_messages" >> $template >> DYNsecure,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_secure" >> $template >> DYNmaillog,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_maillog" >> $template DYNcron,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_cron" >> $template >> DYNspooler,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_spooler" >> $template >> DYNboot,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_boot.log" >> $template >> DYNtraps,"/local/logs/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_traps" >> >> #SET LOGGING CONDITIONS >> if $syslogseverity <= '6' then ?DYNmessages >> if $syslogfacility-text == 'authpriv' then ?DYNsecure >> if $syslogfacility-text == 'mail' then ?DYNmaillog >> if $syslogfacility-text == 'cron' then ?DYNcron >> if $syslogseverity-text == 'crit' then ?DYNspooler >> if $syslogfacility-text == 'local7' then ?DYNboot >> if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' >> then ?DYNtraps >> --- >> >> 3. A little off-topic - I have setup monthly logrotate: >> /DIR_TO_LOGS/*/* >> { >> rotate 36 >> monthly >> missingok >> notifempty >> # delaycompress >> # compress >> create 2750 _rsyslog rme_it >> sharedscripts >> postrotate >> service rsyslog restart >/dev/null 2>&1 || true >> endscript >> } >> 3.a: is this looking at each logfiles fileage (and rotate when the fileage >> is more than 1 month old) or does it rotate the files every 1st of the >> month ? >> 3.b: we use SIS and think of dedublication (from NetAPP) instead of >> compressing data during logrotate - any experience, recommendations ? >> >> >> Br. >> ~maymann >> >> >> >> 2012/1/19 Michael Maymann >> >>> Hi David, >>> >>> Thanks for your reply...:-) ! >>> >>> 2012/1/19 >>> >>> On Thu, 19 Jan 2012, Michael Maymann wrote: >>>> >>>> Hi, >>>>> >>>>> Got it installed...: >>>>> yum install zlib-devel.x86_64 >>>>> yum install byacc.x86_64 >>>>> did the trick for me... >>>>> Thanks for your help Rainer...:-) ! >>>>> >>>>> Trying to look through the man-page and googling a bit... >>>>> 1. do I have to enable DNS cache somewhere, or is this feature on by >>>>> default ? >>>>> >>>> >>>> I don't know >>> >>> >>> Ok, anyone who knows this ? >>> >>>> >>>> >>>> 2. Is there some v6.3.6 version config examples somewhere. I would need >>>>> to: >>>>> a. generally have logfiles in a >>>>> /NFS_logdir/FQDN/equial_to_**local_logfile_name (e.g. messages) >>>>> b. have maillog from mailcluster=host001 (host001a+host001b) being >>>>> merged >>>>> in a single /NFS_logdir/host001/maillog file >>>>> >>>> >>>> unless you are needing some 6.x specific feature, the configs from >>>> earlier versions will still work. >>>> >>> >>> I'm don't think I do...:-) - besides the DNS cache thing. >>> But I'm new to syslog/rsyslog and wondering if someone has a link/example >>> to something similar to what I describe above. >>> >>>> >>>> David Lang >>>> >>>> ______________________________**_________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/**professional-services/ >>>> >>> >>> >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From michael at maymann.org Fri Jan 27 19:45:23 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 27 Jan 2012 19:45:23 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: Hi, David: thanks for another quick reply - much appreciated...:-) ! Ok, I will try to get access to our DNS server, and have a look at the logs there. 1. How does my rsyslog.conf and logrotate.d/rsyslog look ? 2. Is logrotate looking at each logfiles fileage (and rotate when the fileage is more than 1 month old) or does it rotate the files every 1st of the month ? 3. We use SIS and think of dedublication (from NetAPP) instead of compressing data during logrotate - any experience, recommendations ? Thanks in advance :-) ! ~maymann 2012/1/27 > the surest way to tell that the DNS cache is working is to look at traffic > to your DNS server (or more precisely the lack of traffic to that host) > > any way of checking in rsyslog itself is something that is brand new > thing. There may be something in the rsyslog documentation, but I don't > know anything about it. > > David Lang > > On Fri, 27 Jan 2012, Michael Maymann wrote: > > Date: Fri, 27 Jan 2012 15:53:30 +0100 >> >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog-users >> >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> Hi List, >> >> Please - anyone who can help with this ? >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/25 Michael Maymann >> >> Hi, >>> >>> 1. how do I test if dns-cache is enabled in v6.3.6-devel (strace rsyslog >>> -c6... what to look for...?) ? >>> 2. This is my current rsyslog.conf - comments are very welcome (any thing >>> that looks odd to you...?): >>> --- >>> #LOAD MODULES >>> $ModLoad imtcp >>> $InputTCPServerRun 514 >>> >>> #SET DESTINATION FOR LOGS >>> $template >>> DYNmessages,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>> $MONTH%_messages" >>> $template >>> DYNsecure,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>> $MONTH%_secure" >>> $template >>> DYNmaillog,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>> $MONTH%_maillog" >>> $template DYNcron,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>> $MONTH%_cron" >>> $template >>> DYNspooler,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>> $MONTH%_spooler" >>> $template >>> DYNboot,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>> $MONTH%_boot.log" >>> $template >>> DYNtraps,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%**$MONTH%_traps" >>> >>> #SET LOGGING CONDITIONS >>> if $syslogseverity <= '6' then ?DYNmessages >>> if $syslogfacility-text == 'authpriv' then ?DYNsecure >>> if $syslogfacility-text == 'mail' then ?DYNmaillog >>> if $syslogfacility-text == 'cron' then ?DYNcron >>> if $syslogseverity-text == 'crit' then ?DYNspooler >>> if $syslogfacility-text == 'local7' then ?DYNboot >>> if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' >>> then ?DYNtraps >>> --- >>> >>> 3. A little off-topic - I have setup monthly logrotate: >>> /DIR_TO_LOGS/*/* >>> { >>> rotate 36 >>> monthly >>> missingok >>> notifempty >>> # delaycompress >>> # compress >>> create 2750 _rsyslog rme_it >>> sharedscripts >>> postrotate >>> service rsyslog restart >/dev/null 2>&1 || true >>> endscript >>> } >>> 3.a: is this looking at each logfiles fileage (and rotate when the >>> fileage >>> is more than 1 month old) or does it rotate the files every 1st of the >>> month ? >>> 3.b: we use SIS and think of dedublication (from NetAPP) instead of >>> compressing data during logrotate - any experience, recommendations ? >>> >>> >>> Br. >>> ~maymann >>> >>> >>> >>> 2012/1/19 Michael Maymann >>> >>> Hi David, >>>> >>>> Thanks for your reply...:-) ! >>>> >>>> 2012/1/19 >>>> >>>> On Thu, 19 Jan 2012, Michael Maymann wrote: >>>> >>>>> >>>>> Hi, >>>>> >>>>>> >>>>>> Got it installed...: >>>>>> yum install zlib-devel.x86_64 >>>>>> yum install byacc.x86_64 >>>>>> did the trick for me... >>>>>> Thanks for your help Rainer...:-) ! >>>>>> >>>>>> Trying to look through the man-page and googling a bit... >>>>>> 1. do I have to enable DNS cache somewhere, or is this feature on by >>>>>> default ? >>>>>> >>>>>> >>>>> I don't know >>>>> >>>> >>>> >>>> Ok, anyone who knows this ? >>>> >>>> >>>>> >>>>> 2. Is there some v6.3.6 version config examples somewhere. I would >>>>> need >>>>> >>>>>> to: >>>>>> a. generally have logfiles in a >>>>>> /NFS_logdir/FQDN/equial_to_****local_logfile_name (e.g. messages) >>>>>> >>>>>> b. have maillog from mailcluster=host001 (host001a+host001b) being >>>>>> merged >>>>>> in a single /NFS_logdir/host001/maillog file >>>>>> >>>>>> >>>>> unless you are needing some 6.x specific feature, the configs from >>>>> earlier versions will still work. >>>>> >>>>> >>>> I'm don't think I do...:-) - besides the DNS cache thing. >>>> But I'm new to syslog/rsyslog and wondering if someone has a >>>> link/example >>>> to something similar to what I describe above. >>>> >>>> >>>>> David Lang >>>>> >>>>> ______________________________****_________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>>> >>>>> > >>>>> http://www.rsyslog.com/****professional-services/ >>>>> >>>>> > >>>>> >>>>> >>>> >>>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From david at lang.hm Fri Jan 27 19:51:28 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 27 Jan 2012 10:51:28 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: On Fri, 27 Jan 2012, Michael Maymann wrote: > Hi, > > David: thanks for another quick reply - much appreciated...:-) ! > Ok, I will try to get access to our DNS server, and have a look at the logs > there. > > 1. How does my rsyslog.conf and logrotate.d/rsyslog look ? I'm not that familiar with logrotate, but I'm not seeing anything grossly wrong > 2. Is logrotate looking at each logfiles fileage (and rotate when the > fileage > is more than 1 month old) or does it rotate the files every 1st of the > month ? > 3. We use SIS and think of dedublication (from NetAPP) instead of > compressing data during logrotate - any experience, recommendations ? dedup is generally aimed at full files, and even if it goes to lower levels than that it is only going to go down to the block level, so it's very unlikely that dedup will help you with log data at all (the chance of two blocks of logs being identical is very low) David Lang > Thanks in advance :-) ! > ~maymann > > > 2012/1/27 > >> the surest way to tell that the DNS cache is working is to look at traffic >> to your DNS server (or more precisely the lack of traffic to that host) >> >> any way of checking in rsyslog itself is something that is brand new >> thing. There may be something in the rsyslog documentation, but I don't >> know anything about it. >> > >> David Lang >> >> On Fri, 27 Jan 2012, Michael Maymann wrote: >> >> Date: Fri, 27 Jan 2012 15:53:30 +0100 >>> >>> From: Michael Maymann >>> Reply-To: rsyslog-users >>> To: rsyslog-users >>> >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >>> >>> Hi List, >>> >>> Please - anyone who can help with this ? >>> >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> 2012/1/25 Michael Maymann >>> >>> Hi, >>>> >>>> 1. how do I test if dns-cache is enabled in v6.3.6-devel (strace rsyslog >>>> -c6... what to look for...?) ? >>>> 2. This is my current rsyslog.conf - comments are very welcome (any thing >>>> that looks odd to you...?): >>>> --- >>>> #LOAD MODULES >>>> $ModLoad imtcp >>>> $InputTCPServerRun 514 >>>> >>>> #SET DESTINATION FOR LOGS >>>> $template >>>> DYNmessages,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>>> $MONTH%_messages" >>>> $template >>>> DYNsecure,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>>> $MONTH%_secure" >>>> $template >>>> DYNmaillog,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>>> $MONTH%_maillog" >>>> $template DYNcron,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>>> $MONTH%_cron" >>>> $template >>>> DYNspooler,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>>> $MONTH%_spooler" >>>> $template >>>> DYNboot,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%** >>>> $MONTH%_boot.log" >>>> $template >>>> DYNtraps,"/local/logs/%**FROMHOST%/%FROMHOST%_%$YEAR%.%**$MONTH%_traps" >>>> >>>> #SET LOGGING CONDITIONS >>>> if $syslogseverity <= '6' then ?DYNmessages >>>> if $syslogfacility-text == 'authpriv' then ?DYNsecure >>>> if $syslogfacility-text == 'mail' then ?DYNmaillog >>>> if $syslogfacility-text == 'cron' then ?DYNcron >>>> if $syslogseverity-text == 'crit' then ?DYNspooler >>>> if $syslogfacility-text == 'local7' then ?DYNboot >>>> if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' >>>> then ?DYNtraps >>>> --- >>>> >>>> 3. A little off-topic - I have setup monthly logrotate: >>>> /DIR_TO_LOGS/*/* >>>> { >>>> rotate 36 >>>> monthly >>>> missingok >>>> notifempty >>>> # delaycompress >>>> # compress >>>> create 2750 _rsyslog rme_it >>>> sharedscripts >>>> postrotate >>>> service rsyslog restart >/dev/null 2>&1 || true >>>> endscript >>>> } >>>> 3.a: is this looking at each logfiles fileage (and rotate when the >>>> fileage >>>> is more than 1 month old) or does it rotate the files every 1st of the >>>> month ? >>>> 3.b: we use SIS and think of dedublication (from NetAPP) instead of >>>> compressing data during logrotate - any experience, recommendations ? >>>> >>>> >>>> Br. >>>> ~maymann >>>> >>>> >>>> >>>> 2012/1/19 Michael Maymann >>>> >>>> Hi David, >>>>> >>>>> Thanks for your reply...:-) ! >>>>> >>>>> 2012/1/19 >>>>> >>>>> On Thu, 19 Jan 2012, Michael Maymann wrote: >>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>>> >>>>>>> Got it installed...: >>>>>>> yum install zlib-devel.x86_64 >>>>>>> yum install byacc.x86_64 >>>>>>> did the trick for me... >>>>>>> Thanks for your help Rainer...:-) ! >>>>>>> >>>>>>> Trying to look through the man-page and googling a bit... >>>>>>> 1. do I have to enable DNS cache somewhere, or is this feature on by >>>>>>> default ? >>>>>>> >>>>>>> >>>>>> I don't know >>>>>> >>>>> >>>>> >>>>> Ok, anyone who knows this ? >>>>> >>>>> >>>>>> >>>>>> 2. Is there some v6.3.6 version config examples somewhere. I would >>>>>> need >>>>>> >>>>>>> to: >>>>>>> a. generally have logfiles in a >>>>>>> /NFS_logdir/FQDN/equial_to_****local_logfile_name (e.g. messages) >>>>>>> >>>>>>> b. have maillog from mailcluster=host001 (host001a+host001b) being >>>>>>> merged >>>>>>> in a single /NFS_logdir/host001/maillog file >>>>>>> >>>>>>> >>>>>> unless you are needing some 6.x specific feature, the configs from >>>>>> earlier versions will still work. >>>>>> >>>>>> >>>>> I'm don't think I do...:-) - besides the DNS cache thing. >>>>> But I'm new to syslog/rsyslog and wondering if someone has a >>>>> link/example >>>>> to something similar to what I describe above. >>>>> >>>>> >>>>>> David Lang >>>>>> >>>>>> ______________________________****_________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>>>> >>>>>>> >>>>>> http://www.rsyslog.com/****professional-services/ >>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>> http://www.rsyslog.com/**professional-services/ >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From radu0gheorghe at gmail.com Mon Jan 30 16:10:50 2012 From: radu0gheorghe at gmail.com (Radu Gheorghe) Date: Mon, 30 Jan 2012 17:10:50 +0200 Subject: [rsyslog] Load balancing for rsyslog aggregators Message-ID: Hello, I'm currently using rsyslog in the following setup: - a bunch of clients send Syslog messages to a central rsyslog daemon via TCP - right now, the central rsyslog daemon pipes the logs to both a plain text file and an external program. But I guess that's not so relevant Soon, this central rsyslog would be overwhelmed by the amount of logging from the clients. And I'm looking for a solution to be able to deploy new "central" rsyslog daemons. I thought about using DNS round robin: - configure clients to send logs to a single hostname - once a new "central" rsyslog is added, add it to DNS Disadvantages to this are specific to DNS round-robin: - load is not balanced if "central" servers are not the same - there is a lag due to DNS caching And I guess another solution is to have a script to run after deployment, which will change the rsyslog.conf on all the clients. But that's doesn't seem to be a good idea, especially since there's no way to actually balance load on the aggregators. Only to make some clients to log to one, others to another, etc. Do you have any thoughts on how to solve the problem? From rgerhards at hq.adiscon.com Mon Jan 30 16:37:31 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 30 Jan 2012 16:37:31 +0100 Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281813@GRFEXC.intern.adiscon.com> How many messages does the central server process per second? Does the problem persists if you do not pipe to the external program? If so, you could load-balance just that part. rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghe > Sent: Monday, January 30, 2012 4:11 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] Load balancing for rsyslog aggregators > > Hello, > > I'm currently using rsyslog in the following setup: > - a bunch of clients send Syslog messages to a central rsyslog daemon > via TCP > - right now, the central rsyslog daemon pipes the logs to both a plain > text file and an external program. But I guess that's not so relevant > > Soon, this central rsyslog would be overwhelmed by the amount of > logging from the clients. And I'm looking for a solution to be able to > deploy new "central" rsyslog daemons. > > I thought about using DNS round robin: > - configure clients to send logs to a single hostname > - once a new "central" rsyslog is added, add it to DNS > Disadvantages to this are specific to DNS round-robin: > - load is not balanced if "central" servers are not the same > - there is a lag due to DNS caching > > And I guess another solution is to have a script to run after > deployment, which will change the rsyslog.conf on all the clients. But > that's doesn't seem to be a good idea, especially since there's no way > to actually balance load on the aggregators. Only to make some clients > to log to one, others to another, etc. > > Do you have any thoughts on how to solve the problem? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From vgjunkie at gmail.com Mon Jan 30 18:08:08 2012 From: vgjunkie at gmail.com (Video Game Junkie) Date: Mon, 30 Jan 2012 12:08:08 -0500 Subject: [rsyslog] Problems building Rsyslog 6.2 on Centos 6.2 (64 bit) Message-ID: Has anyone had success building Rsyslog 6.2 on a Centos 6.2 system? I'm stuck at trying to compile libee (I've tried versions 0.3.1 and 0.3.2). It ends up with an error trying during make: /usr: file not recognized: Is a directory collect2: ld returned 1 exit status Any help would be appreciated. Thanks Jeff From david at lang.hm Mon Jan 30 21:43:26 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 30 Jan 2012 12:43:26 -0800 (PST) Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: On Mon, 30 Jan 2012, Radu Gheorghe wrote: > Hello, > > I'm currently using rsyslog in the following setup: > - a bunch of clients send Syslog messages to a central rsyslog daemon via TCP > - right now, the central rsyslog daemon pipes the logs to both a plain > text file and an external program. But I guess that's not so relevant > > Soon, this central rsyslog would be overwhelmed by the amount of > logging from the clients. And I'm looking for a solution to be able to > deploy new "central" rsyslog daemons. > > I thought about using DNS round robin: > - configure clients to send logs to a single hostname > - once a new "central" rsyslog is added, add it to DNS > Disadvantages to this are specific to DNS round-robin: > - load is not balanced if "central" servers are not the same > - there is a lag due to DNS caching > > And I guess another solution is to have a script to run after > deployment, which will change the rsyslog.conf on all the clients. But > that's doesn't seem to be a good idea, especially since there's no way > to actually balance load on the aggregators. Only to make some clients > to log to one, others to another, etc. > > Do you have any thoughts on how to solve the problem? what I do is use iptables CLUSTERIP feature on linux to setup one IP address that gets shared across the cluster of systems. heartbeat (with the pacemaker cluster management layer) can keep track of the cluster and make sure that there is always a box handling the traffic what this does is use a multicast MAC address to send the traffic to multiple machines. The kernel then does a hash on (one or more of) source IP, source port, destination IP, destination port. It then divides this hash into buckets (I am machine 1 of 10) and if it falls into the bucket for this machine, it then sends the packet on to the application, otherwise the kernel drops the packet. This has the advantage of not needing any other systems, it can be done entirely on the receiving cluster. Another option you could do with TCP traffic is to setup a LVS (Linux Virtual Server) load balancer (or run it through any commercial load balancer) In any of these configurations, you will want to consider the tcprebindinterval config option of rsyslog on the sending machines so that they will periodically close and re-open their connection (so that the source port changes), otherwise you can end up with the traffic being unbalanced between your systems without any way to re-balance the load. David Lang From david at lang.hm Mon Jan 30 21:52:28 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 30 Jan 2012 12:52:28 -0800 (PST) Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: On Mon, 30 Jan 2012, david at lang.hm wrote: >> Do you have any thoughts on how to solve the problem? > > what I do is use iptables CLUSTERIP feature on linux to setup one IP address > that gets shared across the cluster of systems. heartbeat (with the pacemaker > cluster management layer) can keep track of the cluster and make sure that > there is always a box handling the traffic > > what this does is use a multicast MAC address to send the traffic to multiple > machines. The kernel then does a hash on (one or more of) source IP, source > port, destination IP, destination port. It then divides this hash into > buckets (I am machine 1 of 10) and if it falls into the bucket for this > machine, it then sends the packet on to the application, otherwise the kernel > drops the packet. > > This has the advantage of not needing any other systems, it can be done > entirely on the receiving cluster. here is a page on how to configure pacemaker to do this. www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Clusters_from_Scratch/ch08s06.html I primarily use this with UDP traffic, which gives me the added advantage that I can have multiple clusters receiving the same traffic. I did extensive testing a couple of years ago, and going across a cisco 3500 switch I was able to handle traffic up to ~380K logs/sec (~250 byte log messages) with no message losses with UDP over several billion log messages sent to a dozen destination machines. David Lang From radu0gheorghe at gmail.com Tue Jan 31 10:07:49 2012 From: radu0gheorghe at gmail.com (Radu Gheorghe) Date: Tue, 31 Jan 2012 11:07:49 +0200 Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: Thanks a lot, David. These solutions seem much better than DNS round robin. @Rainer: I'm having trouble with setting up more exact performance tests, but the system is supposed to scale to something like 50K messages per second. And I just assumed that one sever won't handle the load, especially since these machines are slow. But you have a good point, because the external program inserts these logs in ElasticSearch. And I have ElasticSearch on the same server right now, which I don't have to. Using a dedicated server for Rsyslog only might not cut it in the long run, but it would probably work well for a while. More than enough for me to set up a cluster :D So, thanks again. I consider the issue "solved". Best regards, Radu From rgerhards at hq.adiscon.com Tue Jan 31 10:23:56 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 31 Jan 2012 10:23:56 +0100 Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281817@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Radu Gheorghe > Sent: Tuesday, January 31, 2012 10:08 AM > To: rsyslog-users > Subject: Re: [rsyslog] Load balancing for rsyslog aggregators > > Thanks a lot, David. These solutions seem much better than DNS round > robin. > > @Rainer: I'm having trouble with setting up more exact performance > tests, but the system is supposed to scale to something like 50K > messages per second. And I just assumed that one sever won't handle I do 100kmsgs/second on a 2 year old travel notebook regularly, even in a vmware environment with just an "old" intel duo-core notebook processor... ;) Rainer > the load, especially since these machines are slow. > > But you have a good point, because the external program inserts these > logs in ElasticSearch. And I have ElasticSearch on the same server > right now, which I don't have to. Using a dedicated server for Rsyslog > only might not cut it in the long run, but it would probably work well > for a while. More than enough for me to set up a cluster :D > > So, thanks again. I consider the issue "solved". > > Best regards, > Radu > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From david at lang.hm Tue Jan 31 11:48:01 2012 From: david at lang.hm (david at lang.hm) Date: Tue, 31 Jan 2012 02:48:01 -0800 (PST) Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: On Tue, 31 Jan 2012, Radu Gheorghe wrote: > Thanks a lot, David. These solutions seem much better than DNS round robin. > > @Rainer: I'm having trouble with setting up more exact performance > tests, but the system is supposed to scale to something like 50K > messages per second. And I just assumed that one sever won't handle > the load, especially since these machines are slow. > > But you have a good point, because the external program inserts these > logs in ElasticSearch. And I have ElasticSearch on the same server > right now, which I don't have to. Using a dedicated server for Rsyslog > only might not cut it in the long run, but it would probably work well > for a while. More than enough for me to set up a cluster :D the best thing that you could do for your performance is to commission the writng of an output module that would let rsyslog insert the messages into ElasticSearch instead of doing it with an external program. At first glance this may seem like a trivial change, but the killer feature that you can take advantage of with an output module is the ability to handle multiple log messages as a single transaction. I'm not familiar with ElasticSearch, but it's common for databases to be able to handle inserts of 100 or even 1000 records as a single transaction at exactly the same transaction/sec rate as inserting a single record per transaction (or at a very slight reduction in insert rate). I've seen good database setups where 10,000 inserts as a single transaction was only 1/2 the transaction rate of one insert per transaction (a 5,000x speedup) Adiscon does this sort of work (contact Rainer directly if you want a quote) but 50K logs/sec is not likely to end up with rsyslog as the bottlneck. You should setup a test environment and stress test things to see how high you can push the message rate before you can't keep up. There are a number of variables that can end up being the bottleneck and you want to find these in testing, not in production :-) The fist thing is that you want to be running a very recent rsyslog (5.8.x or 6.x), the speedups in rsyslog since 4.x (which is in RHEL5 I believe) are very significant. 6.3.x introduces a DNS cache that can be a drastic speedup if you need DNS lookups (if not, you can start rsyslog with -x to disable them on earlier versions) you also need to define 'slow hardware', one person's slow hardware is another person's mid-range server :-) David Lang From rgerhards at hq.adiscon.com Tue Jan 31 14:55:46 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 31 Jan 2012 14:55:46 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728181F@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, January 27, 2012 7:35 PM > To: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > the surest way to tell that the DNS cache is working is to look at traffic to your > DNS server (or more precisely the lack of traffic to that host) I just checked the code. In the current v6-devel, the dns cache is always active. Rainer From michael at maymann.org Tue Jan 31 15:01:43 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 31 Jan 2012 15:01:43 +0100 Subject: [rsyslog] rsyslog as non-root user Message-ID: Hi, I have now setup a 6.3.6-devel rsyslog server that is working fine running as root. I would like to run it as non-root user as my logfiles are located on NFS (and root export of NFS is generally not a good idea !). Here is my rsyslog.conf: #LOAD MODULES $ModLoad imudp $UDPServerRun 514 $UDPServerAddress 127.0.0.1 $ModLoad imtcp $InputTCPServerRun 514 #SET DESTINATION FOR LOGS $template DYNmessages,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_messages" $template DYNsecure,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_secure" $template DYNmaillog,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_maillog" $template DYNcron,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_cron" $template DYNspooler,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_spooler" $template DYNboot,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_boot.log" $template DYNtraps,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONTH%_traps" #SET LOGGING CONDITIONS if $syslogseverity <= '6' then ?DYNmessages if $syslogfacility-text == 'authpriv' then ?DYNsecure if $syslogfacility-text == 'mail' then ?DYNmaillog if $syslogfacility-text == 'cron' then ?DYNcron if $syslogseverity-text == 'crit' then ?DYNspooler if $syslogfacility-text == 'local7' then ?DYNboot if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' then ?DYNtraps Here is my logfile when I try to start rsyslog as a non-root user: 2012-01-31T15:45:52.997693+02:00 rsyslogd: [origin software="rsyslogd" swVersion="6.3.6" x-pid="26185" x-info=" http://www.rsyslog.com"] start 2012-01-31T15:45:52.997294+02:00 rsyslogd: bind: Permission denied 2012-01-31T15:45:52.997369+02:00 rsyslogd: bind: Permission denied 2012-01-31T15:45:52.997374+02:00 rsyslogd: No UDP listen socket could successfully be initialized, message reception via UDP disabled. 2012-01-31T15:45:52.997376+02:00 rsyslogd: imudp: no listeners could be started, input not activated. 2012-01-31T15:45:52.997379+02:00 rsyslogd3: activation of module imudp failed [try http://www.rsyslog.com/e/-3 ] 2012-01-31T15:45:52.997643+02:00 rsyslogd-2077: Could not create tcp listener, ignoring port 514. [try http://www.rsyslog.com/e/2077 ] So permissions to bind and sockets seems to be the problem... 1. Is it possible to make rsyslog write logfiles as a non-root user - if yes: how ? 2a. Is it possible to add permissions for non-root user to run rsyslog server - if yes: how ? 2b. How do I start rsyslog during boot as non-root user - can chkconfig do this ? do I need to edit /etc/init.d/rsyslog - if yes: how ? Thanks in advance :-) ! ~maymann From michael at maymann.org Tue Jan 31 15:03:11 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 31 Jan 2012 15:03:11 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728181F@GRFEXC.intern.adiscon.com> References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA728181F@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, just noticed the same - see new post...:-) ! Thanks for your help. Br. ~maymann 2012/1/31 Rainer Gerhards > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > > Sent: Friday, January 27, 2012 7:35 PM > > To: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > howto/links/examples > > > > the surest way to tell that the DNS cache is working is to look at > traffic > to your > > DNS server (or more precisely the lack of traffic to that host) > > I just checked the code. In the current v6-devel, the dns cache is always > active. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Tue Jan 31 15:05:31 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 31 Jan 2012 15:05:31 +0100 Subject: [rsyslog] rsyslog as non-root user In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281820@GRFEXC.intern.adiscon.com> The right approach is to use the permission drop features of rsyslog to make it itself run as non-root. I v6, it will run as root only to open critical things (like the UDP ports) and then drops privileges (in a way that never can be undone without a full restart). HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > Sent: Tuesday, January 31, 2012 3:02 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] rsyslog as non-root user > > Hi, > > I have now setup a 6.3.6-devel rsyslog server that is working fine running as > root. > I would like to run it as non-root user as my logfiles are located on NFS (and > root export of NFS is generally not a good idea !). > > Here is my rsyslog.conf: > #LOAD MODULES > $ModLoad imudp > $UDPServerRun 514 > $UDPServerAddress 127.0.0.1 > $ModLoad imtcp > $InputTCPServerRun 514 > #SET DESTINATION FOR LOGS > $template > DYNmessages,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$ > MONTH%_messages" > $template > DYNsecure,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > NTH%_secure" > $template > DYNmaillog,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > NTH%_maillog" > $template > DYNcron,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > H%_cron" > $template > DYNspooler,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > NTH%_spooler" > $template > DYNboot,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > H%_boot.log" > $template > DYNtraps,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > H%_traps" > #SET LOGGING CONDITIONS > if $syslogseverity <= '6' then ?DYNmessages if $syslogfacility-text == > 'authpriv' then ?DYNsecure if $syslogfacility-text == 'mail' then ?DYNmaillog if > $syslogfacility-text == 'cron' then ?DYNcron if $syslogseverity-text == 'crit' > then ?DYNspooler if $syslogfacility-text == 'local7' then ?DYNboot if > $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' > then ?DYNtraps > > Here is my logfile when I try to start rsyslog as a non-root user: > 2012-01-31T15:45:52.997693+02:00 rsyslogd: [origin > software="rsyslogd" swVersion="6.3.6" x-pid="26185" x-info=" > http://www.rsyslog.com"] start > 2012-01-31T15:45:52.997294+02:00 rsyslogd: bind: Permission > denied > 2012-01-31T15:45:52.997369+02:00 rsyslogd: bind: Permission > denied > 2012-01-31T15:45:52.997374+02:00 rsyslogd: No UDP listen > socket could successfully be initialized, message reception via UDP disabled. > 2012-01-31T15:45:52.997376+02:00 rsyslogd: imudp: no listeners > could be started, input not activated. > 2012-01-31T15:45:52.997379+02:00 rsyslogd3: activation of > module imudp failed [try http://www.rsyslog.com/e/-3 ] > 2012-01-31T15:45:52.997643+02:00 rsyslogd-2077: Could not > create tcp listener, ignoring port 514. [try http://www.rsyslog.com/e/2077 ] > > So permissions to bind and sockets seems to be the problem... > > 1. Is it possible to make rsyslog write logfiles as a non-root user - if > yes: how ? > 2a. Is it possible to add permissions for non-root user to run rsyslog server - if > yes: how ? > 2b. How do I start rsyslog during boot as non-root user - can chkconfig do this > ? do I need to edit /etc/init.d/rsyslog - if yes: how ? > > > Thanks in advance :-) ! > ~maymann > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From vladg at illinois.edu Tue Jan 31 15:36:08 2012 From: vladg at illinois.edu (Vlad Grigorescu) Date: Tue, 31 Jan 2012 08:36:08 -0600 Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: <4F27FC58.6050300@illinois.edu> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 1/31/12 4:48 AM, david at lang.hm wrote: > the best thing that you could do for your performance is to commission the writng of an output module that would let rsyslog insert the messages into ElasticSearch instead of doing it with an external program. There's already a user-submitted output module available for this: --Vlad -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) iQIcBAEBAgAGBQJPJ/xYAAoJEMEVj6tjLlJyn0IP/2wgPDovzA7mpTX4I9ok2e+p 3jBdenCMxgpKihcIo9ssA439w0AxQtesZgxmN5zsc6kW/bifQEimTHw6gKTj5jWH lZJMESQH622IGqCnj05vJdurJIYu/EJV0stqH+b7cotpnnRTUEPtIYjZhMebnWUy +AgYdarFF6uJeFlvNNrtU7Sfx5X13b6qIkS4ESVzhiLts8UT8Onv7XekVAngrObg RCy4n/cZQ/p4g9KLZa6Y8W03SkytWAzFR3n975ZWJcA47HTYaZYWSD8Xu0F2Hs66 smeq4yv5qYjRvF52CH5Mg6/jsgWEc2opn0qWamUqY3Cu0R/LV4d1JWEDEiVVXcYq Ig6mOqPqKb5TgBSN7JMC6uFxTHA/WH+rv3yJEr928mMeP7444sfqg9Afq9Q4xai2 ZJSia1fxUyk7B+shrQ6/kXYYzpZCREH0Th7MHYFEowne4q/SRAYf+R+i7DbjDM4H +Ifo8BYKN9PphP7kLTqaA3XSQ6JjaR1QYHFpvtE0jqFBn6CQzi5HEIqoHL5Yv3K6 dvLtr5zQEAXVKxAYERDSy4sfCjADBxd+E3gijWyVyR/3BwaykbeSC4BO+/wxU9HV pJ7i0n1/uMbKzBDxkEnavqyi//+16UF/J0qg+2NfvMpzqTJZ5Z2Bcp3b0hI1uqh3 zDtLKC5YyAEZHzTSDynW =Wq6z -----END PGP SIGNATURE----- From rgerhards at hq.adiscon.com Tue Jan 31 16:00:38 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 31 Jan 2012 16:00:38 +0100 Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: <4F27FC58.6050300@illinois.edu> References: <4F27FC58.6050300@illinois.edu> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281822@GRFEXC.intern.adiscon.com> I recently merged it, so it is in git for all versions (for released ones, see ChangeLog). rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Vlad Grigorescu > Sent: Tuesday, January 31, 2012 3:36 PM > To: rsyslog-users > Subject: Re: [rsyslog] Load balancing for rsyslog aggregators > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 1/31/12 4:48 AM, david at lang.hm wrote: > > the best thing that you could do for your performance is to commission the > writng of an output module that would let rsyslog insert the messages into > ElasticSearch instead of doing it with an external program. > > There's already a user-submitted output module available for this: > > > > > > --Vlad > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > > iQIcBAEBAgAGBQJPJ/xYAAoJEMEVj6tjLlJyn0IP/2wgPDovzA7mpTX4I9ok2e+p > 3jBdenCMxgpKihcIo9ssA439w0AxQtesZgxmN5zsc6kW/bifQEimTHw6gKTj5jW > H > lZJMESQH622IGqCnj05vJdurJIYu/EJV0stqH+b7cotpnnRTUEPtIYjZhMebnWUy > +AgYdarFF6uJeFlvNNrtU7Sfx5X13b6qIkS4ESVzhiLts8UT8Onv7XekVAngrObg > RCy4n/cZQ/p4g9KLZa6Y8W03SkytWAzFR3n975ZWJcA47HTYaZYWSD8Xu0F2Hs > 66 > smeq4yv5qYjRvF52CH5Mg6/jsgWEc2opn0qWamUqY3Cu0R/LV4d1JWEDEiVV > XcYq > Ig6mOqPqKb5TgBSN7JMC6uFxTHA/WH+rv3yJEr928mMeP7444sfqg9Afq9Q4x > ai2 > ZJSia1fxUyk7B+shrQ6/kXYYzpZCREH0Th7MHYFEowne4q/SRAYf+R+i7DbjDM4 > H > +Ifo8BYKN9PphP7kLTqaA3XSQ6JjaR1QYHFpvtE0jqFBn6CQzi5HEIqoHL5Yv3K6 > dvLtr5zQEAXVKxAYERDSy4sfCjADBxd+E3gijWyVyR/3BwaykbeSC4BO+/wxU9 > HV > pJ7i0n1/uMbKzBDxkEnavqyi//+16UF/J0qg+2NfvMpzqTJZ5Z2Bcp3b0hI1uqh3 > zDtLKC5YyAEZHzTSDynW > =Wq6z > -----END PGP SIGNATURE----- > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From radu0gheorghe at gmail.com Tue Jan 31 16:01:08 2012 From: radu0gheorghe at gmail.com (Radu Gheorghe) Date: Tue, 31 Jan 2012 17:01:08 +0200 Subject: [rsyslog] Load balancing for rsyslog aggregators In-Reply-To: References: Message-ID: 2012/1/31 : > the best thing that you could do for your performance is to commission the > writng of an output module that would let rsyslog insert the messages into > ElasticSearch instead of doing it with an external program. > > At first glance this may seem like a trivial change, but the killer feature > that you can take advantage of with an output module is the ability to > handle multiple log messages as a single transaction. > > I'm not familiar with ElasticSearch, but it's common for databases to be > able to handle inserts of 100 or even 1000 records as a single transaction > at exactly the same transaction/sec rate as inserting a single record per > transaction (or at a very slight reduction in insert rate). I've seen good > database setups where 10,000 inserts as a single transaction was only 1/2 > the transaction rate of one insert per transaction (a 5,000x speedup) > I know there is an ES plugin available in the development version but I couldn't get it to work: http://kb.monitorware.com/can-install-elasticsearch-output-module-t11309.html My script does bulk inserts already (I'm inserting each second). So there shouldn't be a significant performance gain by using an rsyslog plugin. Although I would prefer using plugins anyway. > Adiscon does this sort of work (contact Rainer directly if you want a quote) > > but 50K logs/sec is not likely to end up with rsyslog as the bottlneck. You > should setup a test environment and stress test things to see how high you > can push the message rate before you can't keep up. There are a number of > variables that can end up being the bottleneck and you want to find these in > testing, not in production :-) Yes, I will do some proper testing and consider solutions afterwards. Sorry for not doing my homework properly in the first place :( > > The fist thing is that you want to be running a very recent rsyslog (5.8.x > or 6.x), the speedups in rsyslog since 4.x (which is in RHEL5 I believe) are > very significant. 6.3.x introduces a DNS cache that can be a drastic speedup > if you need DNS lookups (if not, you can start rsyslog with -x to disable > them on earlier versions) > > you also need to define 'slow hardware', one person's slow hardware is > another person's mid-range server :-) I guess defining 'slow hardware' must come after proper testing... So I won't go there for now :) From michael at maymann.org Tue Jan 31 16:03:41 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 31 Jan 2012 16:03:41 +0100 Subject: [rsyslog] rsyslog as non-root user In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281820@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281820@GRFEXC.intern.adiscon.com> Message-ID: Hi, Rainer: thanks for this...:-) ! Now I have added the following to rsyslog.conf: $PrivDropToGroup $PrivDropToUser When starting rsyslog, it hangs for a long time and fails - but processes are started fine and everyting is otherwise working fine: # service rsyslog start Starting system logger: [FAILED] Running proccesses during hang...: root 26710 20551 0 16:48 pts/0 00:00:00 /bin/sh /sbin/service rsyslog start root 26715 26710 0 16:48 pts/0 00:00:00 /bin/bash /etc/init.d/rsyslog start root 26718 26715 0 16:48 pts/0 00:00:00 /bin/bash -c ulimit -S -c 0 >/dev/null 2>&1 ; /usr/sbin/rsyslogd -c 6 root 26719 26718 0 16:48 pts/0 00:00:00 /usr/sbin/rsyslogd -c 6 26724 26719 0 16:48 ? 00:00:00 /usr/sbin/rsyslogd -c 6 Running proccesses after "failed" statement: 26724 1 0 16:48 ? 00:00:00 /usr/sbin/rsyslogd -c 6 Stopping rsyslog causes no problem though... # service rsyslog stop Shutting down system logger: [ OK ] It doesn't state anything wrong in the logs: 2012-01-31T16:48:55.612782+02:00 HOSTNAME rsyslogd: [origin software="rsyslogd" swVersion="6.3.6" x-pid="26724" x-info=" http://www.rsyslog.com"] start 2012-01-31T16:48:55.612664+02:00 HOSTNAME rsyslogd: rsyslogd's groupid changed to 2012-01-31T16:48:55.612709+02:00 HOSTNAME rsyslogd: rsyslogd's userid changed to Rainer: It is working now, but FYI: I will send you the debug information directly... Thanks for your help :-) ! Br. ~maymann 2012/1/31 Rainer Gerhards > The right approach is to use the permission drop features of rsyslog to > make > it itself run as non-root. I v6, it will run as root only to open critical > things (like the UDP ports) and then drops privileges (in a way that never > can be undone without a full restart). > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Tuesday, January 31, 2012 3:02 PM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] rsyslog as non-root user > > > > Hi, > > > > I have now setup a 6.3.6-devel rsyslog server that is working fine > running > as > > root. > > I would like to run it as non-root user as my logfiles are located on NFS > (and > > root export of NFS is generally not a good idea !). > > > > Here is my rsyslog.conf: > > #LOAD MODULES > > $ModLoad imudp > > $UDPServerRun 514 > > $UDPServerAddress 127.0.0.1 > > $ModLoad imtcp > > $InputTCPServerRun 514 > > #SET DESTINATION FOR LOGS > > $template > > DYNmessages,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$ > > MONTH%_messages" > > $template > > DYNsecure,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > > NTH%_secure" > > $template > > DYNmaillog,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > > NTH%_maillog" > > $template > > DYNcron,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > > H%_cron" > > $template > > DYNspooler,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > > NTH%_spooler" > > $template > > DYNboot,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > > H%_boot.log" > > $template > > DYNtraps,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > > H%_traps" > > #SET LOGGING CONDITIONS > > if $syslogseverity <= '6' then ?DYNmessages if $syslogfacility-text == > > 'authpriv' then ?DYNsecure if $syslogfacility-text == 'mail' then > ?DYNmaillog if > > $syslogfacility-text == 'cron' then ?DYNcron if $syslogseverity-text == > 'crit' > > then ?DYNspooler if $syslogfacility-text == 'local7' then ?DYNboot if > > $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' > > then ?DYNtraps > > > > Here is my logfile when I try to start rsyslog as a non-root user: > > 2012-01-31T15:45:52.997693+02:00 rsyslogd: [origin > > software="rsyslogd" swVersion="6.3.6" x-pid="26185" x-info=" > > http://www.rsyslog.com"] start > > 2012-01-31T15:45:52.997294+02:00 rsyslogd: bind: Permission > > denied > > 2012-01-31T15:45:52.997369+02:00 rsyslogd: bind: Permission > > denied > > 2012-01-31T15:45:52.997374+02:00 rsyslogd: No UDP listen > > socket could successfully be initialized, message reception via UDP > disabled. > > 2012-01-31T15:45:52.997376+02:00 rsyslogd: imudp: no listeners > > could be started, input not activated. > > 2012-01-31T15:45:52.997379+02:00 rsyslogd3: activation of > > module imudp failed [try http://www.rsyslog.com/e/-3 ] > > 2012-01-31T15:45:52.997643+02:00 rsyslogd-2077: Could not > > create tcp listener, ignoring port 514. [try > http://www.rsyslog.com/e/2077 > ] > > > > So permissions to bind and sockets seems to be the problem... > > > > 1. Is it possible to make rsyslog write logfiles as a non-root user - if > > yes: how ? > > 2a. Is it possible to add permissions for non-root user to run rsyslog > server - if > > yes: how ? > > 2b. How do I start rsyslog during boot as non-root user - can chkconfig > do > this > > ? do I need to edit /etc/init.d/rsyslog - if yes: how ? > > > > > > Thanks in advance :-) ! > > ~maymann > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Tue Jan 31 16:05:50 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Tue, 31 Jan 2012 16:05:50 +0100 Subject: [rsyslog] rsyslog as non-root user In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281820@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281823@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > Sent: Tuesday, January 31, 2012 4:04 PM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog as non-root user > > Hi, > > Rainer: thanks for this...:-) ! > > Now I have added the following to rsyslog.conf: > $PrivDropToGroup > $PrivDropToUser > > When starting rsyslog, it hangs for a long time and fails - but processes are > started fine and everyting is otherwise working fine: > # service rsyslog start > Starting system logger: [FAILED] > > Running proccesses during hang...: > root 26710 20551 0 16:48 pts/0 00:00:00 /bin/sh /sbin/service > rsyslog start > root 26715 26710 0 16:48 pts/0 00:00:00 /bin/bash > /etc/init.d/rsyslog start > root 26718 26715 0 16:48 pts/0 00:00:00 /bin/bash -c ulimit -S -c 0 > >/dev/null 2>&1 ; /usr/sbin/rsyslogd -c 6 > root 26719 26718 0 16:48 pts/0 00:00:00 /usr/sbin/rsyslogd -c 6 > 26724 26719 0 16:48 ? 00:00:00 /usr/sbin/rsyslogd -c 6 > > Running proccesses after "failed" statement: > 26724 1 0 16:48 ? 00:00:00 /usr/sbin/rsyslogd -c 6 > > Stopping rsyslog causes no problem though... > # service rsyslog stop > Shutting down system logger: [ OK ] > > It doesn't state anything wrong in the logs: > 2012-01-31T16:48:55.612782+02:00 HOSTNAME rsyslogd: [origin > software="rsyslogd" swVersion="6.3.6" x-pid="26724" x-info=" > http://www.rsyslog.com"] start > 2012-01-31T16:48:55.612664+02:00 HOSTNAME rsyslogd: rsyslogd's groupid > changed to > 2012-01-31T16:48:55.612709+02:00 HOSTNAME rsyslogd: rsyslogd's userid > changed to > > Rainer: It is working now, but FYI: I will send you the debug information > directly... Yeah, that would be great. Be sure to include a full debug log. Rainer > > Thanks for your help :-) ! > > > Br. > ~maymann > > > 2012/1/31 Rainer Gerhards > > > The right approach is to use the permission drop features of rsyslog > > to make it itself run as non-root. I v6, it will run as root only to > > open critical things (like the UDP ports) and then drops privileges > > (in a way that never can be undone without a full restart). > > > > HTH > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > > Sent: Tuesday, January 31, 2012 3:02 PM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] rsyslog as non-root user > > > > > > Hi, > > > > > > I have now setup a 6.3.6-devel rsyslog server that is working fine > > running > > as > > > root. > > > I would like to run it as non-root user as my logfiles are located > > > on NFS > > (and > > > root export of NFS is generally not a good idea !). > > > > > > Here is my rsyslog.conf: > > > #LOAD MODULES > > > $ModLoad imudp > > > $UDPServerRun 514 > > > $UDPServerAddress 127.0.0.1 > > > $ModLoad imtcp > > > $InputTCPServerRun 514 > > > #SET DESTINATION FOR LOGS > > > $template > > > > DYNmessages,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$ > > > MONTH%_messages" > > > $template > > > > DYNsecure,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > > > NTH%_secure" > > > $template > > > > DYNmaillog,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > > > NTH%_maillog" > > > $template > > > > DYNcron,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > > > H%_cron" > > > $template > > > > DYNspooler,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MO > > > NTH%_spooler" > > > $template > > > > DYNboot,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > > > H%_boot.log" > > > $template > > > > DYNtraps,"/%FROMHOST%/%FROMHOST%_%$YEAR%.%$MONT > > > H%_traps" > > > #SET LOGGING CONDITIONS > > > if $syslogseverity <= '6' then ?DYNmessages if $syslogfacility-text > > > == 'authpriv' then ?DYNsecure if $syslogfacility-text == 'mail' then > > ?DYNmaillog if > > > $syslogfacility-text == 'cron' then ?DYNcron if $syslogseverity-text > > > == > > 'crit' > > > then ?DYNspooler if $syslogfacility-text == 'local7' then ?DYNboot > > > if $syslogfacility-text == 'local6' and $syslogseverity-text == 'WARNING' > > > then ?DYNtraps > > > > > > Here is my logfile when I try to start rsyslog as a non-root user: > > > 2012-01-31T15:45:52.997693+02:00 rsyslogd: [origin > > > software="rsyslogd" swVersion="6.3.6" x-pid="26185" x-info=" > > > http://www.rsyslog.com"] start > > > 2012-01-31T15:45:52.997294+02:00 rsyslogd: bind: > > > Permission denied > > > 2012-01-31T15:45:52.997369+02:00 rsyslogd: bind: > > > Permission denied > > > 2012-01-31T15:45:52.997374+02:00 rsyslogd: No UDP listen > > > socket could successfully be initialized, message reception via UDP > > disabled. > > > 2012-01-31T15:45:52.997376+02:00 rsyslogd: imudp: no > > > listeners could be started, input not activated. > > > 2012-01-31T15:45:52.997379+02:00 rsyslogd3: activation of > > > module imudp failed [try http://www.rsyslog.com/e/-3 ] > > > 2012-01-31T15:45:52.997643+02:00 rsyslogd-2077: Could not > > > create tcp listener, ignoring port 514. [try > > http://www.rsyslog.com/e/2077 > > ] > > > > > > So permissions to bind and sockets seems to be the problem... > > > > > > 1. Is it possible to make rsyslog write logfiles as a non-root user > > > - if > > > yes: how ? > > > 2a. Is it possible to add permissions for non-root user to run > > > rsyslog > > server - if > > > yes: how ? > > > 2b. How do I start rsyslog during boot as non-root user - can > > > chkconfig > > do > > this > > > ? do I need to edit /etc/init.d/rsyslog - if yes: how ? > > > > > > > > > Thanks in advance :-) ! > > > ~maymann > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From bodik at civ.zcu.cz Wed Jan 4 18:09:45 2012 From: bodik at civ.zcu.cz (bodik) Date: Wed, 04 Jan 2012 18:09:45 +0100 Subject: [rsyslog] omgssapi segfault In-Reply-To: <4EE8CDA6.4050400@redhat.com> References: <4EE6843E.7040905@civ.zcu.cz> <4EE78EBA.6050006@redhat.com> <4EE7C388.5060700@civ.zcu.cz> <4EE8CDA6.4050400@redhat.com> Message-ID: <4F0487D9.8060409@civ.zcu.cz> Dne 12/14/11 17:24, Tomas Heinrich napsal(a): > On 12/13/2011 10:28 PM, bodik wrote: >> thanks for reply I tried this one, at it stops client from segfault ... >> is that what you mean to do ? or we shloud discard "case eDestFORW:" >> completely ? > > My guess at the solution is attached. Please let me know if it works for you. hi, i dig a little bit deeper and thgough testing found that suggested patch is not working well while remote server is rejecting connections by RST (server is up, but rsyslog down ..) 17:47:28.878679 IP aaa.bbb.ccc.125.35114 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.878714 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35114: Flags [R.], seq 17:47:28.880490 IP aaa.bbb.ccc.125.35115 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.880524 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35115: Flags [R.], seq 17:47:28.882414 IP aaa.bbb.ccc.125.35116 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.882445 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35116: Flags [R.], seq 17:47:28.884585 IP aaa.bbb.ccc.125.35117 > aaa.bbb.ccc.124.515: Flags [S], seq 17:47:28.884623 IP aaa.bbb.ccc.124.515 > aaa.bbb.ccc.125.35117: Flags [R.], seq ... very many packets possible from many hosts ... i managed to follow the instruction you left us, about omfwd.c and i have refactored old plugin in favor of new omfwd.c. I'd like to ask is attached solution could be accepted to trunk. it's a little bit dirty because i did not strip all of UDP stuff, but we can work on it either way to support it or remove completely .. best regards bodik -------------- next part -------------- A non-text attachment was scrubbed... Name: omgssapi.c Type: text/x-csrc Size: 20790 bytes Desc: not available URL: From bodik at civ.zcu.cz Wed Jan 4 18:13:42 2012 From: bodik at civ.zcu.cz (bodik) Date: Wed, 04 Jan 2012 18:13:42 +0100 Subject: [rsyslog] omgssapi segfault In-Reply-To: <4F0487D9.8060409@civ.zcu.cz> References: <4EE6843E.7040905@civ.zcu.cz> <4EE78EBA.6050006@redhat.com> <4EE7C388.5060700@civ.zcu.cz> <4EE8CDA6.4050400@redhat.com> <4F0487D9.8060409@civ.zcu.cz> Message-ID: <4F0488C6.1000308@civ.zcu.cz> > I'd like to ask is attached solution could be accepted to trunk. it's a little sory, that was a wrong version .. here comes the right one bodik -------------- next part -------------- A non-text attachment was scrubbed... Name: omgssapi.c Type: text/x-csrc Size: 25806 bytes Desc: not available URL: From kaiwang.chen at gmail.com Thu Jan 5 15:29:40 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Thu, 5 Jan 2012 22:29:40 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: Message-ID: Hello, I check ChangeLog and found 5.8.4 fixed some bug related to race condition as discussed in http://bugzilla.adiscon.com/show_bug.cgi?id=275 with patch protecting modification in the getMSGID(): http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb356cc230685c5b7c90f8590b4 I guess the situation is almost identical to what in my case: corruption is always at the APPNAME field and there is, I believe, a typo in line 1955 from getAPPNAME() effectively ruins the mutex proctection: 1947 /* rgerhards, 2005-11-24 1948 */ 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) 1950 { 1951 uchar *pszRet; 1952 1953 assert(pM != NULL); 1954 if(bLockMutex == LOCK_MUTEX) 1955 MsgUnlock(pM); /* should be MsgLock(pM) */ 1956 prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); 1957 if(pM->pCSAPPNAME == NULL) 1958 pszRet = UCHAR_CONSTANT(""); 1959 else 1960 pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); 1961 if(bLockMutex == LOCK_MUTEX) 1962 MsgUnlock(pM); 1963 return (char*)pszRet; 1964 } Similar cases were spotted in getPROCID(), getStructuredData() and getProgramName(). Hopefully the following patch against 5.8.6 tarball should solve the problem, I will report back later. --- rsyslog-5.8.6/runtime/msg.c.orig 2011-10-21 17:53:02.000000000 +0800 +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 @@ -1609,7 +1609,7 @@ ISOBJ_TYPE_assert(pM, msg); if(bLockMutex == LOCK_MUTEX) - MsgUnlock(pM); + MsgLock(pM); preparePROCID(pM, MUTEX_ALREADY_LOCKED); if(pM->pCSPROCID == NULL) pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ { uchar *pszRet; - MsgUnlock(pM); + MsgLock(pM); if(pM->pCSStrucData == NULL) pszRet = UCHAR_CONSTANT("-"); else @@ -1894,7 +1894,7 @@ uchar *pszRet; if(bLockMutex == LOCK_MUTEX) - MsgUnlock(pM); + MsgLock(pM); prepareProgramName(pM, MUTEX_ALREADY_LOCKED); if(pM->pCSProgName == NULL) pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ assert(pM != NULL); if(bLockMutex == LOCK_MUTEX) - MsgUnlock(pM); + MsgLock(pM); prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); if(pM->pCSAPPNAME == NULL) pszRet = UCHAR_CONSTANT(""); By the way, many crashes led to the convergence, for example //// db1 core.26581 signal 11, Segmentation fault. (gdb) p *pThis $1 = { pBuf = 0x7f7f5c000cd0 "sshd\177\177", pszBuf = 0x7f7f5c000dd0 "sshd", iBufSize = 4, iStrLen = 4 } (gdb) where #0 0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at stringbuf.c:334 #1 0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 '\001') at msg.c:1960 #2 0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, pTpe=0x12ef350, propID=115 's', pPropLen=0x43fb8f00, pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 #3 0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 #4 0x0000000000438498 in prepareBatch (pAction=0x12f72d0, pBatch=0x12f7898, pbShutdownImmediate=) at ../action.c:738 #5 processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, pbShutdownImmediate=) at ../action.c:1163 #6 0x000000000043190d in ConsumerReg (pThis=0x12f73f0, pWti=0x12f7870) at queue.c:1705 #7 0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 #8 0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 #9 0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 #10 0x0000000000000000 in ?? () Thanks, Kaiwang 2011/12/31 Kaiwang Chen : > Even worse case > <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog 5.8.6, > log source = /proc/kmsg started. > ...several SYN flooding entries in the same second... > *** glibc detected *** /sbin/rsyslogd: free(): invalid next size > (fast): 0x00007fafb8000e00 *** > *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: > 0x00007fafb8000e20 *** > > ?And possibly hours later the host was not responsive to ssh > connection requests, and even TTY login attempts failed... > Interestingly the debug log in another file continues a little beyond: > > Debug line with all properties: > FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, > syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', > APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > > Debug line with all properties: > FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, > syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', > APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > > Fortunately the squid service on that host was not badly affected. So > I did graceful failover and reboot the host, everything became OK. > Although there was no direct timestamp associated, I guess the problem > occured soon after rsyslogd started. I also spotted several corrupted > messages on other hosts when rsyslogd started. They are same version: > > # rsyslogd -v > rsyslogd 5.8.6, compiled with: > ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > ? ? ? ?FEATURE_DEBUG (debug build, slow code): No > ? ? ? ?32bit Atomic operations supported: ? ? ?Yes > ? ? ? ?64bit Atomic operations supported: ? ? ?Yes > ? ? ? ?Runtime Instrumentation (slow code): ? ?No > > > Thanks, > Kaiwang > > 2011/12/31 Kaiwang Chen : >> Also spotted corrupted messages from imuxsock: >> >> Debug line with all properties: >> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', PRI: 47, >> syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> ratelimit.numratelimiters=213 ' >> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> ratelimit.numratelimiters=213 ' >> inputname: impstats rawmsg: 'imuxsock: submitted=429 >> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >> >> >> Thanks, >> Kaiwang >> >> 2011/12/27 Kaiwang Chen : >>> 2011/12/25 ?: >>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >>>> >>>>> Hi all, >>>>> >>>>> I found rsyslogd occasionally produced corrupted log entries like >>>>> >>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >>>>> device eth0 entered promiscuous mode >>>>> >>>>> I believe that problem appeared in earlier versions including 5.8.2. >>>>> Looks like corruption never occur before hostname field. Is it a >>>>> reported bug? Any clue? >>>> >>>> >>>> the question is if this problem is in the raw message being sent to you, or >>>> is it something added by the rsyslog processing. >>>> >>>> I would suggest adding something along the following line. >>>> >>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >>>> >>>> this will look for any messages with the hex 01 character in them and spit >>>> out all the info that you can use about the message in to the file >>>> /var/log/badmessages >>>> >>>> run this for a little while and look to see what the raw message that is >>>> being received over the wire looks like. If it's bad, then you need to look >>>> at the sender. If the raw message looks sane, but rsyslog isn't handling it >>>> right, then we can troubleshoot from there. >>> >>> Looks like it's related to tcp transmission. On the original server >>> that get input from unix socket, it is OK; while on the centeral log >>> server, the raw message is already corrupted. >>> >>> ===== on central log server >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>> UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>> packet(s) from UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >>> Connection from UDP: [172.25.0.230]:53547 >>> >>> Debug line with all properties: >>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[2#032#0251]', programname: 'snmpd', APP-NAME: >>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>> UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>> packet(s) from UDP: [172.25.0.230]:53547 >>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>> UDP: [172.25.0.230]:53547 >>> >>> Debug line with all properties: >>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> Connection from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>> Debug line with all properties: >>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>> PROCID: '24254', MSGID: '-', >>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> Connection from UDP: [172.25.0.230]:53547' >>> >>> >>> Thanks, >>> Kaiwang >>>> >>>> David Lang >>>> >>>> ?> rsyslogd 5.8.6, compiled with: >>>>> >>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >>>>> >>>>> /etc/rsyslog.conf >>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>> >>>>> $ModLoad imklog >>>>> $ModLoad imuxsock >>>>> $ModLoad impstats >>>>> >>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >>>>> $SystemLogUsePIDFromSystem on >>>>> >>>>> $PStatInterval 600 >>>>> $PStatSeverity 7 >>>>> >>>>> $WorkDirectory /var/spool/rsyslog >>>>> >>>>> $MainMsgQueueSaveOnShutdown on >>>>> $MainMsgQueueFileName mq >>>>> $MainMsgQueueMaxFileSize 5m >>>>> >>>>> $ActionQueueType LinkedList >>>>> $ActionQueueSaveOnShutdown on >>>>> $ActionQueueFileName dbq >>>>> $ActionQueueMaxFileSize 10m >>>>> $ActionResumeRetryCount -1 >>>>> *.* ? ? ? @@(o)10.2.3.4 >>>>> >>>>> >>>>> # Log all kernel messages to the console. >>>>> # Logging much else clutters up the screen. >>>>> #kern.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /dev/console >>>>> >>>>> # Log anything (except mail) of level info or higher. >>>>> # Don't log private authentication messages! >>>>> *.info;mail.none;authpriv.none;cron.none ? ? ? ? ? ? ? ?/var/log/messages >>>>> >>>>> # The authpriv file has restricted access. >>>>> authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure >>>>> >>>>> # Log all the mail messages in one place. >>>>> mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog >>>>> >>>>> >>>>> # Log cron stuff >>>>> cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron >>>>> >>>>> # Everybody gets emergency messages >>>>> *.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? * >>>>> >>>>> # Save news errors of level crit and higher in a special file. >>>>> uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler >>>>> >>>>> # Save boot messages also to boot.log >>>>> local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot.log >>>>> >>>>> >>>>> Thanks, >>>>> Kaiwang >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> >>>> _______________________________________________ >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ From penght at cn.fujitsu.com Fri Jan 6 07:48:18 2012 From: penght at cn.fujitsu.com (Peng Haitao) Date: Fri, 06 Jan 2012 14:48:18 +0800 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID Message-ID: <4F069932.9070408@cn.fujitsu.com> [add list to the CC list] When HAVE_SETSID is not defined, rsyslogd will use ioctl() make itself to daemon, but this can not make rsyslogd process become the process group leader of a new process group. In RHEL6.1, the status is as follows: # uname -a Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux # /etc/init.d/rsyslog restart Shutting down system logger: [ OK ] Starting system logger: [ OK ] # ps axo pgrp,ppid,pid,comm | grep rsyslog 6290 1 6301 rsyslogd When we send SIGTERM signal to 6290, rsyslogd will die:( So I think we should call setpgid() before ioctl(). Signed-off-by: Peng Haitao --- tools/syslogd.c | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diff --git a/tools/syslogd.c b/tools/syslogd.c index d1224f2..ac954b7 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -356,8 +356,15 @@ void untty(void) #else { int i; + pid_t pid; if(!Debug) { + pid = getpid(); + if (setpgid(pid, pid) < 0) { + perror("setpgid"); + exit(1); + } + i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); if (i >= 0) { # if !defined(__hpux) -- 1.7.1 -- Best Regards, Peng Haitao From kaiwang.chen at gmail.com Mon Jan 9 09:29:05 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Mon, 9 Jan 2012 16:29:05 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: Message-ID: To report that coredump and message corruption disappeared after the application of my patch; I guess it's fixed. Thanks, Kaiwang 2012/1/5 Kaiwang Chen : > Hello, > > I check ChangeLog and found 5.8.4 fixed some bug related to race > condition as discussed in > http://bugzilla.adiscon.com/show_bug.cgi?id=275 > > with patch protecting modification in the getMSGID(): > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb356cc230685c5b7c90f8590b4 > > > I guess the situation is almost identical to what in my case: > corruption is always at the APPNAME field and there is, I believe, a > typo in line 1955 from getAPPNAME() effectively ruins the mutex > proctection: > 1947 /* rgerhards, 2005-11-24 > 1948 ?*/ > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) > 1950 { > 1951 ? ? ? ? uchar *pszRet; > 1952 > 1953 ? ? ? ? assert(pM != NULL); > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); > 1959 ? ? ? ? else > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); > 1963 ? ? ? ? return (char*)pszRet; > 1964 } > > Similar cases were spotted in getPROCID(), getStructuredData() and > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball > should solve the problem, I will report back later. > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 +0800 > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 > @@ -1609,7 +1609,7 @@ > > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > - ? ? ? ? ? ? ? MsgUnlock(pM); > + ? ? ? ? ? ? ? MsgLock(pM); > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); > ? ? ? ?if(pM->pCSPROCID == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > @@ -1846,7 +1846,7 @@ > ?{ > ? ? ? ?uchar *pszRet; > > - ? ? ? MsgUnlock(pM); > + ? ? ? MsgLock(pM); > ? ? ? ?if(pM->pCSStrucData == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); > ? ? ? ?else > @@ -1894,7 +1894,7 @@ > ? ? ? ?uchar *pszRet; > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > - ? ? ? ? ? ? ? MsgUnlock(pM); > + ? ? ? ? ? ? ? MsgLock(pM); > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); > ? ? ? ?if(pM->pCSProgName == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > @@ -1952,7 +1952,7 @@ > > ? ? ? ?assert(pM != NULL); > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > - ? ? ? ? ? ? ? MsgUnlock(pM); > + ? ? ? ? ? ? ? MsgLock(pM); > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > ? ? ? ?if(pM->pCSAPPNAME == NULL) > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > > > By the way, many crashes led to the convergence, for example > //// db1 core.26581 signal 11, Segmentation fault. > (gdb) p *pThis > $1 = { > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", > ?pszBuf = 0x7f7f5c000dd0 "sshd", > ?iBufSize = 4, > ?iStrLen = 4 > } > (gdb) where > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at stringbuf.c:334 > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 > '\001') at msg.c:1960 > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, pTpe=0x12ef350, > propID=115 's', pPropLen=0x43fb8f00, > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, > pBatch=0x12f7898, pbShutdownImmediate=) > ? ?at ../action.c:738 > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, > pbShutdownImmediate=) at ../action.c:1163 > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, > pWti=0x12f7870) at queue.c:1705 > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 > #10 0x0000000000000000 in ?? () > > > > Thanks, > Kaiwang > > 2011/12/31 Kaiwang Chen : >> Even worse case >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog 5.8.6, >> log source = /proc/kmsg started. >> ...several SYN flooding entries in the same second... >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size >> (fast): 0x00007fafb8000e00 *** >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: >> 0x00007fafb8000e20 *** >> >> ?And possibly hours later the host was not responsive to ssh >> connection requests, and even TTY login attempts failed... >> Interestingly the debug log in another file continues a little beyond: >> >> Debug line with all properties: >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, >> syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> >> Debug line with all properties: >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', PRI: 47, >> syslogtag 'rsyslogd-pstats:', programname: 'rsyslogd-pstats', >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> >> Fortunately the squid service on that host was not badly affected. So >> I did graceful failover and reboot the host, everything became OK. >> Although there was no direct timestamp associated, I guess the problem >> occured soon after rsyslogd started. I also spotted several corrupted >> messages on other hosts when rsyslogd started. They are same version: >> >> # rsyslogd -v >> rsyslogd 5.8.6, compiled with: >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> Thanks, >> Kaiwang >> >> 2011/12/31 Kaiwang Chen : >>> Also spotted corrupted messages from imuxsock: >>> >>> Debug line with all properties: >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', PRI: 47, >>> syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> ratelimit.numratelimiters=213 ' >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> ratelimit.numratelimiters=213 ' >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >>> >>> >>> Thanks, >>> Kaiwang >>> >>> 2011/12/27 Kaiwang Chen : >>>> 2011/12/25 ?: >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> I found rsyslogd occasionally produced corrupted log entries like >>>>>> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >>>>>> device eth0 entered promiscuous mode >>>>>> >>>>>> I believe that problem appeared in earlier versions including 5.8.2. >>>>>> Looks like corruption never occur before hostname field. Is it a >>>>>> reported bug? Any clue? >>>>> >>>>> >>>>> the question is if this problem is in the raw message being sent to you, or >>>>> is it something added by the rsyslog processing. >>>>> >>>>> I would suggest adding something along the following line. >>>>> >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >>>>> >>>>> this will look for any messages with the hex 01 character in them and spit >>>>> out all the info that you can use about the message in to the file >>>>> /var/log/badmessages >>>>> >>>>> run this for a little while and look to see what the raw message that is >>>>> being received over the wire looks like. If it's bad, then you need to look >>>>> at the sender. If the raw message looks sane, but rsyslog isn't handling it >>>>> right, then we can troubleshoot from there. >>>> >>>> Looks like it's related to tcp transmission. On the original server >>>> that get input from unix socket, it is OK; while on the centeral log >>>> server, the raw message is already corrupted. >>>> >>>> ===== on central log server >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>>> UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>>> packet(s) from UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >>>> Connection from UDP: [172.25.0.230]:53547 >>>> >>>> Debug line with all properties: >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[2#032#0251]', programname: 'snmpd', APP-NAME: >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 snmpd >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >>>> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>>> UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received SNMP >>>> packet(s) from UDP: [172.25.0.230]:53547 >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection from >>>> UDP: [172.25.0.230]:53547 >>>> >>>> Debug line with all properties: >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>>> Connection from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>>> >>>> Debug line with all properties: >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: 30, >>>> syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: 'snmpd', >>>> PROCID: '24254', MSGID: '-', >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>>> Connection from UDP: [172.25.0.230]:53547' >>>> >>>> >>>> Thanks, >>>> Kaiwang >>>>> >>>>> David Lang >>>>> >>>>> ?> rsyslogd 5.8.6, compiled with: >>>>>> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >>>>>> >>>>>> /etc/rsyslog.conf >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >>>>>> >>>>>> $ModLoad imklog >>>>>> $ModLoad imuxsock >>>>>> $ModLoad impstats >>>>>> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >>>>>> $SystemLogUsePIDFromSystem on >>>>>> >>>>>> $PStatInterval 600 >>>>>> $PStatSeverity 7 >>>>>> >>>>>> $WorkDirectory /var/spool/rsyslog >>>>>> >>>>>> $MainMsgQueueSaveOnShutdown on >>>>>> $MainMsgQueueFileName mq >>>>>> $MainMsgQueueMaxFileSize 5m >>>>>> >>>>>> $ActionQueueType LinkedList >>>>>> $ActionQueueSaveOnShutdown on >>>>>> $ActionQueueFileName dbq >>>>>> $ActionQueueMaxFileSize 10m >>>>>> $ActionResumeRetryCount -1 >>>>>> *.* ? ? ? @@(o)10.2.3.4 >>>>>> >>>>>> >>>>>> # Log all kernel messages to the console. >>>>>> # Logging much else clutters up the screen. >>>>>> #kern.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /dev/console >>>>>> >>>>>> # Log anything (except mail) of level info or higher. >>>>>> # Don't log private authentication messages! >>>>>> *.info;mail.none;authpriv.none;cron.none ? ? ? ? ? ? ? ?/var/log/messages >>>>>> >>>>>> # The authpriv file has restricted access. >>>>>> authpriv.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/secure >>>>>> >>>>>> # Log all the mail messages in one place. >>>>>> mail.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?-/var/log/maillog >>>>>> >>>>>> >>>>>> # Log cron stuff >>>>>> cron.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/cron >>>>>> >>>>>> # Everybody gets emergency messages >>>>>> *.emerg ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? * >>>>>> >>>>>> # Save news errors of level crit and higher in a special file. >>>>>> uucp,news.crit ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/spooler >>>>>> >>>>>> # Save boot messages also to boot.log >>>>>> local7.* ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/var/log/boot.log >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Kaiwang >>>>>> _______________________________________________ >>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> >>>>> _______________________________________________ >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ From francois at fmarier.org Mon Jan 9 10:35:19 2012 From: francois at fmarier.org (Francois Marier) Date: Mon, 9 Jan 2012 22:35:19 +1300 Subject: [rsyslog] Use of the "discard" action Message-ID: <20120109093519.GE6656@isafjordur.dyndns.org> Hi, I must be missing something because I'm trying to have messages received (via UDP) from 192.168.1.1: 1- go to a separate file, and 2- not show up in /var/log/syslog by loosely following the instructions in the cookbook [1] and the config file documentation [2]: $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log & ~ Unfortunately, only #1 works. I've got the log entries from 192.168.1.1 both in gargoyle-router.log and in /var/log/syslog just like when I use this config. I've also tried to get #2 working by itself (not logging to the gargoyle-router.log file and not logging in /var/log/syslog) using this config: $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then ~ but it also didn't work. I suspect that there is something I don't understand about the discard command because I can't figure out why it's not discarding the message. Any ideas on how I should go about debugging this further? (I'm running rsyslog 4.2.0-2ubuntu8.1 on Ubuntu Lucid and I have attached my full concatenated rsyslog.conf to this email.) Cheers, Francois [1] http://rsyslog.com/writing-specific-messages-to-a-file-and-discarding-them/ [2] http://rsyslog.com/doc/rsyslog_conf_actions.html -- Francois Marier identi.ca/fmarier http://fmarier.org twitter.com/fmarier -------------- next part -------------- # /etc/rsyslog.conf Configuration file for rsyslog. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html # # Default logging rules can be found in /etc/rsyslog.d/50-default.conf ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability $KLogPath /proc/kmsg # provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log & ~ # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use traditional timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # Filter duplicated messages $RepeatedMsgReduction on # # Set the default permissions for all log files. # $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 $PrivDropToUser syslog $PrivDropToGroup syslog # Default rules for rsyslog. # # For more information see rsyslog.conf(5) and /etc/rsyslog.conf # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # daemon.*;mail.*;\ news.err;\ *.=debug;*.=info;\ *.=notice;*.=warn |/dev/xconsole # Create an additional socket in postfix's chroot in order not to break # mail logging when rsyslog is restarted. If the directory is missing, # rsyslog will silently skip creating the socket. $AddUnixListenSocket /var/spool/postfix/dev/log From bodik at civ.zcu.cz Mon Jan 9 10:37:58 2012 From: bodik at civ.zcu.cz (bodik) Date: Mon, 09 Jan 2012 10:37:58 +0100 Subject: [rsyslog] Use of the "discard" action In-Reply-To: <20120109093519.GE6656@isafjordur.dyndns.org> References: <20120109093519.GE6656@isafjordur.dyndns.org> Message-ID: <4F0AB576.5030605@civ.zcu.cz> hi, lately i ends up with this configuration, remote configuration must come first ... http://home.zcu.cz/~bodik/rsyslog/rsyslog.srvr.conf b Dne 01/09/12 10:35, Francois Marier napsal(a): > Hi, > > I must be missing something because I'm trying to have messages received > (via UDP) from 192.168.1.1: > From rgerhards at hq.adiscon.com Mon Jan 9 14:34:57 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 9 Jan 2012 14:34:57 +0100 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Thanks for reporting and tracking down this bug. I was on vacation and have just returned. I applied your patch, it is now merged: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e338a6e5 fa14c0394bde6f1d I'd appreciate if you could double-check that the git version is correct (you never know...). I will release this version during the next days. Once again for your help! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen > Sent: Monday, January 09, 2012 9:29 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption > > To report that coredump and message corruption disappeared after the > application of my patch; I guess it's fixed. > > Thanks, > Kaiwang > > 2012/1/5 Kaiwang Chen : > > Hello, > > > > I check ChangeLog and found 5.8.4 fixed some bug related to race > > condition as discussed in > > http://bugzilla.adiscon.com/show_bug.cgi?id=275 > > > > with patch protecting modification in the getMSGID(): > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 > > 6cc230685c5b7c90f8590b4 > > > > > > I guess the situation is almost identical to what in my case: > > corruption is always at the APPNAME field and there is, I believe, a > > typo in line 1955 from getAPPNAME() effectively ruins the mutex > > proctection: > > 1947 /* rgerhards, 2005-11-24 > > 1948 ?*/ > > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) > > 1950 { > > 1951 ? ? ? ? uchar *pszRet; > > 1952 > > 1953 ? ? ? ? assert(pM != NULL); > > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ > > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) > > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); > > 1959 ? ? ? ? else > > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); > > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); > > 1963 ? ? ? ? return (char*)pszRet; > > 1964 } > > > > Similar cases were spotted in getPROCID(), getStructuredData() and > > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball > > should solve the problem, I will report back later. > > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 > > +0800 > > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 > > @@ -1609,7 +1609,7 @@ > > > > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > > - ? ? ? ? ? ? ? MsgUnlock(pM); > > + ? ? ? ? ? ? ? MsgLock(pM); > > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); > > ? ? ? ?if(pM->pCSPROCID == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ > > ?{ > > ? ? ? ?uchar *pszRet; > > > > - ? ? ? MsgUnlock(pM); > > + ? ? ? MsgLock(pM); > > ? ? ? ?if(pM->pCSStrucData == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); > > ? ? ? ?else > > @@ -1894,7 +1894,7 @@ > > ? ? ? ?uchar *pszRet; > > > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > > - ? ? ? ? ? ? ? MsgUnlock(pM); > > + ? ? ? ? ? ? ? MsgLock(pM); > > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); > > ? ? ? ?if(pM->pCSProgName == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ > > > > ? ? ? ?assert(pM != NULL); > > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > > - ? ? ? ? ? ? ? MsgUnlock(pM); > > + ? ? ? ? ? ? ? MsgLock(pM); > > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > > ? ? ? ?if(pM->pCSAPPNAME == NULL) > > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > > > > > > By the way, many crashes led to the convergence, for example //// db1 > > core.26581 signal 11, Segmentation fault. > > (gdb) p *pThis > > $1 = { > > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", > > ?pszBuf = 0x7f7f5c000dd0 "sshd", > > ?iBufSize = 4, > > ?iStrLen = 4 > > } > > (gdb) where > > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at > > stringbuf.c:334 > > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 > > '\001') at msg.c:1960 > > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, > pTpe=0x12ef350, > > propID=115 's', pPropLen=0x43fb8f00, > > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 > > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, > > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 > > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, > > pBatch=0x12f7898, pbShutdownImmediate=) > > ? ?at ../action.c:738 > > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, > > pbShutdownImmediate=) at ../action.c:1163 > > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, > > pWti=0x12f7870) at queue.c:1705 > > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 > > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 > > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 > > #10 0x0000000000000000 in ?? () > > > > > > > > Thanks, > > Kaiwang > > > > 2011/12/31 Kaiwang Chen : > >> Even worse case > >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog > >> 5.8.6, log source = /proc/kmsg started. > >> ...several SYN flooding entries in the same second... > >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size > >> (fast): 0x00007fafb8000e00 *** > >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: > >> 0x00007fafb8000e20 *** > >> > >> ?And possibly hours later the host was not responsive to ssh > >> connection requests, and even TTY login attempts failed... > >> Interestingly the debug log in another file continues a little beyond: > >> > >> Debug line with all properties: > >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', > >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >> 'rsyslogd-pstats', > >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 > maxqsize=0 ' > >> > >> Debug line with all properties: > >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', > >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >> 'rsyslogd-pstats', > >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 > maxqsize=5 ' > >> > >> Fortunately the squid service on that host was not badly affected. So > >> I did graceful failover and reboot the host, everything became OK. > >> Although there was no direct timestamp associated, I guess the > >> problem occured soon after rsyslogd started. I also spotted several > >> corrupted messages on other hosts when rsyslogd started. They are same > version: > >> > >> # rsyslogd -v > >> rsyslogd 5.8.6, compiled with: > >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No > >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes > >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes > >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No > >> > >> > >> Thanks, > >> Kaiwang > >> > >> 2011/12/31 Kaiwang Chen : > >>> Also spotted corrupted messages from imuxsock: > >>> > >>> Debug line with all properties: > >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', > >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 > >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', > >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', > >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> ratelimit.numratelimiters=213 ' > >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> ratelimit.numratelimiters=213 ' > >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 > >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' > >>> > >>> > >>> Thanks, > >>> Kaiwang > >>> > >>> 2011/12/27 Kaiwang Chen : > >>>> 2011/12/25 ?: > >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> I found rsyslogd occasionally produced corrupted log entries like > >>>>>> > >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - > >>>>>> device eth0 entered promiscuous mode > >>>>>> > >>>>>> I believe that problem appeared in earlier versions including 5.8.2. > >>>>>> Looks like corruption never occur before hostname field. Is it a > >>>>>> reported bug? Any clue? > >>>>> > >>>>> > >>>>> the question is if this problem is in the raw message being sent > >>>>> to you, or is it something added by the rsyslog processing. > >>>>> > >>>>> I would suggest adding something along the following line. > >>>>> > >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG > >>>>> > >>>>> this will look for any messages with the hex 01 character in them > >>>>> and spit out all the info that you can use about the message in to > >>>>> the file /var/log/badmessages > >>>>> > >>>>> run this for a little while and look to see what the raw message > >>>>> that is being received over the wire looks like. If it's bad, then > >>>>> you need to look at the sender. If the raw message looks sane, but > >>>>> rsyslog isn't handling it right, then we can troubleshoot from there. > >>>> > >>>> Looks like it's related to tcp transmission. On the original server > >>>> that get input from unix socket, it is OK; while on the centeral > >>>> log server, the raw message is already corrupted. > >>>> > >>>> ===== on central log server > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection > >>>> from > >>>> UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > SNMP > >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - > >>>> Connection from UDP: [172.25.0.230]:53547 > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', > >>>> APP-NAME: 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 > >>>> snmpd > >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', > >>>> APP-NAME: 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Received SNMP packet(s) from UDP: > [172.25.0.230]:53547' > >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 > >>>> snmpd > >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', programname: > 'snmpd', APP-NAME: > >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 > >>>> snmpd > >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection > >>>> from > >>>> UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > SNMP > >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection > >>>> from > >>>> UDP: [172.25.0.230]:53547 > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: > >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>>> 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>>> Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: > >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>>> 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Received SNMP packet(s) from UDP: > [172.25.0.230]:53547' > >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>>> > >>>> Debug line with all properties: > >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: > >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>>> 'snmpd', > >>>> PROCID: '24254', MSGID: '-', > >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>>> Connection from UDP: [172.25.0.230]:53547' > >>>> > >>>> > >>>> Thanks, > >>>> Kaiwang > >>>>> > >>>>> David Lang > >>>>> > >>>>> ?> rsyslogd 5.8.6, compiled with: > >>>>>> > >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No > >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes > >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes > >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No > >>>>>> > >>>>>> /etc/rsyslog.conf > >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>>>>> > >>>>>> $ModLoad imklog > >>>>>> $ModLoad imuxsock > >>>>>> $ModLoad impstats > >>>>>> > >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off > >>>>>> $SystemLogUsePIDFromSystem on > >>>>>> > >>>>>> $PStatInterval 600 > >>>>>> $PStatSeverity 7 > >>>>>> > >>>>>> $WorkDirectory /var/spool/rsyslog > >>>>>> > >>>>>> $MainMsgQueueSaveOnShutdown on > >>>>>> $MainMsgQueueFileName mq > >>>>>> $MainMsgQueueMaxFileSize 5m > >>>>>> > >>>>>> $ActionQueueType LinkedList > >>>>>> $ActionQueueSaveOnShutdown on > >>>>>> $ActionQueueFileName dbq > >>>>>> $ActionQueueMaxFileSize 10m > >>>>>> $ActionResumeRetryCount -1 > >>>>>> *.* ? ? ? @@(o)10.2.3.4 > >>>>>> > >>>>>> > >>>>>> # Log all kernel messages to the console. > >>>>>> # Logging much else clutters up the screen. > >>>>>> #kern.* > >>>>>> /dev/console > >>>>>> > >>>>>> # Log anything (except mail) of level info or higher. > >>>>>> # Don't log private authentication messages! > >>>>>> *.info;mail.none;authpriv.none;cron.none > >>>>>> /var/log/messages > >>>>>> > >>>>>> # The authpriv file has restricted access. > >>>>>> authpriv.* > >>>>>> /var/log/secure > >>>>>> > >>>>>> # Log all the mail messages in one place. > >>>>>> mail.* > >>>>>> -/var/log/maillog > >>>>>> > >>>>>> > >>>>>> # Log cron stuff > >>>>>> cron.* > >>>>>> /var/log/cron > >>>>>> > >>>>>> # Everybody gets emergency messages > *.emerg > >>>>>> * > >>>>>> > >>>>>> # Save news errors of level crit and higher in a special file. > >>>>>> uucp,news.crit > >>>>>> /var/log/spooler > >>>>>> > >>>>>> # Save boot messages also to boot.log > >>>>>> local7.* > >>>>>> /var/log/boot.log > >>>>>> > >>>>>> > >>>>>> Thanks, > >>>>>> Kaiwang > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From friedl at hq.adiscon.com Mon Jan 9 16:39:04 2012 From: friedl at hq.adiscon.com (Florian Riedl) Date: Mon, 9 Jan 2012 16:39:04 +0100 Subject: [rsyslog] rsyslog 6.2.0 (stable) released Message-ID: <9B6E2A8877C38245BFB15CC491A11DA72816DC@GRFEXC.intern.adiscon.com> This is the initial stable release of rsyslog v6. It is basically the last beta version plus some more fixes. This version provides functional and performance enhancements, for example the Hadoop (HDFS) support has been considerably speeded up by supporting batched insert mode. Also, TCP transmission overhead for TLS has been dramatically improved. TCP now also supports input worker thread pools. Most importantly, rsyslog now supports log normalization via liblognorm rule bases. This permits very high performance normalization of semantically equal messages from different devices (and thus in different syntaxes). Note that config scoping, available in the beta versions, is NOT supported by v6-stable. This was decided because it would have been functionality equivalent to the new config language upcoming in v6.3 (already available as part of the devel version). As scoping was not available in any earlier versions, introducing it in v6.2 would have added, in the long term, just another method of doing some identicaly thing via the ugly old config language. This would have lead to user confusion and more complex than necessary code. If you are interested in the cleaner config language, we strongly encourage you to have a look at rsyslog 6.3. With the arrival of the stable v6 version, version 4 will be retired and is no longer officially supported (but support is provided under maintenance contracts, of course). ChangeLog: http://www.rsyslog.com/changelog-for-6-2-0-v6-stable/ Download: http://www.rsyslog.com/rsyslog-6-2-0-v6-stable/ As always, feedback is appreciated. Best regards, Florian Riedl From david at lang.hm Tue Jan 10 03:03:29 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 9 Jan 2012 18:03:29 -0800 (PST) Subject: [rsyslog] Use of the "discard" action In-Reply-To: <20120109093519.GE6656@isafjordur.dyndns.org> References: <20120109093519.GE6656@isafjordur.dyndns.org> Message-ID: On Mon, 9 Jan 2012, Francois Marier wrote: > Hi, > > I must be missing something because I'm trying to have messages received > (via UDP) from 192.168.1.1: > > 1- go to a separate file, and > 2- not show up in /var/log/syslog > > by loosely following the instructions in the cookbook [1] and the config > file documentation [2]: > > $ModLoad imudp > $UDPServerRun 514 > if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log > & ~ > > Unfortunately, only #1 works. I've got the log entries from 192.168.1.1 both > in gargoyle-router.log and in /var/log/syslog just like when I use this > config. unfortunantly, this config snippet is not sufficient for us to understand what's happening because it doesn't show anything writing to /var/log/syslog. the output to /var/log/syslog would need to be after the discard step (otherwise you are writing to /var/log/syslog before you even run the if statement) also note that doing a full "if then" is slower than doing the property-based conditional :fromhost-ip, startswith, "192.168.1.1" /var/log/gargoyle-router.log and finally, this will match 192.168.1.100 as well as 192.168.1.1 so you probably want equals instead of startswith. David Lang > I've also tried to get #2 working by itself (not logging to the > gargoyle-router.log file and not logging in /var/log/syslog) using this > config: > > $ModLoad imudp > $UDPServerRun 514 > if $fromhost-ip startswith '192.168.1.1' then ~ > > but it also didn't work. > > I suspect that there is something I don't understand about the discard > command because I can't figure out why it's not discarding the message. Any > ideas on how I should go about debugging this further? > > (I'm running rsyslog 4.2.0-2ubuntu8.1 on Ubuntu Lucid and I have attached my > full concatenated rsyslog.conf to this email.) > > Cheers, > Francois > > [1] http://rsyslog.com/writing-specific-messages-to-a-file-and-discarding-them/ > [2] http://rsyslog.com/doc/rsyslog_conf_actions.html > > -------------- next part -------------- # /etc/rsyslog.conf Configuration file for rsyslog. # # For more information see # /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html # # Default logging rules can be found in /etc/rsyslog.d/50-default.conf ################# #### MODULES #### ################# $ModLoad imuxsock # provides support for local system logging $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability $KLogPath /proc/kmsg # provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '192.168.1.1' then -/var/log/gargoyle-router.log & ~ # provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 ########################### #### GLOBAL DIRECTIVES #### ########################### # # Use traditional timestamp format. # To enable high precision timestamps, comment out the following line. # $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # Filter duplicated messages $RepeatedMsgReduction on # # Set the default permissions for all log files. # $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 $PrivDropToUser syslog $PrivDropToGroup syslog # Default rules for rsyslog. # # For more information see rsyslog.conf(5) and /etc/rsyslog.conf # # First some standard log files. Log by facility. # auth,authpriv.* /var/log/auth.log *.*;auth,authpriv.none -/var/log/syslog #cron.* /var/log/cron.log daemon.* -/var/log/daemon.log kern.* -/var/log/kern.log lpr.* -/var/log/lpr.log mail.* -/var/log/mail.log user.* -/var/log/user.log # # Logging for the mail system. Split it up so that # it is easy to write scripts to parse these files. # mail.info -/var/log/mail.info mail.warn -/var/log/mail.warn mail.err /var/log/mail.err # # Logging for INN news system. # news.crit /var/log/news/news.crit news.err /var/log/news/news.err news.notice -/var/log/news/news.notice # # Some "catch-all" log files. # *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages # # Emergencies are sent to everybody logged in. # *.emerg * # # I like to have messages displayed on the console, but only on a virtual # console I usually leave idle. # #daemon,mail.*;\ # news.=crit;news.=err;news.=notice;\ # *.=debug;*.=info;\ # *.=notice;*.=warn /dev/tty8 # The named pipe /dev/xconsole is for the `xconsole' utility. To use it, # you must invoke `xconsole' with the `-file' option: # # $ xconsole -file /dev/xconsole [...] # # NOTE: adjust the list below, or you'll go crazy if you have a reasonably # busy site.. # daemon.*;mail.*;\ news.err;\ *.=debug;*.=info;\ *.=notice;*.=warn |/dev/xconsole # Create an additional socket in postfix's chroot in order not to break # mail logging when rsyslog is restarted. If the directory is missing, # rsyslog will silently skip creating the socket. $AddUnixListenSocket /var/spool/postfix/dev/log -------------- next part -------------- _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ From kaiwang.chen at gmail.com Tue Jan 10 10:29:14 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Tue, 10 Jan 2012 17:29:14 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Message-ID: Applied to four hosts where coredump had been spotted; rpm was built with the following instructions: git clone git://git.adiscon.com/git/rsyslog.git cd rsyslog git checkout v5-stable ./autogen.sh cd .. mv rsyslog rsyslog-5.8.7 tar czf rsyslog-5.8.7-beta.tar.gz rsyslog-5.8.7 --exclude '.git*' rpmbuild -bs rsyslog.spec mock /usr/src/redhat/SRPMS/rsyslog-5.8.7-beta.el5.kc.src.rpm I will report back in two days. Thanks, Kaiwang 2012/1/9 Rainer Gerhards : > Thanks for reporting and tracking down this bug. I was on vacation and have > just returned. I applied your patch, it is now merged: > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e338a6e5 > fa14c0394bde6f1d > > I'd appreciate if you could double-check that the git version is correct (you > never know...). I will release this version during the next days. > > Once again for your help! > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen >> Sent: Monday, January 09, 2012 9:29 AM >> To: rsyslog-users >> Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption >> >> To report that coredump and message corruption disappeared after the >> application of my patch; I guess it's fixed. >> >> Thanks, >> Kaiwang >> >> 2012/1/5 Kaiwang Chen : >> > Hello, >> > >> > I check ChangeLog and found 5.8.4 fixed some bug related to race >> > condition as discussed in >> > http://bugzilla.adiscon.com/show_bug.cgi?id=275 >> > >> > with patch protecting modification in the getMSGID(): >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 >> > 6cc230685c5b7c90f8590b4 >> > >> > >> > I guess the situation is almost identical to what in my case: >> > corruption is always at the APPNAME field and there is, I believe, a >> > typo in line 1955 from getAPPNAME() effectively ruins the mutex >> > proctection: >> > 1947 /* rgerhards, 2005-11-24 >> > 1948 ?*/ >> > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) >> > 1950 { >> > 1951 ? ? ? ? uchar *pszRet; >> > 1952 >> > 1953 ? ? ? ? assert(pM != NULL); >> > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >> > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ >> > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >> > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) >> > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); >> > 1959 ? ? ? ? else >> > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); >> > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >> > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); >> > 1963 ? ? ? ? return (char*)pszRet; >> > 1964 } >> > >> > Similar cases were spotted in getPROCID(), getStructuredData() and >> > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball >> > should solve the problem, I will report back later. >> > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 >> > +0800 >> > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 >> > @@ -1609,7 +1609,7 @@ >> > >> > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); >> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >> > - ? ? ? ? ? ? ? MsgUnlock(pM); >> > + ? ? ? ? ? ? ? MsgLock(pM); >> > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); >> > ? ? ? ?if(pM->pCSPROCID == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ >> > ?{ >> > ? ? ? ?uchar *pszRet; >> > >> > - ? ? ? MsgUnlock(pM); >> > + ? ? ? MsgLock(pM); >> > ? ? ? ?if(pM->pCSStrucData == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); >> > ? ? ? ?else >> > @@ -1894,7 +1894,7 @@ >> > ? ? ? ?uchar *pszRet; >> > >> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >> > - ? ? ? ? ? ? ? MsgUnlock(pM); >> > + ? ? ? ? ? ? ? MsgLock(pM); >> > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); >> > ? ? ? ?if(pM->pCSProgName == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ >> > >> > ? ? ? ?assert(pM != NULL); >> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >> > - ? ? ? ? ? ? ? MsgUnlock(pM); >> > + ? ? ? ? ? ? ? MsgLock(pM); >> > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >> > ? ? ? ?if(pM->pCSAPPNAME == NULL) >> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); >> > >> > >> > By the way, many crashes led to the convergence, for example //// db1 >> > core.26581 signal 11, Segmentation fault. >> > (gdb) p *pThis >> > $1 = { >> > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", >> > ?pszBuf = 0x7f7f5c000dd0 "sshd", >> > ?iBufSize = 4, >> > ?iStrLen = 4 >> > } >> > (gdb) where >> > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at >> > stringbuf.c:334 >> > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 >> > '\001') at msg.c:1960 >> > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, >> pTpe=0x12ef350, >> > propID=115 's', pPropLen=0x43fb8f00, >> > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 >> > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, >> > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 >> > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, >> > pBatch=0x12f7898, pbShutdownImmediate=) >> > ? ?at ../action.c:738 >> > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, >> > pbShutdownImmediate=) at ../action.c:1163 >> > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, >> > pWti=0x12f7870) at queue.c:1705 >> > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 >> > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 >> > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 >> > #10 0x0000000000000000 in ?? () >> > >> > >> > >> > Thanks, >> > Kaiwang >> > >> > 2011/12/31 Kaiwang Chen : >> >> Even worse case >> >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog >> >> 5.8.6, log source = /proc/kmsg started. >> >> ...several SYN flooding entries in the same second... >> >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size >> >> (fast): 0x00007fafb8000e00 *** >> >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: >> >> 0x00007fafb8000e20 *** >> >> >> >> ?And possibly hours later the host was not responsive to ssh >> >> connection requests, and even TTY login attempts failed... >> >> Interestingly the debug log in another file continues a little beyond: >> >> >> >> Debug line with all properties: >> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >> >> 'rsyslogd-pstats', >> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >> >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 >> maxqsize=0 ' >> >> >> >> Debug line with all properties: >> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >> >> 'rsyslogd-pstats', >> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >> >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >> >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 >> maxqsize=5 ' >> >> >> >> Fortunately the squid service on that host was not badly affected. So >> >> I did graceful failover and reboot the host, everything became OK. >> >> Although there was no direct timestamp associated, I guess the >> >> problem occured soon after rsyslogd started. I also spotted several >> >> corrupted messages on other hosts when rsyslogd started. They are same >> version: >> >> >> >> # rsyslogd -v >> >> rsyslogd 5.8.6, compiled with: >> >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >> >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No >> >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes >> >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes >> >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No >> >> >> >> >> >> Thanks, >> >> Kaiwang >> >> >> >> 2011/12/31 Kaiwang Chen : >> >>> Also spotted corrupted messages from imuxsock: >> >>> >> >>> Debug line with all properties: >> >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', >> >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >> >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >> >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >> >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> >>> ratelimit.numratelimiters=213 ' >> >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >> >>> ratelimit.numratelimiters=213 ' >> >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 >> >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >> >>> >> >>> >> >>> Thanks, >> >>> Kaiwang >> >>> >> >>> 2011/12/27 Kaiwang Chen : >> >>>> 2011/12/25 ?: >> >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >> >>>>> >> >>>>>> Hi all, >> >>>>>> >> >>>>>> I found rsyslogd occasionally produced corrupted log entries like >> >>>>>> >> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >> >>>>>> device eth0 entered promiscuous mode >> >>>>>> >> >>>>>> I believe that problem appeared in earlier versions including 5.8.2. >> >>>>>> Looks like corruption never occur before hostname field. Is it a >> >>>>>> reported bug? Any clue? >> >>>>> >> >>>>> >> >>>>> the question is if this problem is in the raw message being sent >> >>>>> to you, or is it something added by the rsyslog processing. >> >>>>> >> >>>>> I would suggest adding something along the following line. >> >>>>> >> >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >> >>>>> >> >>>>> this will look for any messages with the hex 01 character in them >> >>>>> and spit out all the info that you can use about the message in to >> >>>>> the file /var/log/badmessages >> >>>>> >> >>>>> run this for a little while and look to see what the raw message >> >>>>> that is being received over the wire looks like. If it's bad, then >> >>>>> you need to look at the sender. If the raw message looks sane, but >> >>>>> rsyslog isn't handling it right, then we can troubleshoot from there. >> >>>> >> >>>> Looks like it's related to tcp transmission. On the original server >> >>>> that get input from unix socket, it is OK; while on the centeral >> >>>> log server, the raw message is already corrupted. >> >>>> >> >>>> ===== on central log server >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >> >>>> from >> >>>> UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >> SNMP >> >>>> packet(s) from UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >> >>>> Connection from UDP: [172.25.0.230]:53547 >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >> >>>> APP-NAME: 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >> >>>> snmpd >> >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >> >>>> APP-NAME: 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >> [172.25.0.230]:53547' >> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >> >>>> snmpd >> >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', programname: >> 'snmpd', APP-NAME: >> >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >> >>>> snmpd >> >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >> >>>> from >> >>>> UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >> SNMP >> >>>> packet(s) from UDP: [172.25.0.230]:53547 >> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >> >>>> from >> >>>> UDP: [172.25.0.230]:53547 >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >> >>>> 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >> >>>> Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >> >>>> 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >> [172.25.0.230]:53547' >> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >> >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> Debug line with all properties: >> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >> >>>> 'snmpd', >> >>>> PROCID: '24254', MSGID: '-', >> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >> >>>> Connection from UDP: [172.25.0.230]:53547' >> >>>> >> >>>> >> >>>> Thanks, >> >>>> Kaiwang >> >>>>> >> >>>>> David Lang >> >>>>> >> >>>>> ?> rsyslogd 5.8.6, compiled with: >> >>>>>> >> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >> >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >> >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >> >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >> >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >> >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >> >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >> >>>>>> >> >>>>>> /etc/rsyslog.conf >> >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >> >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >> >>>>>> >> >>>>>> $ModLoad imklog >> >>>>>> $ModLoad imuxsock >> >>>>>> $ModLoad impstats >> >>>>>> >> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >> >>>>>> $SystemLogUsePIDFromSystem on >> >>>>>> >> >>>>>> $PStatInterval 600 >> >>>>>> $PStatSeverity 7 >> >>>>>> >> >>>>>> $WorkDirectory /var/spool/rsyslog >> >>>>>> >> >>>>>> $MainMsgQueueSaveOnShutdown on >> >>>>>> $MainMsgQueueFileName mq >> >>>>>> $MainMsgQueueMaxFileSize 5m >> >>>>>> >> >>>>>> $ActionQueueType LinkedList >> >>>>>> $ActionQueueSaveOnShutdown on >> >>>>>> $ActionQueueFileName dbq >> >>>>>> $ActionQueueMaxFileSize 10m >> >>>>>> $ActionResumeRetryCount -1 >> >>>>>> *.* ? ? ? @@(o)10.2.3.4 >> >>>>>> >> >>>>>> >> >>>>>> # Log all kernel messages to the console. >> >>>>>> # Logging much else clutters up the screen. >> >>>>>> #kern.* >> >>>>>> /dev/console >> >>>>>> >> >>>>>> # Log anything (except mail) of level info or higher. >> >>>>>> # Don't log private authentication messages! >> >>>>>> *.info;mail.none;authpriv.none;cron.none >> >>>>>> /var/log/messages >> >>>>>> >> >>>>>> # The authpriv file has restricted access. >> >>>>>> authpriv.* >> >>>>>> /var/log/secure >> >>>>>> >> >>>>>> # Log all the mail messages in one place. >> >>>>>> mail.* >> >>>>>> -/var/log/maillog >> >>>>>> >> >>>>>> >> >>>>>> # Log cron stuff >> >>>>>> cron.* >> >>>>>> /var/log/cron >> >>>>>> >> >>>>>> # Everybody gets emergency messages >> *.emerg >> >>>>>> * >> >>>>>> >> >>>>>> # Save news errors of level crit and higher in a special file. >> >>>>>> uucp,news.crit >> >>>>>> /var/log/spooler >> >>>>>> >> >>>>>> # Save boot messages also to boot.log >> >>>>>> local7.* >> >>>>>> /var/log/boot.log >> >>>>>> >> >>>>>> >> >>>>>> Thanks, >> >>>>>> Kaiwang >> >>>>>> _______________________________________________ >> >>>>>> rsyslog mailing list >> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>>>> http://www.rsyslog.com/professional-services/ >> >>>>>> >> >>>>> _______________________________________________ >> >>>>> rsyslog mailing list >> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >> >>>>> http://www.rsyslog.com/professional-services/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From francois at fmarier.org Wed Jan 11 09:05:10 2012 From: francois at fmarier.org (Francois Marier) Date: Wed, 11 Jan 2012 21:05:10 +1300 Subject: [rsyslog] Use of the "discard" action In-Reply-To: References: <20120109093519.GE6656@isafjordur.dyndns.org> Message-ID: <20120111080510.GE20032@isafjordur.dyndns.org> On 2012-01-09 at 18:03:29, david at lang.hm wrote: > :fromhost-ip, startswith, "192.168.1.1" /var/log/gargoyle-router.log > > and finally, this will match 192.168.1.100 as well as 192.168.1.1 so > you probably want equals instead of startswith. Thanks to your suggestion, I now have a config that works: $ModLoad imudp $UDPServerRun 514 :fromhost-ip, isequal, "192.168.1.1" -/var/log/gargoyle-router.log & ~ (just before the line that outputs to /var/log/syslog) Cheers, Francois -- Francois Marier identi.ca/fmarier http://fmarier.org twitter.com/fmarier From david at lang.hm Wed Jan 11 12:12:36 2012 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Jan 2012 03:12:36 -0800 (PST) Subject: [rsyslog] Use of the "discard" action In-Reply-To: <20120111080510.GE20032@isafjordur.dyndns.org> References: <20120109093519.GE6656@isafjordur.dyndns.org> <20120111080510.GE20032@isafjordur.dyndns.org> Message-ID: note that in rsyslog, it defaults to async writes so the - is not needed ahead of the filename (and there are various config paramters to set if you want more safety at the expense of speed) David Lang On Wed, 11 Jan 2012, Francois Marier wrote: > On 2012-01-09 at 18:03:29, david at lang.hm wrote: >> :fromhost-ip, startswith, "192.168.1.1" /var/log/gargoyle-router.log >> >> and finally, this will match 192.168.1.100 as well as 192.168.1.1 so >> you probably want equals instead of startswith. > > Thanks to your suggestion, I now have a config that works: > > $ModLoad imudp > $UDPServerRun 514 > :fromhost-ip, isequal, "192.168.1.1" -/var/log/gargoyle-router.log > & ~ > > (just before the line that outputs to /var/log/syslog) > > Cheers, > Francois From kaushalshriyan at gmail.com Thu Jan 12 00:45:13 2012 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Thu, 12 Jan 2012 05:15:13 +0530 Subject: [rsyslog] rsyslog Centralized Logging Server along with rsyslog clients Message-ID: Hi I have Centralized Rsyslog server running on Ubuntu Linux Server 10.04 and also rsyslog client running on it. http://pastebin.ubuntu.com/801187/ -> Centralised Rsyslog Server http://pastebin.ubuntu.com/801189/ -> Rsyslog Client and http://pastebin.ubuntu.com/801190/ -> cat /etc/rsyslog.d/50-default.conf Please help me understand if my above set up is correct and correct me if i am wrong. Regards Kaushal From david at lang.hm Thu Jan 12 01:38:43 2012 From: david at lang.hm (david at lang.hm) Date: Wed, 11 Jan 2012 16:38:43 -0800 (PST) Subject: [rsyslog] rsyslog Centralized Logging Server along with rsyslog clients In-Reply-To: References: Message-ID: On Thu, 12 Jan 2012, Kaushal Shriyan wrote: > Hi > > I have Centralized Rsyslog server running on Ubuntu Linux Server 10.04 and > also rsyslog client running on it. > > http://pastebin.ubuntu.com/801187/ -> Centralised Rsyslog Server > http://pastebin.ubuntu.com/801189/ -> Rsyslog Client and > http://pastebin.ubuntu.com/801190/ -> cat /etc/rsyslog.d/50-default.conf > > Please help me understand if my above set up is correct and correct me if i > am wrong. before anyone could tell you if it is correct, you would need to explain more about what you are trying to do. Also, 'correct' can vary a lot based on the details of your network. you are really better off trying something, and if it doesn't work the way you expect it to, then asking for help rather than asking people to create your configs (or validate your configs) cold without any indication of a problem. I doubt that you intend it to come across this way, but you are sounding very much like "please do my homework for me" (both here and on the SEC list). There is a lot of documentation for both products, including examples. You need to spend some time looking through them before asking for help on the lists. David Lang From kaiwang.chen at gmail.com Thu Jan 12 08:42:37 2012 From: kaiwang.chen at gmail.com (Kaiwang Chen) Date: Thu, 12 Jan 2012 15:42:37 +0800 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Message-ID: To report that such problem is not spotted on these four hosts since upgraded to 5.8.7-beta. Thanks, Kaiwang 2012/1/10 Kaiwang Chen : > Applied to four hosts where coredump had been spotted; rpm was built > with the following instructions: > > git clone git://git.adiscon.com/git/rsyslog.git > cd rsyslog > git checkout v5-stable > ./autogen.sh > cd .. > mv rsyslog rsyslog-5.8.7 > tar czf rsyslog-5.8.7-beta.tar.gz rsyslog-5.8.7 --exclude '.git*' > > rpmbuild -bs rsyslog.spec > mock /usr/src/redhat/SRPMS/rsyslog-5.8.7-beta.el5.kc.src.rpm > > > I will report back in two days. > > Thanks, > Kaiwang > > 2012/1/9 Rainer Gerhards : >> Thanks for reporting and tracking down this bug. I was on vacation and have >> just returned. I applied your patch, it is now merged: >> >> http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e338a6e5 >> fa14c0394bde6f1d >> >> I'd appreciate if you could double-check that the git version is correct (you >> never know...). I will release this version during the next days. >> >> Once again for your help! >> >> Rainer >> >>> -----Original Message----- >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen >>> Sent: Monday, January 09, 2012 9:29 AM >>> To: rsyslog-users >>> Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption >>> >>> To report that coredump and message corruption disappeared after the >>> application of my patch; I guess it's fixed. >>> >>> Thanks, >>> Kaiwang >>> >>> 2012/1/5 Kaiwang Chen : >>> > Hello, >>> > >>> > I check ChangeLog and found 5.8.4 fixed some bug related to race >>> > condition as discussed in >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=275 >>> > >>> > with patch protecting modification in the getMSGID(): >>> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 >>> > 6cc230685c5b7c90f8590b4 >>> > >>> > >>> > I guess the situation is almost identical to what in my case: >>> > corruption is always at the APPNAME field and there is, I believe, a >>> > typo in line 1955 from getAPPNAME() effectively ruins the mutex >>> > proctection: >>> > 1947 /* rgerhards, 2005-11-24 >>> > 1948 ?*/ >>> > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) >>> > 1950 { >>> > 1951 ? ? ? ? uchar *pszRet; >>> > 1952 >>> > 1953 ? ? ? ? assert(pM != NULL); >>> > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >>> > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ >>> > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >>> > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) >>> > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); >>> > 1959 ? ? ? ? else >>> > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM->pCSAPPNAME); >>> > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) >>> > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); >>> > 1963 ? ? ? ? return (char*)pszRet; >>> > 1964 } >>> > >>> > Similar cases were spotted in getPROCID(), getStructuredData() and >>> > getProgramName(). ?Hopefully the following patch against 5.8.6 tarball >>> > should solve the problem, I will report back later. >>> > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 17:53:02.000000000 >>> > +0800 >>> > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 +0800 >>> > @@ -1609,7 +1609,7 @@ >>> > >>> > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); >>> > + ? ? ? ? ? ? ? MsgLock(pM); >>> > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); >>> > ? ? ? ?if(pM->pCSPROCID == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ >>> > ?{ >>> > ? ? ? ?uchar *pszRet; >>> > >>> > - ? ? ? MsgUnlock(pM); >>> > + ? ? ? MsgLock(pM); >>> > ? ? ? ?if(pM->pCSStrucData == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); >>> > ? ? ? ?else >>> > @@ -1894,7 +1894,7 @@ >>> > ? ? ? ?uchar *pszRet; >>> > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); >>> > + ? ? ? ? ? ? ? MsgLock(pM); >>> > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); >>> > ? ? ? ?if(pM->pCSProgName == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ >>> > >>> > ? ? ? ?assert(pM != NULL); >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); >>> > + ? ? ? ? ? ? ? MsgLock(pM); >>> > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); >>> > ? ? ? ?if(pM->pCSAPPNAME == NULL) >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); >>> > >>> > >>> > By the way, many crashes led to the convergence, for example //// db1 >>> > core.26581 signal 11, Segmentation fault. >>> > (gdb) p *pThis >>> > $1 = { >>> > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", >>> > ?pszBuf = 0x7f7f5c000dd0 "sshd", >>> > ?iBufSize = 4, >>> > ?iStrLen = 4 >>> > } >>> > (gdb) where >>> > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at >>> > stringbuf.c:334 >>> > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 >>> > '\001') at msg.c:1960 >>> > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, >>> pTpe=0x12ef350, >>> > propID=115 's', pPropLen=0x43fb8f00, >>> > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 >>> > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, pMsg=0x131c140, >>> > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 >>> > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, >>> > pBatch=0x12f7898, pbShutdownImmediate=) >>> > ? ?at ../action.c:738 >>> > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, >>> > pbShutdownImmediate=) at ../action.c:1163 >>> > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, >>> > pWti=0x12f7870) at queue.c:1705 >>> > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at wti.c:315 >>> > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 >>> > #9 ?0x00000031d5a06367 in __nptl_setxid () from /lib64/libpthread.so.0 >>> > #10 0x0000000000000000 in ?? () >>> > >>> > >>> > >>> > Thanks, >>> > Kaiwang >>> > >>> > 2011/12/31 Kaiwang Chen : >>> >> Even worse case >>> >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog >>> >> 5.8.6, log source = /proc/kmsg started. >>> >> ...several SYN flooding entries in the same second... >>> >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size >>> >> (fast): 0x00007fafb8000e00 *** >>> >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory corruption: >>> >> 0x00007fafb8000e20 *** >>> >> >>> >> ?And possibly hours later the host was not responsive to ssh >>> >> connection requests, and even TTY login attempts failed... >>> >> Interestingly the debug log in another file continues a little beyond: >>> >> >>> >> Debug line with all properties: >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >>> >> 'rsyslogd-pstats', >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >>> >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >>> >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' >>> >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 full=0 >>> maxqsize=0 ' >>> >> >>> >> Debug line with all properties: >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid6', >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: >>> >> 'rsyslogd-pstats', >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', >>> >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >>> >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' >>> >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 >>> maxqsize=5 ' >>> >> >>> >> Fortunately the squid service on that host was not badly affected. So >>> >> I did graceful failover and reboot the host, everything became OK. >>> >> Although there was no direct timestamp associated, I guess the >>> >> problem occured soon after rsyslogd started. I also spotted several >>> >> corrupted messages on other hosts when rsyslogd started. They are same >>> version: >>> >> >>> >> # rsyslogd -v >>> >> rsyslogd 5.8.6, compiled with: >>> >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>> >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>> >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>> >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No >>> >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes >>> >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes >>> >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No >>> >> >>> >> >>> >> Thanks, >>> >> Kaiwang >>> >> >>> >> 2011/12/31 Kaiwang Chen : >>> >>> Also spotted corrupted messages from imuxsock: >>> >>> >>> >>> Debug line with all properties: >>> >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: 'squid9', >>> >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: '<88>+?<99>2 >>> >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: '-', >>> >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', >>> >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> >>> ratelimit.numratelimiters=213 ' >>> >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 >>> >>> ratelimit.numratelimiters=213 ' >>> >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 >>> >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' >>> >>> >>> >>> >>> >>> Thanks, >>> >>> Kaiwang >>> >>> >>> >>> 2011/12/27 Kaiwang Chen : >>> >>>> 2011/12/25 ?: >>> >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: >>> >>>>> >>> >>>>>> Hi all, >>> >>>>>> >>> >>>>>> I found rsyslogd occasionally produced corrupted log entries like >>> >>>>>> >>> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel ?- - >>> >>>>>> device eth0 entered promiscuous mode >>> >>>>>> >>> >>>>>> I believe that problem appeared in earlier versions including 5.8.2. >>> >>>>>> Looks like corruption never occur before hostname field. Is it a >>> >>>>>> reported bug? Any clue? >>> >>>>> >>> >>>>> >>> >>>>> the question is if this problem is in the raw message being sent >>> >>>>> to you, or is it something added by the rsyslog processing. >>> >>>>> >>> >>>>> I would suggest adding something along the following line. >>> >>>>> >>> >>>>> :rawmesg, :contains, "#001" ?/var/log/badmessages:RSYSLOG_DEBUG >>> >>>>> >>> >>>>> this will look for any messages with the hex 01 character in them >>> >>>>> and spit out all the info that you can use about the message in to >>> >>>>> the file /var/log/badmessages >>> >>>>> >>> >>>>> run this for a little while and look to see what the raw message >>> >>>>> that is being received over the wire looks like. If it's bad, then >>> >>>>> you need to look at the sender. If the raw message looks sane, but >>> >>>>> rsyslog isn't handling it right, then we can troubleshoot from there. >>> >>>> >>> >>>> Looks like it's related to tcp transmission. On the original server >>> >>>> that get input from unix socket, it is OK; while on the centeral >>> >>>> log server, the raw message is already corrupted. >>> >>>> >>> >>>> ===== on central log server >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >>> >>>> from >>> >>>> UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >>> SNMP >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - >>> >>>> Connection from UDP: [172.25.0.230]:53547 >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >>> >>>> APP-NAME: 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >>> >>>> snmpd >>> >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: 'snmpd', >>> >>>> APP-NAME: 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >>> [172.25.0.230]:53547' >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >>> >>>> snmpd >>> >>>> 24254 - - ?Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', programname: >>> 'snmpd', APP-NAME: >>> >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 gw71 >>> >>>> snmpd >>> >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >>> >>>> from >>> >>>> UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received >>> SNMP >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Connection >>> >>>> from >>> >>>> UDP: [172.25.0.230]:53547 >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >>> >>>> 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> >>>> Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >>> >>>> 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: >>> [172.25.0.230]:53547' >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> Debug line with all properties: >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', PRI: >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: >>> >>>> 'snmpd', >>> >>>> PROCID: '24254', MSGID: '-', >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: >>> >>>> Connection from UDP: [172.25.0.230]:53547' >>> >>>> >>> >>>> >>> >>>> Thanks, >>> >>>> Kaiwang >>> >>>>> >>> >>>>> David Lang >>> >>>>> >>> >>>>> ?> rsyslogd 5.8.6, compiled with: >>> >>>>>> >>> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes >>> >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No >>> >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes >>> >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No >>> >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes >>> >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes >>> >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No >>> >>>>>> >>> >>>>>> /etc/rsyslog.conf >>> >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format >>> >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format >>> >>>>>> >>> >>>>>> $ModLoad imklog >>> >>>>>> $ModLoad imuxsock >>> >>>>>> $ModLoad impstats >>> >>>>>> >>> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off >>> >>>>>> $SystemLogUsePIDFromSystem on >>> >>>>>> >>> >>>>>> $PStatInterval 600 >>> >>>>>> $PStatSeverity 7 >>> >>>>>> >>> >>>>>> $WorkDirectory /var/spool/rsyslog >>> >>>>>> >>> >>>>>> $MainMsgQueueSaveOnShutdown on >>> >>>>>> $MainMsgQueueFileName mq >>> >>>>>> $MainMsgQueueMaxFileSize 5m >>> >>>>>> >>> >>>>>> $ActionQueueType LinkedList >>> >>>>>> $ActionQueueSaveOnShutdown on >>> >>>>>> $ActionQueueFileName dbq >>> >>>>>> $ActionQueueMaxFileSize 10m >>> >>>>>> $ActionResumeRetryCount -1 >>> >>>>>> *.* ? ? ? @@(o)10.2.3.4 >>> >>>>>> >>> >>>>>> >>> >>>>>> # Log all kernel messages to the console. >>> >>>>>> # Logging much else clutters up the screen. >>> >>>>>> #kern.* >>> >>>>>> /dev/console >>> >>>>>> >>> >>>>>> # Log anything (except mail) of level info or higher. >>> >>>>>> # Don't log private authentication messages! >>> >>>>>> *.info;mail.none;authpriv.none;cron.none >>> >>>>>> /var/log/messages >>> >>>>>> >>> >>>>>> # The authpriv file has restricted access. >>> >>>>>> authpriv.* >>> >>>>>> /var/log/secure >>> >>>>>> >>> >>>>>> # Log all the mail messages in one place. >>> >>>>>> mail.* >>> >>>>>> -/var/log/maillog >>> >>>>>> >>> >>>>>> >>> >>>>>> # Log cron stuff >>> >>>>>> cron.* >>> >>>>>> /var/log/cron >>> >>>>>> >>> >>>>>> # Everybody gets emergency messages >>> *.emerg >>> >>>>>> * >>> >>>>>> >>> >>>>>> # Save news errors of level crit and higher in a special file. >>> >>>>>> uucp,news.crit >>> >>>>>> /var/log/spooler >>> >>>>>> >>> >>>>>> # Save boot messages also to boot.log >>> >>>>>> local7.* >>> >>>>>> /var/log/boot.log >>> >>>>>> >>> >>>>>> >>> >>>>>> Thanks, >>> >>>>>> Kaiwang >>> >>>>>> _______________________________________________ >>> >>>>>> rsyslog mailing list >>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>>>>> http://www.rsyslog.com/professional-services/ >>> >>>>>> >>> >>>>> _______________________________________________ >>> >>>>> rsyslog mailing list >>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> >>>>> http://www.rsyslog.com/professional-services/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 09:40:03 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 03:40:03 -0500 Subject: [rsyslog] udp forwarding performance Message-ID: Hi all, I am wondering if anybody has performance testing numbers for rsyslog forwarding (using UDP) ? Currently, I am forwarding all of my rsyslog messages from one host to another host (also running rsyslog). Both hosts are running version 4.2.0. I am seeing throughput of about 2k-3k messages/second on receiver. If I try to send more messages from sender, I find that receiver's rsyslog is dropping logs. Both hosts share same virtual network with network latency of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it seems that rsyslog is capable of performing processing 100k messages/second. Can someone please clarify that this number (100k) is for local syslog messages only? (as opposed to receiving 100k messages/second over network) My intent is to find out what is the expected number of messages that rsyslog can forward to other rsyslog host? (assuming average host spec- dual core, 2 GB RAM) Cheers, Gaurav From rgerhards at hq.adiscon.com Thu Jan 12 09:45:01 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 09:45:01 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> I guess your problem is rooted in name resolution. A full blown DNS cache has only recently been introduced in v6. Previous version had limited caching, which usually works well, but not in all cases. Also 4.2.0 is *way outdated*, so probably the issue (if it is one rooted in code) may be solved by running the current 4.8.6 version. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > Sent: Thursday, January 12, 2012 9:40 AM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] udp forwarding performance > > Hi all, > I am wondering if anybody has performance testing numbers for rsyslog > forwarding (using UDP) ? > > Currently, I am forwarding all of my rsyslog messages from one host to > another host (also running rsyslog). Both hosts are running version > 4.2.0. > I am seeing throughput of about 2k-3k messages/second on receiver. If I > try > to send more messages from sender, I find that receiver's rsyslog is > dropping logs. Both hosts share same virtual network with network > latency > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > seems > that rsyslog is capable of performing processing 100k messages/second. > > Can someone please clarify that this number (100k) is for local syslog > messages only? (as opposed to receiving 100k messages/second over > network) > > My intent is to find out what is the expected number of messages that > rsyslog can forward to other rsyslog host? (assuming average host spec- > dual core, 2 GB RAM) > > Cheers, > Gaurav > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From rgerhards at hq.adiscon.com Thu Jan 12 09:45:40 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 09:45:40 +0100 Subject: [rsyslog] rsyslog-5.8.6 log entry corruption In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA72816DB@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281716@GRFEXC.intern.adiscon.com> Thanks for that info and all your help. I'll probably very soon do an official 5.8.7 release. Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen > Sent: Thursday, January 12, 2012 8:43 AM > To: rsyslog-users > Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption > > To report that such problem is not spotted on these four hosts since > upgraded to 5.8.7-beta. > > Thanks, > Kaiwang > > 2012/1/10 Kaiwang Chen : > > Applied to four hosts where coredump had been spotted; rpm was built > > with the following instructions: > > > > git clone git://git.adiscon.com/git/rsyslog.git > > cd rsyslog > > git checkout v5-stable > > ./autogen.sh > > cd .. > > mv rsyslog rsyslog-5.8.7 > > tar czf rsyslog-5.8.7-beta.tar.gz rsyslog-5.8.7 --exclude '.git*' > > > > rpmbuild -bs rsyslog.spec > > mock /usr/src/redhat/SRPMS/rsyslog-5.8.7-beta.el5.kc.src.rpm > > > > > > I will report back in two days. > > > > Thanks, > > Kaiwang > > > > 2012/1/9 Rainer Gerhards : > >> Thanks for reporting and tracking down this bug. I was on vacation > and have > >> just returned. I applied your patch, it is now merged: > >> > >> > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=791b16ce06d75944e3 > 38a6e5 > >> fa14c0394bde6f1d > >> > >> I'd appreciate if you could double-check that the git version is > correct (you > >> never know...). I will release this version during the next days. > >> > >> Once again for your help! > >> > >> Rainer > >> > >>> -----Original Message----- > >>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> bounces at lists.adiscon.com] On Behalf Of Kaiwang Chen > >>> Sent: Monday, January 09, 2012 9:29 AM > >>> To: rsyslog-users > >>> Subject: Re: [rsyslog] rsyslog-5.8.6 log entry corruption > >>> > >>> To report that coredump and message corruption disappeared after > the > >>> application of my patch; I guess it's fixed. > >>> > >>> Thanks, > >>> Kaiwang > >>> > >>> 2012/1/5 Kaiwang Chen : > >>> > Hello, > >>> > > >>> > I check ChangeLog and found 5.8.4 fixed some bug related to race > >>> > condition as discussed in > >>> > http://bugzilla.adiscon.com/show_bug.cgi?id=275 > >>> > > >>> > with patch protecting modification in the getMSGID(): > >>> > > http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=d66f7b95e081ecb35 > >>> > 6cc230685c5b7c90f8590b4 > >>> > > >>> > > >>> > I guess the situation is almost identical to what in my case: > >>> > corruption is always at the APPNAME field and there is, I > believe, a > >>> > typo in line 1955 from getAPPNAME() effectively ruins the mutex > >>> > proctection: > >>> > 1947 /* rgerhards, 2005-11-24 > >>> > 1948 ?*/ > >>> > 1949 char *getAPPNAME(msg_t *pM, sbool bLockMutex) > >>> > 1950 { > >>> > 1951 ? ? ? ? uchar *pszRet; > >>> > 1952 > >>> > 1953 ? ? ? ? assert(pM != NULL); > >>> > 1954 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > >>> > 1955 ? ? ? ? ? ? ? ? MsgUnlock(pM); ?/* should be MsgLock(pM) */ > >>> > 1956 ? ? ? ? prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > >>> > 1957 ? ? ? ? if(pM->pCSAPPNAME == NULL) > >>> > 1958 ? ? ? ? ? ? ? ? pszRet = UCHAR_CONSTANT(""); > >>> > 1959 ? ? ? ? else > >>> > 1960 ? ? ? ? ? ? ? ? pszRet = rsCStrGetSzStrNoNULL(pM- > >pCSAPPNAME); > >>> > 1961 ? ? ? ? if(bLockMutex == LOCK_MUTEX) > >>> > 1962 ? ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > 1963 ? ? ? ? return (char*)pszRet; > >>> > 1964 } > >>> > > >>> > Similar cases were spotted in getPROCID(), getStructuredData() > and > >>> > getProgramName(). ?Hopefully the following patch against 5.8.6 > tarball > >>> > should solve the problem, I will report back later. > >>> > --- rsyslog-5.8.6/runtime/msg.c.orig ? ?2011-10-21 > 17:53:02.000000000 > >>> > +0800 > >>> > +++ rsyslog-5.8.6/runtime/msg.c 2012-01-05 21:54:26.594666510 > +0800 > >>> > @@ -1609,7 +1609,7 @@ > >>> > > >>> > ? ? ? ?ISOBJ_TYPE_assert(pM, msg); > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > + ? ? ? ? ? ? ? MsgLock(pM); > >>> > ? ? ? ?preparePROCID(pM, MUTEX_ALREADY_LOCKED); > >>> > ? ? ? ?if(pM->pCSPROCID == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1846,7 +1846,7 @@ > >>> > ?{ > >>> > ? ? ? ?uchar *pszRet; > >>> > > >>> > - ? ? ? MsgUnlock(pM); > >>> > + ? ? ? MsgLock(pM); > >>> > ? ? ? ?if(pM->pCSStrucData == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT("-"); > >>> > ? ? ? ?else > >>> > @@ -1894,7 +1894,7 @@ > >>> > ? ? ? ?uchar *pszRet; > >>> > > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > + ? ? ? ? ? ? ? MsgLock(pM); > >>> > ? ? ? ?prepareProgramName(pM, MUTEX_ALREADY_LOCKED); > >>> > ? ? ? ?if(pM->pCSProgName == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); @@ -1952,7 +1952,7 @@ > >>> > > >>> > ? ? ? ?assert(pM != NULL); > >>> > ? ? ? ?if(bLockMutex == LOCK_MUTEX) > >>> > - ? ? ? ? ? ? ? MsgUnlock(pM); > >>> > + ? ? ? ? ? ? ? MsgLock(pM); > >>> > ? ? ? ?prepareAPPNAME(pM, MUTEX_ALREADY_LOCKED); > >>> > ? ? ? ?if(pM->pCSAPPNAME == NULL) > >>> > ? ? ? ? ? ? ? ?pszRet = UCHAR_CONSTANT(""); > >>> > > >>> > > >>> > By the way, many crashes led to the convergence, for example //// > db1 > >>> > core.26581 signal 11, Segmentation fault. > >>> > (gdb) p *pThis > >>> > $1 = { > >>> > ?pBuf = 0x7f7f5c000cd0 "sshd\177\177", > >>> > ?pszBuf = 0x7f7f5c000dd0 "sshd", > >>> > ?iBufSize = 4, > >>> > ?iStrLen = 4 > >>> > } > >>> > (gdb) where > >>> > #0 ?0x000000000041e6a9 in rsCStrGetSzStr (pThis=0x131cb10) at > >>> > stringbuf.c:334 > >>> > #1 ?0x000000000041b1bf in getAPPNAME (pM=0x131c140, bLockMutex=1 > >>> > '\001') at msg.c:1960 > >>> > #2 ?0x000000000041c0f8 in MsgGetProp (pMsg=0x131c140, > >>> pTpe=0x12ef350, > >>> > propID=115 's', pPropLen=0x43fb8f00, > >>> > ? ?pbMustBeFreed=0xfefefefefefefeff) at msg.c:2429 > >>> > #3 ?0x000000000043be88 in tplToString (pTpl=0x12eeca0, > pMsg=0x131c140, > >>> > ppBuf=0x12f7910, pLenBuf=0x12f7930) at ../template.c:119 > >>> > #4 ?0x0000000000438498 in prepareBatch (pAction=0x12f72d0, > >>> > pBatch=0x12f7898, pbShutdownImmediate=) > >>> > ? ?at ../action.c:738 > >>> > #5 ?processBatchMain (pAction=0x12f72d0, pBatch=0x12f7898, > >>> > pbShutdownImmediate=) at ../action.c:1163 > >>> > #6 ?0x000000000043190d in ConsumerReg (pThis=0x12f73f0, > >>> > pWti=0x12f7870) at queue.c:1705 > >>> > #7 ?0x000000000042b0bd in wtiWorker (pThis=0x12f7870) at > wti.c:315 > >>> > #8 ?0x000000000042abd8 in wtpWorker (arg=0x12f7870) at wtp.c:387 > >>> > #9 ?0x00000031d5a06367 in __nptl_setxid () from > /lib64/libpthread.so.0 > >>> > #10 0x0000000000000000 in ?? () > >>> > > >>> > > >>> > > >>> > Thanks, > >>> > Kaiwang > >>> > > >>> > 2011/12/31 Kaiwang Chen : > >>> >> Even worse case > >>> >> <6>1 2011-12-31T12:04:05.036812+08:00 squid6 kernel ?- - imklog > >>> >> 5.8.6, log source = /proc/kmsg started. > >>> >> ...several SYN flooding entries in the same second... > >>> >> *** glibc detected *** /sbin/rsyslogd: free(): invalid next size > >>> >> (fast): 0x00007fafb8000e00 *** > >>> >> *** glibc detected *** /sbin/rsyslogd: malloc(): memory > corruption: > >>> >> 0x00007fafb8000e20 *** > >>> >> > >>> >> ?And possibly hours later the host was not responsive to ssh > >>> >> connection requests, and even TTY login attempts failed... > >>> >> Interestingly the debug log in another file continues a little > beyond: > >>> >> > >>> >> Debug line with all properties: > >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: > 'squid6', > >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >>> >> 'rsyslogd-pstats', > >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >>> >> msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >>> >> escaped msg: 'main Q[DA]: size=0 enqueued=0 full=0 maxqsize=0 ' > >>> >> inputname: impstats rawmsg: 'main Q[DA]: size=0 enqueued=0 > full=0 > >>> maxqsize=0 ' > >>> >> > >>> >> Debug line with all properties: > >>> >> FROMHOST: 'squid6', fromhost-ip: '127.0.0.1', HOSTNAME: > 'squid6', > >>> >> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > >>> >> 'rsyslogd-pstats', > >>> >> APP-NAME: 'rsyslogd-pstats', PROCID: '', MSGID: '-', > >>> >> TIMESTAMP: 'Dec 31 12:24:05', STRUCTURED-DATA: '-', > >>> >> msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >>> >> escaped msg: 'main Q: size=4 enqueued=30 full=0 maxqsize=5 ' > >>> >> inputname: impstats rawmsg: 'main Q: size=4 enqueued=30 full=0 > >>> maxqsize=5 ' > >>> >> > >>> >> Fortunately the squid service on that host was not badly > affected. So > >>> >> I did graceful failover and reboot the host, everything became > OK. > >>> >> Although there was no direct timestamp associated, I guess the > >>> >> problem occured soon after rsyslogd started. I also spotted > several > >>> >> corrupted messages on other hosts when rsyslogd started. They > are same > >>> version: > >>> >> > >>> >> # rsyslogd -v > >>> >> rsyslogd 5.8.6, compiled with: > >>> >> ? ? ? ?FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >>> >> ? ? ? ?FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >>> >> ? ? ? ?GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >>> >> ? ? ? ?FEATURE_DEBUG (debug build, slow code): No > >>> >> ? ? ? ?32bit Atomic operations supported: ? ? ?Yes > >>> >> ? ? ? ?64bit Atomic operations supported: ? ? ?Yes > >>> >> ? ? ? ?Runtime Instrumentation (slow code): ? ?No > >>> >> > >>> >> > >>> >> Thanks, > >>> >> Kaiwang > >>> >> > >>> >> 2011/12/31 Kaiwang Chen : > >>> >>> Also spotted corrupted messages from imuxsock: > >>> >>> > >>> >>> Debug line with all properties: > >>> >>> FROMHOST: 'squid9', fromhost-ip: '127.0.0.1', HOSTNAME: > 'squid9', > >>> >>> PRI: 47, syslogtag 'rsyslogd-pstats:', programname: > '<88>+?<99>2 > >>> >>> <88>+?<99>rssyslogd-pstats', APP-NAME: '', PROCID: '', MSGID: > '-', > >>> >>> TIMESTAMP: 'Dec 30 21:46:14', STRUCTURED-DATA: '-', > >>> >>> msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> >>> ratelimit.numratelimiters=213 ' > >>> >>> escaped msg: 'imuxsock: submitted=429 ratelimit.discarded=0 > >>> >>> ratelimit.numratelimiters=213 ' > >>> >>> inputname: impstats rawmsg: 'imuxsock: submitted=429 > >>> >>> ratelimit.discarded=0 ratelimit.numratelimiters=213 ' > >>> >>> > >>> >>> > >>> >>> Thanks, > >>> >>> Kaiwang > >>> >>> > >>> >>> 2011/12/27 Kaiwang Chen : > >>> >>>> 2011/12/25 ?: > >>> >>>>> On Fri, 23 Dec 2011, Kaiwang Chen wrote: > >>> >>>>> > >>> >>>>>> Hi all, > >>> >>>>>> > >>> >>>>>> I found rsyslogd occasionally produced corrupted log entries > like > >>> >>>>>> > >>> >>>>>> <6>1 2011-12-23T23:03:18.089938+08:00 db1 #001 D^kernel > ?- - > >>> >>>>>> device eth0 entered promiscuous mode > >>> >>>>>> > >>> >>>>>> I believe that problem appeared in earlier versions > including 5.8.2. > >>> >>>>>> Looks like corruption never occur before hostname field. Is > it a > >>> >>>>>> reported bug? Any clue? > >>> >>>>> > >>> >>>>> > >>> >>>>> the question is if this problem is in the raw message being > sent > >>> >>>>> to you, or is it something added by the rsyslog processing. > >>> >>>>> > >>> >>>>> I would suggest adding something along the following line. > >>> >>>>> > >>> >>>>> :rawmesg, :contains, "#001" > ?/var/log/badmessages:RSYSLOG_DEBUG > >>> >>>>> > >>> >>>>> this will look for any messages with the hex 01 character in > them > >>> >>>>> and spit out all the info that you can use about the message > in to > >>> >>>>> the file /var/log/badmessages > >>> >>>>> > >>> >>>>> run this for a little while and look to see what the raw > message > >>> >>>>> that is being received over the wire looks like. If it's bad, > then > >>> >>>>> you need to look at the sender. If the raw message looks > sane, but > >>> >>>>> rsyslog isn't handling it right, then we can troubleshoot > from there. > >>> >>>> > >>> >>>> Looks like it's related to tcp transmission. On the original > server > >>> >>>> that get input from unix socket, it is OK; while on the > centeral > >>> >>>> log server, the raw message is already corrupted. > >>> >>>> > >>> >>>> ===== on central log server > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - > ?Connection > >>> >>>> from > >>> >>>> UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > >>> SNMP > >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 2#032#0251 - - > >>> >>>> Connection from UDP: [172.25.0.230]:53547 > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: > 'snmpd', > >>> >>>> APP-NAME: 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 > gw71 > >>> >>>> snmpd > >>> >>>> 24254 - - ?Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[24254]', programname: > 'snmpd', > >>> >>>> APP-NAME: 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: > >>> [172.25.0.230]:53547' > >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 > gw71 > >>> >>>> snmpd > >>> >>>> 24254 - - ?Received SNMP packet(s) from UDP: > [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: '172.25.0.71', fromhost-ip: '172.25.0.71', HOSTNAME: > >>> >>>> 'gw71', PRI: 30, syslogtag 'snmpd[2#032#0251]', > programname: > >>> 'snmpd', APP-NAME: > >>> >>>> 'snmpd', PROCID: '2#032#0251', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imptcp rawmsg: '<30>1 2011-12-27T14:06:56+08:00 > gw71 > >>> >>>> snmpd > >>> >>>> 2#032#0251 - - ?Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> ===== on the orginal host ?(*.* ? ? ? @@(o)172.25.0.230:514) > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - > ?Connection > >>> >>>> from > >>> >>>> UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - ?Received > >>> SNMP > >>> >>>> packet(s) from UDP: [172.25.0.230]:53547 > >>> >>>> <30>1 2011-12-27T14:06:56+08:00 gw71 snmpd 24254 - - > ?Connection > >>> >>>> from > >>> >>>> UDP: [172.25.0.230]:53547 > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', > PRI: > >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>> >>>> 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>> >>>> Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', > PRI: > >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>> >>>> 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Received SNMP packet(s) from UDP: > >>> [172.25.0.230]:53547' > >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>> >>>> Received SNMP packet(s) from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> Debug line with all properties: > >>> >>>> FROMHOST: 'gw71', fromhost-ip: '127.0.0.1', HOSTNAME: 'gw71', > PRI: > >>> >>>> 30, syslogtag 'snmpd[24254]:', programname: 'snmpd', APP-NAME: > >>> >>>> 'snmpd', > >>> >>>> PROCID: '24254', MSGID: '-', > >>> >>>> TIMESTAMP: 'Dec 27 14:06:56', STRUCTURED-DATA: '-', > >>> >>>> msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> escaped msg: ' Connection from UDP: [172.25.0.230]:53547' > >>> >>>> inputname: imuxsock rawmsg: '<30>Dec 27 14:06:56 snmpd[24254]: > >>> >>>> Connection from UDP: [172.25.0.230]:53547' > >>> >>>> > >>> >>>> > >>> >>>> Thanks, > >>> >>>> Kaiwang > >>> >>>>> > >>> >>>>> David Lang > >>> >>>>> > >>> >>>>> ?> rsyslogd 5.8.6, compiled with: > >>> >>>>>> > >>> >>>>>> ? ? ? FEATURE_REGEXP: ? ? ? ? ? ? ? ? ? ? ? ? Yes > >>> >>>>>> ? ? ? FEATURE_LARGEFILE: ? ? ? ? ? ? ? ? ? ? ?No > >>> >>>>>> ? ? ? GSSAPI Kerberos 5 support: ? ? ? ? ? ? ?Yes > >>> >>>>>> ? ? ? FEATURE_DEBUG (debug build, slow code): No > >>> >>>>>> ? ? ? 32bit Atomic operations supported: ? ? ?Yes > >>> >>>>>> ? ? ? 64bit Atomic operations supported: ? ? ?Yes > >>> >>>>>> ? ? ? Runtime Instrumentation (slow code): ? ?No > >>> >>>>>> > >>> >>>>>> /etc/rsyslog.conf > >>> >>>>>> $ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>> >>>>>> $ActionForwardDefaultTemplate RSYSLOG_SyslogProtocol23Format > >>> >>>>>> > >>> >>>>>> $ModLoad imklog > >>> >>>>>> $ModLoad imuxsock > >>> >>>>>> $ModLoad impstats > >>> >>>>>> > >>> >>>>>> $SystemLogSocketIgnoreMsgTimestamp ?off > >>> >>>>>> $SystemLogUsePIDFromSystem on > >>> >>>>>> > >>> >>>>>> $PStatInterval 600 > >>> >>>>>> $PStatSeverity 7 > >>> >>>>>> > >>> >>>>>> $WorkDirectory /var/spool/rsyslog > >>> >>>>>> > >>> >>>>>> $MainMsgQueueSaveOnShutdown on > >>> >>>>>> $MainMsgQueueFileName mq > >>> >>>>>> $MainMsgQueueMaxFileSize 5m > >>> >>>>>> > >>> >>>>>> $ActionQueueType LinkedList > >>> >>>>>> $ActionQueueSaveOnShutdown on > >>> >>>>>> $ActionQueueFileName dbq > >>> >>>>>> $ActionQueueMaxFileSize 10m > >>> >>>>>> $ActionResumeRetryCount -1 > >>> >>>>>> *.* ? ? ? @@(o)10.2.3.4 > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> # Log all kernel messages to the console. > >>> >>>>>> # Logging much else clutters up the screen. > >>> >>>>>> #kern.* > >>> >>>>>> /dev/console > >>> >>>>>> > >>> >>>>>> # Log anything (except mail) of level info or higher. > >>> >>>>>> # Don't log private authentication messages! > >>> >>>>>> *.info;mail.none;authpriv.none;cron.none > >>> >>>>>> /var/log/messages > >>> >>>>>> > >>> >>>>>> # The authpriv file has restricted access. > >>> >>>>>> authpriv.* > >>> >>>>>> /var/log/secure > >>> >>>>>> > >>> >>>>>> # Log all the mail messages in one place. > >>> >>>>>> mail.* > >>> >>>>>> -/var/log/maillog > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> # Log cron stuff > >>> >>>>>> cron.* > >>> >>>>>> /var/log/cron > >>> >>>>>> > >>> >>>>>> # Everybody gets emergency messages > >>> *.emerg > >>> >>>>>> * > >>> >>>>>> > >>> >>>>>> # Save news errors of level crit and higher in a special > file. > >>> >>>>>> uucp,news.crit > >>> >>>>>> /var/log/spooler > >>> >>>>>> > >>> >>>>>> # Save boot messages also to boot.log > >>> >>>>>> local7.* > >>> >>>>>> /var/log/boot.log > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Thanks, > >>> >>>>>> Kaiwang > >>> >>>>>> _______________________________________________ > >>> >>>>>> rsyslog mailing list > >>> >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> >>>>>> http://www.rsyslog.com/professional-services/ > >>> >>>>>> > >>> >>>>> _______________________________________________ > >>> >>>>> rsyslog mailing list > >>> >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> >>>>> http://www.rsyslog.com/professional-services/ > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 09:48:44 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 03:48:44 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> Message-ID: Thanks Rainer! I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets install by apt-get. I will try to install newer version. BTW, what should be the expected performance (messages/sec) for sending and receiving messages over UDP? Thanks again, Gaurav On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards wrote: > I guess your problem is rooted in name resolution. A full blown DNS cache > has > only recently been introduced in v6. Previous version had limited caching, > which usually works well, but not in all cases. Also 4.2.0 is *way > outdated*, > so probably the issue (if it is one rooted in code) may be solved by > running > the current 4.8.6 version. > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > Sent: Thursday, January 12, 2012 9:40 AM > > To: rsyslog at lists.adiscon.com > > Subject: [rsyslog] udp forwarding performance > > > > Hi all, > > I am wondering if anybody has performance testing numbers for rsyslog > > forwarding (using UDP) ? > > > > Currently, I am forwarding all of my rsyslog messages from one host to > > another host (also running rsyslog). Both hosts are running version > > 4.2.0. > > I am seeing throughput of about 2k-3k messages/second on receiver. If I > > try > > to send more messages from sender, I find that receiver's rsyslog is > > dropping logs. Both hosts share same virtual network with network > > latency > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > > seems > > that rsyslog is capable of performing processing 100k messages/second. > > > > Can someone please clarify that this number (100k) is for local syslog > > messages only? (as opposed to receiving 100k messages/second over > > network) > > > > My intent is to find out what is the expected number of messages that > > rsyslog can forward to other rsyslog host? (assuming average host spec- > > dual core, 2 GB RAM) > > > > Cheers, > > Gaurav > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Thu Jan 12 09:52:12 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 09:52:12 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > Sent: Thursday, January 12, 2012 9:49 AM > To: rsyslog-users > Subject: Re: [rsyslog] udp forwarding performance > > Thanks Rainer! > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets install > by > apt-get. I will try to install newer version. > > BTW, what should be the expected performance (messages/sec) for sending > and > receiving messages over UDP? Wiht v6 > 500k/second, but you possibly need to enable realtime priority in order for the OS to know that rsyslog must be sufficiently often activated. This number assumes lightweight processing (no database inserts or such). Try first without realtime. Rainer > > Thanks again, > Gaurav > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > wrote: > > > I guess your problem is rooted in name resolution. A full blown DNS > cache > > has > > only recently been introduced in v6. Previous version had limited > caching, > > which usually works well, but not in all cases. Also 4.2.0 is *way > > outdated*, > > so probably the issue (if it is one rooted in code) may be solved by > > running > > the current 4.8.6 version. > > > > HTH > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > Sent: Thursday, January 12, 2012 9:40 AM > > > To: rsyslog at lists.adiscon.com > > > Subject: [rsyslog] udp forwarding performance > > > > > > Hi all, > > > I am wondering if anybody has performance testing numbers for > rsyslog > > > forwarding (using UDP) ? > > > > > > Currently, I am forwarding all of my rsyslog messages from one host > to > > > another host (also running rsyslog). Both hosts are running version > > > 4.2.0. > > > I am seeing throughput of about 2k-3k messages/second on receiver. > If I > > > try > > > to send more messages from sender, I find that receiver's rsyslog > is > > > dropping logs. Both hosts share same virtual network with network > > > latency > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > > > seems > > > that rsyslog is capable of performing processing 100k > messages/second. > > > > > > Can someone please clarify that this number (100k) is for local > syslog > > > messages only? (as opposed to receiving 100k messages/second over > > > network) > > > > > > My intent is to find out what is the expected number of messages > that > > > rsyslog can forward to other rsyslog host? (assuming average host > spec- > > > dual core, 2 GB RAM) > > > > > > Cheers, > > > Gaurav > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 10:32:55 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 04:32:55 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, I am now running version 5.8.1. I was running into rate-limiting issues so I've set following- $IMUXSockRateLimitInterval 0 $SystemLogRateLimitInterval 0 After this, the sender host is sending only about 300-400 messages, then dropping a lot of them (literally thousands). I ran Wireshark to confirm whether if it is receiver's issue or sender's and found that indeed sender is not sending messages. Can you please suggest which directives I should set in config to troubleshoot this? Thanks, Gaurav On Thu, Jan 12, 2012 at 3:52 AM, Rainer Gerhards wrote: > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > Sent: Thursday, January 12, 2012 9:49 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] udp forwarding performance > > > > Thanks Rainer! > > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets install > > by > > apt-get. I will try to install newer version. > > > > BTW, what should be the expected performance (messages/sec) for sending > > and > > receiving messages over UDP? > > Wiht v6 > 500k/second, but you possibly need to enable realtime priority in > order for the OS to know that rsyslog must be sufficiently often activated. > This number assumes lightweight processing (no database inserts or such). > > Try first without realtime. > > Rainer > > > > Thanks again, > > Gaurav > > > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > > wrote: > > > > > I guess your problem is rooted in name resolution. A full blown DNS > > cache > > > has > > > only recently been introduced in v6. Previous version had limited > > caching, > > > which usually works well, but not in all cases. Also 4.2.0 is *way > > > outdated*, > > > so probably the issue (if it is one rooted in code) may be solved by > > > running > > > the current 4.8.6 version. > > > > > > HTH > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > Sent: Thursday, January 12, 2012 9:40 AM > > > > To: rsyslog at lists.adiscon.com > > > > Subject: [rsyslog] udp forwarding performance > > > > > > > > Hi all, > > > > I am wondering if anybody has performance testing numbers for > > rsyslog > > > > forwarding (using UDP) ? > > > > > > > > Currently, I am forwarding all of my rsyslog messages from one host > > to > > > > another host (also running rsyslog). Both hosts are running version > > > > 4.2.0. > > > > I am seeing throughput of about 2k-3k messages/second on receiver. > > If I > > > > try > > > > to send more messages from sender, I find that receiver's rsyslog > > is > > > > dropping logs. Both hosts share same virtual network with network > > > > latency > > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it > > > > seems > > > > that rsyslog is capable of performing processing 100k > > messages/second. > > > > > > > > Can someone please clarify that this number (100k) is for local > > syslog > > > > messages only? (as opposed to receiving 100k messages/second over > > > > network) > > > > > > > > My intent is to find out what is the expected number of messages > > that > > > > rsyslog can forward to other rsyslog host? (assuming average host > > spec- > > > > dual core, 2 GB RAM) > > > > > > > > Cheers, > > > > Gaurav > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Thu Jan 12 10:55:53 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 10:55:53 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA728171A@GRFEXC.intern.adiscon.com> Have your turned off name resolution for a test? Why not use 5.8.6? rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > Sent: Thursday, January 12, 2012 10:33 AM > To: rsyslog-users > Subject: Re: [rsyslog] udp forwarding performance > > Hi Rainer, > I am now running version 5.8.1. I was running into rate-limiting issues > so > I've set following- > $IMUXSockRateLimitInterval 0 > $SystemLogRateLimitInterval 0 > > After this, the sender host is sending only about 300-400 messages, > then > dropping a lot of them (literally thousands). I ran Wireshark to > confirm > whether if it is receiver's issue or sender's and found that indeed > sender > is not sending messages. > > Can you please suggest which directives I should set in config to > troubleshoot this? > > Thanks, > Gaurav > > > > On Thu, Jan 12, 2012 at 3:52 AM, Rainer Gerhards > wrote: > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > Sent: Thursday, January 12, 2012 9:49 AM > > > To: rsyslog-users > > > Subject: Re: [rsyslog] udp forwarding performance > > > > > > Thanks Rainer! > > > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets > install > > > by > > > apt-get. I will try to install newer version. > > > > > > BTW, what should be the expected performance (messages/sec) for > sending > > > and > > > receiving messages over UDP? > > > > Wiht v6 > 500k/second, but you possibly need to enable realtime > priority in > > order for the OS to know that rsyslog must be sufficiently often > activated. > > This number assumes lightweight processing (no database inserts or > such). > > > > Try first without realtime. > > > > Rainer > > > > > > Thanks again, > > > Gaurav > > > > > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > > > wrote: > > > > > > > I guess your problem is rooted in name resolution. A full blown > DNS > > > cache > > > > has > > > > only recently been introduced in v6. Previous version had limited > > > caching, > > > > which usually works well, but not in all cases. Also 4.2.0 is > *way > > > > outdated*, > > > > so probably the issue (if it is one rooted in code) may be solved > by > > > > running > > > > the current 4.8.6 version. > > > > > > > > HTH > > > > Rainer > > > > > > > > > -----Original Message----- > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > > Sent: Thursday, January 12, 2012 9:40 AM > > > > > To: rsyslog at lists.adiscon.com > > > > > Subject: [rsyslog] udp forwarding performance > > > > > > > > > > Hi all, > > > > > I am wondering if anybody has performance testing numbers for > > > rsyslog > > > > > forwarding (using UDP) ? > > > > > > > > > > Currently, I am forwarding all of my rsyslog messages from one > host > > > to > > > > > another host (also running rsyslog). Both hosts are running > version > > > > > 4.2.0. > > > > > I am seeing throughput of about 2k-3k messages/second on > receiver. > > > If I > > > > > try > > > > > to send more messages from sender, I find that receiver's > rsyslog > > > is > > > > > dropping logs. Both hosts share same virtual network with > network > > > > > latency > > > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB > RAM. > > > > > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog- > performance.html it > > > > > seems > > > > > that rsyslog is capable of performing processing 100k > > > messages/second. > > > > > > > > > > Can someone please clarify that this number (100k) is for local > > > syslog > > > > > messages only? (as opposed to receiving 100k messages/second > over > > > > > network) > > > > > > > > > > My intent is to find out what is the expected number of > messages > > > that > > > > > rsyslog can forward to other rsyslog host? (assuming average > host > > > spec- > > > > > dual core, 2 GB RAM) > > > > > > > > > > Cheers, > > > > > Gaurav > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com/professional-services/ > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > > > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From gk at pivotalsecurity.com Thu Jan 12 11:05:03 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 05:05:03 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA728171A@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281715@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281717@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA728171A@GRFEXC.intern.adiscon.com> Message-ID: I will try to install 5.8.6, btw, disabling dns resolution didn't help. On Thu, Jan 12, 2012 at 4:55 AM, Rainer Gerhards wrote: > Have your turned off name resolution for a test? Why not use 5.8.6? > > rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > Sent: Thursday, January 12, 2012 10:33 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] udp forwarding performance > > > > Hi Rainer, > > I am now running version 5.8.1. I was running into rate-limiting issues > > so > > I've set following- > > $IMUXSockRateLimitInterval 0 > > $SystemLogRateLimitInterval 0 > > > > After this, the sender host is sending only about 300-400 messages, > > then > > dropping a lot of them (literally thousands). I ran Wireshark to > > confirm > > whether if it is receiver's issue or sender's and found that indeed > > sender > > is not sending messages. > > > > Can you please suggest which directives I should set in config to > > troubleshoot this? > > > > Thanks, > > Gaurav > > > > > > > > On Thu, Jan 12, 2012 at 3:52 AM, Rainer Gerhards > > wrote: > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > Sent: Thursday, January 12, 2012 9:49 AM > > > > To: rsyslog-users > > > > Subject: Re: [rsyslog] udp forwarding performance > > > > > > > > Thanks Rainer! > > > > I am using Ubuntu 10.04 LTS and 4.2.0 is the version which gets > > install > > > > by > > > > apt-get. I will try to install newer version. > > > > > > > > BTW, what should be the expected performance (messages/sec) for > > sending > > > > and > > > > receiving messages over UDP? > > > > > > Wiht v6 > 500k/second, but you possibly need to enable realtime > > priority in > > > order for the OS to know that rsyslog must be sufficiently often > > activated. > > > This number assumes lightweight processing (no database inserts or > > such). > > > > > > Try first without realtime. > > > > > > Rainer > > > > > > > > Thanks again, > > > > Gaurav > > > > > > > > On Thu, Jan 12, 2012 at 3:45 AM, Rainer Gerhards > > > > wrote: > > > > > > > > > I guess your problem is rooted in name resolution. A full blown > > DNS > > > > cache > > > > > has > > > > > only recently been introduced in v6. Previous version had limited > > > > caching, > > > > > which usually works well, but not in all cases. Also 4.2.0 is > > *way > > > > > outdated*, > > > > > so probably the issue (if it is one rooted in code) may be solved > > by > > > > > running > > > > > the current 4.8.6 version. > > > > > > > > > > HTH > > > > > Rainer > > > > > > > > > > > -----Original Message----- > > > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > > > bounces at lists.adiscon.com] On Behalf Of Gaurav Kumar > > > > > > Sent: Thursday, January 12, 2012 9:40 AM > > > > > > To: rsyslog at lists.adiscon.com > > > > > > Subject: [rsyslog] udp forwarding performance > > > > > > > > > > > > Hi all, > > > > > > I am wondering if anybody has performance testing numbers for > > > > rsyslog > > > > > > forwarding (using UDP) ? > > > > > > > > > > > > Currently, I am forwarding all of my rsyslog messages from one > > host > > > > to > > > > > > another host (also running rsyslog). Both hosts are running > > version > > > > > > 4.2.0. > > > > > > I am seeing throughput of about 2k-3k messages/second on > > receiver. > > > > If I > > > > > > try > > > > > > to send more messages from sender, I find that receiver's > > rsyslog > > > > is > > > > > > dropping logs. Both hosts share same virtual network with > > network > > > > > > latency > > > > > > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB > > RAM. > > > > > > > > > > > > As per http://blog.gerhards.net/2008/10/rsyslog- > > performance.html it > > > > > > seems > > > > > > that rsyslog is capable of performing processing 100k > > > > messages/second. > > > > > > > > > > > > Can someone please clarify that this number (100k) is for local > > > > syslog > > > > > > messages only? (as opposed to receiving 100k messages/second > > over > > > > > > network) > > > > > > > > > > > > My intent is to find out what is the expected number of > > messages > > > > that > > > > > > rsyslog can forward to other rsyslog host? (assuming average > > host > > > > spec- > > > > > > dual core, 2 GB RAM) > > > > > > > > > > > > Cheers, > > > > > > Gaurav > > > > > > _______________________________________________ > > > > > > rsyslog mailing list > > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > > > > rsyslog mailing list > > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > > http://www.rsyslog.com/professional-services/ > > > > > > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > -- From rgerhards at hq.adiscon.com Thu Jan 12 15:14:55 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Thu, 12 Jan 2012 15:14:55 +0100 Subject: [rsyslog] update on rsyslog licensing and funding Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> Hi all, we had a lengthy discussion last November on rsyslog funding and licensing. At that time, I promised to keep you updated. I have blogged about the latest state: http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html Feedback is appreciated. Rainer From david at lang.hm Thu Jan 12 20:51:37 2012 From: david at lang.hm (david at lang.hm) Date: Thu, 12 Jan 2012 11:51:37 -0800 (PST) Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: On Thu, 12 Jan 2012, Gaurav Kumar wrote: > Hi all, > I am wondering if anybody has performance testing numbers for rsyslog > forwarding (using UDP) ? > > Currently, I am forwarding all of my rsyslog messages from one host to > another host (also running rsyslog). Both hosts are running version 4.2.0. > I am seeing throughput of about 2k-3k messages/second on receiver. If I try > to send more messages from sender, I find that receiver's rsyslog is > dropping logs. Both hosts share same virtual network with network latency > of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. > > As per http://blog.gerhards.net/2008/10/rsyslog-performance.html it seems > that rsyslog is capable of performing processing 100k messages/second. > > Can someone please clarify that this number (100k) is for local syslog > messages only? (as opposed to receiving 100k messages/second over network) I have received >380K logs/sec (pretty close to gig-e wire speed) with no packet loss. I routinely do tens of thousands of logs/sec between machines. > My intent is to find out what is the expected number of messages that > rsyslog can forward to other rsyslog host? (assuming average host spec- > dual core, 2 GB RAM) it is going to depend a lot on what processing you are doing. disable name resolution (-x on the command line) when you have the systems dropping logs, what is the utilization of the rsyslog threads? David Lang From gk at pivotalsecurity.com Fri Jan 13 04:36:50 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Thu, 12 Jan 2012 22:36:50 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: I was finally able to resolve the issue. The root cause was improper tcp/ip config on reciver host. Thanks all for the hints. On Jan 12, 2012 2:51 PM, wrote: > On Thu, 12 Jan 2012, Gaurav Kumar wrote: > > Hi all, >> I am wondering if anybody has performance testing numbers for rsyslog >> forwarding (using UDP) ? >> >> Currently, I am forwarding all of my rsyslog messages from one host to >> another host (also running rsyslog). Both hosts are running version 4.2.0. >> I am seeing throughput of about 2k-3k messages/second on receiver. If I >> try >> to send more messages from sender, I find that receiver's rsyslog is >> dropping logs. Both hosts share same virtual network with network latency >> of just about 1-2 ms. Both hosts have 2 virtual CPUs and 2 GB RAM. >> >> As per http://blog.gerhards.net/2008/**10/rsyslog-performance.htmlit seems >> that rsyslog is capable of performing processing 100k messages/second. >> >> Can someone please clarify that this number (100k) is for local syslog >> messages only? (as opposed to receiving 100k messages/second over network) >> > > I have received >380K logs/sec (pretty close to gig-e wire speed) with no > packet loss. I routinely do tens of thousands of logs/sec between machines. > > My intent is to find out what is the expected number of messages that >> rsyslog can forward to other rsyslog host? (assuming average host spec- >> dual core, 2 GB RAM) >> > > it is going to depend a lot on what processing you are doing. > > disable name resolution (-x on the command line) > > when you have the systems dropping logs, what is the utilization of the > rsyslog threads? > > David Lang > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From r.bhatia at ipax.at Fri Jan 13 09:32:10 2012 From: r.bhatia at ipax.at (Raoul Bhatia [IPAX]) Date: Fri, 13 Jan 2012 09:32:10 +0100 Subject: [rsyslog] udp forwarding performance In-Reply-To: References: Message-ID: <4F0FEC0A.9080204@ipax.at> On 2012-01-13 04:36, Gaurav Kumar wrote: > I was finally able to resolve the issue. The root cause was improper tcp/ip > config on reciver host. Thanks all for the hints. hi gaurav! could you please elaborate on the settings you had to tune/ which settings you are now using? thanks, raoul -- ____________________________________________________________________ DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at Technischer Leiter IPAX - Aloy Bhatia Hava OG web. http://www.ipax.at Barawitzkagasse 10/2/2/11 email. office at ipax.at 1190 Wien tel. +43 1 3670030 FN 277995t HG Wien fax. +43 1 3670030 15 ____________________________________________________________________ From michael at maymann.org Fri Jan 13 13:15:05 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 13 Jan 2012 13:15:05 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples Message-ID: Hi List, I'm new to rsyslog/syslog in general. I would like to syslog from all my 100+ network devices. Preferably I would like a FQDN.log file for each host (or a FQDN-dir containing logs from this host if more logfiles per host are best practice)... Can anyone give me an example of (or link to) best practice of this kind of setup. Thanks in advance :-) ! ~maymann From michael at maymann.org Fri Jan 13 14:43:06 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 13 Jan 2012 14:43:06 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Furthermore: would it be possible to validate FQDN from DNS and not from syslog-info hostname. We are getting a lot of weird logfiles as some applications are not including the hostname as the first parameter in the syslog-entries, e.g.: Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList text /tmp/hostCompatList"'#012#01212/16/11 09:47:40 Would be nice to validate FQDN from sender DNS query... Thanks in advance :-) ! ~maymann 2012/1/13 Michael Maymann > Hi List, > > I'm new to rsyslog/syslog in general. > > I would like to syslog from all my 100+ network devices. > Preferably I would like a FQDN.log file for each host (or a FQDN-dir > containing logs from this host if more logfiles per host are best > practice)... > > Can anyone give me an example of (or link to) best practice of this kind > of setup. > > > Thanks in advance :-) ! > > ~maymann > From rgerhards at hq.adiscon.com Fri Jan 13 18:38:40 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Fri, 13 Jan 2012 18:38:40 +0100 Subject: [rsyslog] update on rsyslog licensing and funding In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> Hi all, I created some confusion if the rsyslog project stays under GPLv3 with yesterday's post. In fact, it does! I wrote a new posting today in the hope to clarify some things: http://blog.gerhards.net/2012/01/rsyslog-will-remain-gplv3-licensed.html Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Thursday, January 12, 2012 3:15 PM > To: rsyslog at lists.adiscon.com > Subject: [rsyslog] update on rsyslog licensing and funding > > Hi all, > > we had a lengthy discussion last November on rsyslog funding and > licensing. > At that time, I promised to keep you updated. I have blogged about the > latest > state: > > http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html > > Feedback is appreciated. > > Rainer > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From david at lang.hm Fri Jan 13 21:34:32 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Jan 2012 12:34:32 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: you need to be aware that doing the DNS queries is rather expensive (although I think I saw a comment that in the very latest 6.2 version there may now be a DNS cache that will drastically help) you would need to create a template with FROMHOST in it and use that as the filename to write to (look for dynafile in the documentation) note that if you are relaying logs from one machine to another, only the first machine will see the true source in FROMHOST, machines after that will only see the relay box. let me know if this doesn't give you enough clues to learn how to do this. David Lang On Fri, 13 Jan 2012, Michael Maymann wrote: > Date: Fri, 13 Jan 2012 14:43:06 +0100 > From: Michael Maymann > Reply-To: rsyslog-users > To: rsyslog at lists.adiscon.com > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Furthermore: would it be possible to validate FQDN from DNS and not from > syslog-info hostname. > We are getting a lot of weird logfiles as some applications are not > including the hostname as the first parameter in the syslog-entries, e.g.: > Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] > #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList > text /tmp/hostCompatList"'#012#01212/16/11 09:47:40 > > Would be nice to validate FQDN from sender DNS query... > > Thanks in advance :-) ! > ~maymann > > > 2012/1/13 Michael Maymann > >> Hi List, >> >> I'm new to rsyslog/syslog in general. >> >> I would like to syslog from all my 100+ network devices. >> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >> containing logs from this host if more logfiles per host are best >> practice)... >> >> Can anyone give me an example of (or link to) best practice of this kind >> of setup. >> >> >> Thanks in advance :-) ! >> >> ~maymann >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From david at lang.hm Fri Jan 13 21:44:13 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Jan 2012 12:44:13 -0800 (PST) Subject: [rsyslog] update on rsyslog licensing and funding In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> Message-ID: If the licensign text for rsyslog clearly states that the authors of rsyslog do not consider plugins to be defivitive works of rsyslog, that basically makes it so that even if someone tries to claim that the GPL did apply to the plugin, anyone could point to that statement and any judge would dismiss the case (I think the legal term is enstopple, basically you can't give people permission to do something and then sue them for doing it) some people call this an 'exception', but usually the people granting the 'exception' consider it common sense. with linux you have the 'userspace exception' that says that software running in userspace and using the standard system calls is not considered a derived work of the linux kernel. Most people consider this an obvious truth that doesn't really matter, but the fact that it has been clearly stated for a long time means that nobody can now claim otherwise. I belive that you have had a similar statement in place about the plugins for a long time, so it's not that it takes creative thinking to make plugins not be required to be GPL licensed, but it takes creative thinking to justify ignoring that statement and claiming that they are required to be under the GPL. David Lang On Fri, 13 Jan 2012, Rainer Gerhards wrote: > Date: Fri, 13 Jan 2012 18:38:40 +0100 > From: Rainer Gerhards > Reply-To: rsyslog-users > To: rsyslog-users > Subject: Re: [rsyslog] update on rsyslog licensing and funding > > Hi all, > > I created some confusion if the rsyslog project stays under GPLv3 with > yesterday's post. In fact, it does! I wrote a new posting today in the hope > to clarify some things: > > http://blog.gerhards.net/2012/01/rsyslog-will-remain-gplv3-licensed.html > > Rainer > >> -----Original Message----- >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards >> Sent: Thursday, January 12, 2012 3:15 PM >> To: rsyslog at lists.adiscon.com >> Subject: [rsyslog] update on rsyslog licensing and funding >> >> Hi all, >> >> we had a lengthy discussion last November on rsyslog funding and >> licensing. >> At that time, I promised to keep you updated. I have blogged about the >> latest >> state: >> >> http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html >> >> Feedback is appreciated. >> >> Rainer >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From gk at pivotalsecurity.com Fri Jan 13 22:34:06 2012 From: gk at pivotalsecurity.com (Gaurav Kumar) Date: Fri, 13 Jan 2012 16:34:06 -0500 Subject: [rsyslog] udp forwarding performance In-Reply-To: <4F0FEC0A.9080204@ipax.at> References: <4F0FEC0A.9080204@ipax.at> Message-ID: Hi Raoul, I don't know which exact config resolved the issue. I followed general guidance on tuning TCP/IP stack - http://www.psc.edu/networking/projects/tcptune/historical.php Cheers, Gaurav On Fri, Jan 13, 2012 at 3:32 AM, Raoul Bhatia [IPAX] wrote: > On 2012-01-13 04:36, Gaurav Kumar wrote: > >> I was finally able to resolve the issue. The root cause was improper >> tcp/ip >> config on reciver host. Thanks all for the hints. >> > > hi gaurav! > > could you please elaborate on the settings you had to tune/ > which settings you are now using? > > thanks, > raoul > -- > ______________________________**______________________________**________ > DI (FH) Raoul Bhatia M.Sc. email. r.bhatia at ipax.at > Technischer Leiter > > IPAX - Aloy Bhatia Hava OG web. http://www.ipax.at > Barawitzkagasse 10/2/2/11 email. office at ipax.at > 1190 Wien tel. +43 1 3670030 > FN 277995t HG Wien fax. +43 1 3670030 15 > ______________________________**______________________________**________ > From david at lang.hm Sat Jan 14 07:46:55 2012 From: david at lang.hm (david at lang.hm) Date: Fri, 13 Jan 2012 22:46:55 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: http://rsyslog.com/article60/ David Lang On Sat, 14 Jan 2012, Michael Maymann wrote: > Date: Sat, 14 Jan 2012 07:23:57 +0100 > From: Michael Maymann > To: rsyslog-users , david at lang.hm, > Michael Maymann > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi David, > > thanks for you kind reply...:-) ! > --- > This didn't seem to get through to the archives for some reason...: > http://lists.adiscon.net/pipermail/rsyslog/2012-January/thread.html > Hope I will not dobbel-post... > --- > I don't use syslog-relays, so this will not cause me any problems. > Don't actually know what version we are running - can see this Monday > morning though... Thanks for this hint... will upgrade to 6.2 if not > already then. > I have to configure this into a already running live production system - > our previous syslog-admin left...:-(. > Could I perhaps ask you to be so kind as to give an configuration example > of how this is done, if I ask really nicely... :-) ? > > Thanks in advance :-) ! > ~maymann > > 2012/1/13 > >> you need to be aware that doing the DNS queries is rather expensive >> (although I think I saw a comment that in the very latest 6.2 version there >> may now be a DNS cache that will drastically help) >> >> you would need to create a template with FROMHOST in it and use that as >> the filename to write to (look for dynafile in the documentation) >> >> note that if you are relaying logs from one machine to another, only the >> first machine will see the true source in FROMHOST, machines after that >> will only see the relay box. >> >> let me know if this doesn't give you enough clues to learn how to do this. >> >> David Lang >> >> On Fri, 13 Jan 2012, Michael Maymann wrote: >> >> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> From: Michael Maymann >>> Reply-To: rsyslog-users >>> To: rsyslog at lists.adiscon.com >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >>> >>> >>> Furthermore: would it be possible to validate FQDN from DNS and not from >>> syslog-info hostname. >>> We are getting a lot of weird logfiles as some applications are not >>> including the hostname as the first parameter in the syslog-entries, e.g.: >>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >>> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >>> >>> Would be nice to validate FQDN from sender DNS query... >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> >>> 2012/1/13 Michael Maymann >>> >>> Hi List, >>>> >>>> I'm new to rsyslog/syslog in general. >>>> >>>> I would like to syslog from all my 100+ network devices. >>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>> containing logs from this host if more logfiles per host are best >>>> practice)... >>>> >>>> Can anyone give me an example of (or link to) best practice of this kind >>>> of setup. >>>> >>>> >>>> Thanks in advance :-) ! >>>> >>>> ~maymann >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>> http://www.rsyslog.com/**professional-services/ >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> > From david at lang.hm Sat Jan 14 09:13:58 2012 From: david at lang.hm (david at lang.hm) Date: Sat, 14 Jan 2012 00:13:58 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: On Sat, 14 Jan 2012, Michael Maymann wrote: > Hi David, > > thanks for this...this is super info...:-) ! > If I have to create different logs per host, will this be the a valid > configuration: > $template DynaFile_messages,?/logfiles_on_nfs/%HOSTNAME%/messages? > *.* -?DynaFile_messages > $template DynaFile_secure,?/logfiles_on_nfs/%HOSTNAME%/secure? > *.* -?DynaFile_secure > $template DynaFile_auth.log,?/logfiles_on_nfs/%HOSTNAME%/auth.log? > *.* -?DynaFile_auth.log I believe so. > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to > create every hosts dir upfront... ? it will create it for you (make sure it's running with the appropriate permissions, if you have rsyslog configured to drop privileges, the lower privileges need the ability to create the directories) > 2. Is DNS caching default enabled or do I have to enable this somewhere > first...? I don't know, I haven't had a chance to look into that yet. David Lang > > Thanks in advance :-) ! > ~maymann > > > 2012/1/14 > >> http://rsyslog.com/article60/ >> >> David Lang >> >> On Sat, 14 Jan 2012, Michael Maymann wrote: >> >> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> From: Michael Maymann >>> To: rsyslog-users , david at lang.hm, >>> Michael Maymann >>> >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >>> >>> Hi David, >>> >>> thanks for you kind reply...:-) ! >>> --- >>> This didn't seem to get through to the archives for some reason...: >>> http://lists.adiscon.net/**pipermail/rsyslog/2012-**January/thread.html >>> Hope I will not dobbel-post... >>> --- >>> I don't use syslog-relays, so this will not cause me any problems. >>> Don't actually know what version we are running - can see this Monday >>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>> already then. >>> I have to configure this into a already running live production system - >>> our previous syslog-admin left...:-(. >>> Could I perhaps ask you to be so kind as to give an configuration example >>> of how this is done, if I ask really nicely... :-) ? >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> 2012/1/13 >>> >>> you need to be aware that doing the DNS queries is rather expensive >>>> (although I think I saw a comment that in the very latest 6.2 version >>>> there >>>> may now be a DNS cache that will drastically help) >>>> >>>> you would need to create a template with FROMHOST in it and use that as >>>> the filename to write to (look for dynafile in the documentation) >>>> >>>> note that if you are relaying logs from one machine to another, only the >>>> first machine will see the true source in FROMHOST, machines after that >>>> will only see the relay box. >>>> >>>> let me know if this doesn't give you enough clues to learn how to do >>>> this. >>>> >>>> David Lang >>>> >>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>> >>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>> >>>>> From: Michael Maymann >>>>> Reply-To: rsyslog-users >>>>> To: rsyslog at lists.adiscon.com >>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>> howto/links/examples >>>>> >>>>> >>>>> Furthermore: would it be possible to validate FQDN from DNS and not from >>>>> syslog-info hostname. >>>>> We are getting a lot of weird logfiles as some applications are not >>>>> including the hostname as the first parameter in the syslog-entries, >>>>> e.g.: >>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>> HostCompatList >>>>> text /tmp/hostCompatList"'#012#****01212/16/11 09:47:40 >>>>> >>>>> >>>>> Would be nice to validate FQDN from sender DNS query... >>>>> >>>>> Thanks in advance :-) ! >>>>> ~maymann >>>>> >>>>> >>>>> 2012/1/13 Michael Maymann >>>>> >>>>> Hi List, >>>>> >>>>>> >>>>>> I'm new to rsyslog/syslog in general. >>>>>> >>>>>> I would like to syslog from all my 100+ network devices. >>>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>>> containing logs from this host if more logfiles per host are best >>>>>> practice)... >>>>>> >>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>> kind >>>>>> of setup. >>>>>> >>>>>> >>>>>> Thanks in advance :-) ! >>>>>> >>>>>> ~maymann >>>>>> >>>>>> ______________________________****_________________ >>>>>> >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>>> >>>>>> >>>>> http://www.rsyslog.com/****professional-services/ >>>>> >>>>>> >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>> >>>>> >>>> http://www.rsyslog.com/****professional-services/ >>>> >>>>> >>>> >>>> >>> > From michael at maymann.org Sat Jan 14 00:01:53 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 00:01:53 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: This didn't seem to get through to the archives or some reason...: http://lists.adiscon.net/pipermail/rsyslog/2012-January/thread.html Hope I will not dobbel-post... ~maymann 2012/1/13 Michael Maymann > Hi David, > > thanks for you kind reply...:-) ! > > I don't use syslog-relays, so this will not cause me any problems. > Don't actually know what version we are running - can see this Monday > morning though... Thanks for this hint... will upgrade to 6.2 if not > already then. > I have to configure this into a already running live production system - > our previous syslog-admin left...:-(. > Could I perhaps ask you to be so kind as to give an configuration example > of how this is done, if I ask really nicely... :-) ? > > > Thanks in advance :-) ! > ~maymann > > > 2012/1/13 > > you need to be aware that doing the DNS queries is rather expensive >> (although I think I saw a comment that in the very latest 6.2 version there >> may now be a DNS cache that will drastically help) >> >> you would need to create a template with FROMHOST in it and use that as >> the filename to write to (look for dynafile in the documentation) >> >> note that if you are relaying logs from one machine to another, only the >> first machine will see the true source in FROMHOST, machines after that >> will only see the relay box. >> >> let me know if this doesn't give you enough clues to learn how to do this. >> >> David Lang >> >> On Fri, 13 Jan 2012, Michael Maymann wrote: >> >> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> From: Michael Maymann >>> Reply-To: rsyslog-users >>> To: rsyslog at lists.adiscon.com >>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> howto/links/examples >>> >>> >>> Furthermore: would it be possible to validate FQDN from DNS and not from >>> syslog-info hostname. >>> We are getting a lot of weird logfiles as some applications are not >>> including the hostname as the first parameter in the syslog-entries, >>> e.g.: >>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>> HostCompatList >>> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >>> >>> Would be nice to validate FQDN from sender DNS query... >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> >>> 2012/1/13 Michael Maymann >>> >>> Hi List, >>>> >>>> I'm new to rsyslog/syslog in general. >>>> >>>> I would like to syslog from all my 100+ network devices. >>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>> containing logs from this host if more logfiles per host are best >>>> practice)... >>>> >>>> Can anyone give me an example of (or link to) best practice of this kind >>>> of setup. >>>> >>>> >>>> Thanks in advance :-) ! >>>> >>>> ~maymann >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/**mailman/listinfo/rsyslog >>> http://www.rsyslog.com/**professional-services/ >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> > > From michael at maymann.org Sat Jan 14 07:20:45 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 07:20:45 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for you kind reply...:-) ! I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to 6.2 if not already then. I have to configure this into a already running live production system - our previous syslog-admin left...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann 2012/1/13 > you need to be aware that doing the DNS queries is rather expensive > (although I think I saw a comment that in the very latest 6.2 version there > may now be a DNS cache that will drastically help) > > you would need to create a template with FROMHOST in it and use that as > the filename to write to (look for dynafile in the documentation) > > note that if you are relaying logs from one machine to another, only the > first machine will see the true source in FROMHOST, machines after that > will only see the relay box. > > let me know if this doesn't give you enough clues to learn how to do this. > > David Lang > > On Fri, 13 Jan 2012, Michael Maymann wrote: > > Date: Fri, 13 Jan 2012 14:43:06 +0100 >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> >> Furthermore: would it be possible to validate FQDN from DNS and not from >> syslog-info hostname. >> We are getting a lot of weird logfiles as some applications are not >> including the hostname as the first parameter in the syslog-entries, e.g.: >> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >> >> Would be nice to validate FQDN from sender DNS query... >> >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/13 Michael Maymann >> >> Hi List, >>> >>> I'm new to rsyslog/syslog in general. >>> >>> I would like to syslog from all my 100+ network devices. >>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>> containing logs from this host if more logfiles per host are best >>> practice)... >>> >>> Can anyone give me an example of (or link to) best practice of this kind >>> of setup. >>> >>> >>> Thanks in advance :-) ! >>> >>> ~maymann >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From rgerhards at hq.adiscon.com Sat Jan 14 11:30:40 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Sat, 14 Jan 2012 11:30:40 +0100 Subject: [rsyslog] update on rsyslog licensing and funding In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281722@GRFEXC.intern.adiscon.com><9B6E2A8877C38245BFB15CC491A11DA7281741@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281742@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of david at lang.hm > Sent: Friday, January 13, 2012 9:44 PM > To: rsyslog-users > Subject: Re: [rsyslog] update on rsyslog licensing and funding > > If the licensign text for rsyslog clearly states that the authors of > rsyslog do not consider plugins to be defivitive works of rsyslog, that > basically makes it so that even if someone tries to claim that the GPL > did > apply to the plugin, anyone could point to that statement and any judge > would dismiss the case (I think the legal term is enstopple, basically > you > can't give people permission to do something and then sue them for > doing > it) > > some people call this an 'exception', but usually the people granting > the > 'exception' consider it common sense. > > with linux you have the 'userspace exception' that says that software > running in userspace and using the standard system calls is not > considered > a derived work of the linux kernel. Most people consider this an > obvious > truth that doesn't really matter, but the fact that it has been clearly > stated for a long time means that nobody can now claim otherwise. > > I belive that you have had a similar statement in place about the > plugins > for a long time, so it's not that it takes creative thinking to make > plugins not be required to be GPL licensed, but it takes creative > thinking > to justify ignoring that statement and claiming that they are required > to > be under the GPL. I have searched the various documents (the past days and now again). While they express that we consider plugins as separate projects and don't care about their licensing, the exact text is somewhat weak. I agree, at least under German law, that this probably comes close enough to an exception. On the other hand, the ultimate answer can only be found in court (at least under German law, where a judge would need to interpret the statements). This level of uncertainty, how small it may be, drives some potential users away. We have actually seen this happen and it was a big loss for the project. With the licensing change, we try to solve, or at least reduce that uncertainty. That's also why I wrote that I think the situation does not change so much over how it is right now. Rainer > David Lang > > On Fri, 13 Jan 2012, Rainer Gerhards wrote: > > > Date: Fri, 13 Jan 2012 18:38:40 +0100 > > From: Rainer Gerhards > > Reply-To: rsyslog-users > > To: rsyslog-users > > Subject: Re: [rsyslog] update on rsyslog licensing and funding > > > > Hi all, > > > > I created some confusion if the rsyslog project stays under GPLv3 > with > > yesterday's post. In fact, it does! I wrote a new posting today in > the hope > > to clarify some things: > > > > http://blog.gerhards.net/2012/01/rsyslog-will-remain-gplv3- > licensed.html > > > > Rainer > > > >> -----Original Message----- > >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > >> Sent: Thursday, January 12, 2012 3:15 PM > >> To: rsyslog at lists.adiscon.com > >> Subject: [rsyslog] update on rsyslog licensing and funding > >> > >> Hi all, > >> > >> we had a lengthy discussion last November on rsyslog funding and > >> licensing. > >> At that time, I promised to keep you updated. I have blogged about > the > >> latest > >> state: > >> > >> http://blog.gerhards.net/2012/01/rsyslog-licensing-update.html > >> > >> Feedback is appreciated. > >> > >> Rainer > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Sat Jan 14 08:50:06 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 08:50:06 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for this...this is super info...:-) ! If I have to create different logs per host, will this be the a valid configuration: $template DynaFile_messages,?/logfiles_on_nfs/%HOSTNAME%/messages? *.* -?DynaFile_messages $template DynaFile_secure,?/logfiles_on_nfs/%HOSTNAME%/secure? *.* -?DynaFile_secure $template DynaFile_auth.log,?/logfiles_on_nfs/%HOSTNAME%/auth.log? *.* -?DynaFile_auth.log 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to create every hosts dir upfront... ? 2. Is DNS caching default enabled or do I have to enable this somewhere first...? Thanks in advance :-) ! ~maymann 2012/1/14 > http://rsyslog.com/article60/ > > David Lang > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > Date: Sat, 14 Jan 2012 07:23:57 +0100 >> From: Michael Maymann >> To: rsyslog-users , david at lang.hm, >> Michael Maymann >> >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> Hi David, >> >> thanks for you kind reply...:-) ! >> --- >> This didn't seem to get through to the archives for some reason...: >> http://lists.adiscon.net/**pipermail/rsyslog/2012-**January/thread.html >> Hope I will not dobbel-post... >> --- >> I don't use syslog-relays, so this will not cause me any problems. >> Don't actually know what version we are running - can see this Monday >> morning though... Thanks for this hint... will upgrade to 6.2 if not >> already then. >> I have to configure this into a already running live production system - >> our previous syslog-admin left...:-(. >> Could I perhaps ask you to be so kind as to give an configuration example >> of how this is done, if I ask really nicely... :-) ? >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/13 >> >> you need to be aware that doing the DNS queries is rather expensive >>> (although I think I saw a comment that in the very latest 6.2 version >>> there >>> may now be a DNS cache that will drastically help) >>> >>> you would need to create a template with FROMHOST in it and use that as >>> the filename to write to (look for dynafile in the documentation) >>> >>> note that if you are relaying logs from one machine to another, only the >>> first machine will see the true source in FROMHOST, machines after that >>> will only see the relay box. >>> >>> let me know if this doesn't give you enough clues to learn how to do >>> this. >>> >>> David Lang >>> >>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>> >>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> >>>> From: Michael Maymann >>>> Reply-To: rsyslog-users >>>> To: rsyslog at lists.adiscon.com >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>> howto/links/examples >>>> >>>> >>>> Furthermore: would it be possible to validate FQDN from DNS and not from >>>> syslog-info hostname. >>>> We are getting a lot of weird logfiles as some applications are not >>>> including the hostname as the first parameter in the syslog-entries, >>>> e.g.: >>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>> HostCompatList >>>> text /tmp/hostCompatList"'#012#****01212/16/11 09:47:40 >>>> >>>> >>>> Would be nice to validate FQDN from sender DNS query... >>>> >>>> Thanks in advance :-) ! >>>> ~maymann >>>> >>>> >>>> 2012/1/13 Michael Maymann >>>> >>>> Hi List, >>>> >>>>> >>>>> I'm new to rsyslog/syslog in general. >>>>> >>>>> I would like to syslog from all my 100+ network devices. >>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>> containing logs from this host if more logfiles per host are best >>>>> practice)... >>>>> >>>>> Can anyone give me an example of (or link to) best practice of this >>>>> kind >>>>> of setup. >>>>> >>>>> >>>>> Thanks in advance :-) ! >>>>> >>>>> ~maymann >>>>> >>>>> ______________________________****_________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>>> >>>> > >>>> http://www.rsyslog.com/****professional-services/ >>>> >>>> > >>>> >>>> ______________________________****_________________ >>>> >>> rsyslog mailing list >>> http://lists.adiscon.net/****mailman/listinfo/rsyslog >>> >>> > >>> http://www.rsyslog.com/****professional-services/ >>> >>> > >>> >>> >> From michael at maymann.org Sat Jan 14 07:23:57 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 07:23:57 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for you kind reply...:-) ! --- This didn't seem to get through to the archives for some reason...: http://lists.adiscon.net/pipermail/rsyslog/2012-January/thread.html Hope I will not dobbel-post... --- I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to 6.2 if not already then. I have to configure this into a already running live production system - our previous syslog-admin left...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann 2012/1/13 > you need to be aware that doing the DNS queries is rather expensive > (although I think I saw a comment that in the very latest 6.2 version there > may now be a DNS cache that will drastically help) > > you would need to create a template with FROMHOST in it and use that as > the filename to write to (look for dynafile in the documentation) > > note that if you are relaying logs from one machine to another, only the > first machine will see the true source in FROMHOST, machines after that > will only see the relay box. > > let me know if this doesn't give you enough clues to learn how to do this. > > David Lang > > On Fri, 13 Jan 2012, Michael Maymann wrote: > > Date: Fri, 13 Jan 2012 14:43:06 +0100 >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> >> Furthermore: would it be possible to validate FQDN from DNS and not from >> syslog-info hostname. >> We are getting a lot of weird logfiles as some applications are not >> including the hostname as the first parameter in the syslog-entries, e.g.: >> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >> >> Would be nice to validate FQDN from sender DNS query... >> >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/13 Michael Maymann >> >> Hi List, >>> >>> I'm new to rsyslog/syslog in general. >>> >>> I would like to syslog from all my 100+ network devices. >>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>> containing logs from this host if more logfiles per host are best >>> practice)... >>> >>> Can anyone give me an example of (or link to) best practice of this kind >>> of setup. >>> >>> >>> Thanks in advance :-) ! >>> >>> ~maymann >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From michael at maymann.org Sat Jan 14 07:07:14 2012 From: michael at maymann.org (Michael Maymann) Date: Sat, 14 Jan 2012 07:07:14 +0100 Subject: [rsyslog] Syslogging FQDN logfile,logdir howto,links,examples Message-ID: Hi David, thanks for you kind reply...:-) ! Don't know why this is not getting through when i reply-all...? I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to latest 6.2 stable if not already then. I have to configure this into a already running live production system - our previous syslog-admin left for other tasks...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann From michael at maymann.org Fri Jan 13 21:45:14 2012 From: michael at maymann.org (Michael Maymann) Date: Fri, 13 Jan 2012 21:45:14 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi David, thanks for you kind reply...:-) ! I don't use syslog-relays, so this will not cause me any problems. Don't actually know what version we are running - can see this Monday morning though... Thanks for this hint... will upgrade to 6.2 if not already then. I have to configure this into a already running live production system - our previous syslog-admin left...:-(. Could I perhaps ask you to be so kind as to give an configuration example of how this is done, if I ask really nicely... :-) ? Thanks in advance :-) ! ~maymann 2012/1/13 > you need to be aware that doing the DNS queries is rather expensive > (although I think I saw a comment that in the very latest 6.2 version there > may now be a DNS cache that will drastically help) > > you would need to create a template with FROMHOST in it and use that as > the filename to write to (look for dynafile in the documentation) > > note that if you are relaying logs from one machine to another, only the > first machine will see the true source in FROMHOST, machines after that > will only see the relay box. > > let me know if this doesn't give you enough clues to learn how to do this. > > David Lang > > On Fri, 13 Jan 2012, Michael Maymann wrote: > > Date: Fri, 13 Jan 2012 14:43:06 +0100 >> From: Michael Maymann >> Reply-To: rsyslog-users >> To: rsyslog at lists.adiscon.com >> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples >> >> >> Furthermore: would it be possible to validate FQDN from DNS and not from >> syslog-info hostname. >> We are getting a lot of weird logfiles as some applications are not >> including the hostname as the first parameter in the syslog-entries, e.g.: >> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> [issue_cmd ] #012#01212/16/11 09:47:10 [set_host_compat_list] >> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData HostCompatList >> text /tmp/hostCompatList"'#012#**01212/16/11 09:47:40 >> >> Would be nice to validate FQDN from sender DNS query... >> >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/13 Michael Maymann >> >> Hi List, >>> >>> I'm new to rsyslog/syslog in general. >>> >>> I would like to syslog from all my 100+ network devices. >>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>> containing logs from this host if more logfiles per host are best >>> practice)... >>> >>> Can anyone give me an example of (or link to) best practice of this kind >>> of setup. >>> >>> >>> Thanks in advance :-) ! >>> >>> ~maymann >>> >>> ______________________________**_________________ >> rsyslog mailing list >> http://lists.adiscon.net/**mailman/listinfo/rsyslog >> http://www.rsyslog.com/**professional-services/ >> >> ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog > http://www.rsyslog.com/**professional-services/ > From rodney.mckee at gmail.com Sun Jan 15 20:55:46 2012 From: rodney.mckee at gmail.com (Rodney McKee) Date: Mon, 16 Jan 2012 06:55:46 +1100 (EST) Subject: [rsyslog] dropping messages In-Reply-To: <287e447b-fd75-4f02-97b9-d7e2caf30606@wsrmckee> Message-ID: <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> Hello, Is their a way to match the text in a message and drop it based on the content? Rgds Rodney From mbiebl at gmail.com Sun Jan 15 21:07:46 2012 From: mbiebl at gmail.com (Michael Biebl) Date: Sun, 15 Jan 2012 21:07:46 +0100 Subject: [rsyslog] dropping messages In-Reply-To: <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> References: <287e447b-fd75-4f02-97b9-d7e2caf30606@wsrmckee> <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> Message-ID: Am 15. Januar 2012 20:55 schrieb Rodney McKee : > Hello, > > Is their a way to match the text in a message and drop it based on the content? Google for "rsyslog filter by message", click on the first hit. -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? From rodney.mckee at gmail.com Sun Jan 15 21:39:57 2012 From: rodney.mckee at gmail.com (Rodney McKee) Date: Mon, 16 Jan 2012 07:39:57 +1100 Subject: [rsyslog] dropping messages In-Reply-To: References: <287e447b-fd75-4f02-97b9-d7e2caf30606@wsrmckee> <1075dee8-0f42-4d7d-9d0e-e71df0fb0fcf@wsrmckee> Message-ID: That looks good, cheers On 16/01/2012 7:07 AM, "Michael Biebl" wrote: > Am 15. Januar 2012 20:55 schrieb Rodney McKee : > > Hello, > > > > Is their a way to match the text in a message and drop it based on the > content? > > Google for "rsyslog filter by message", click on the first hit. > > > -- > Why is it that all of the instruments seeking intelligent life in the > universe are pointed away from Earth? > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From penght at cn.fujitsu.com Mon Jan 16 06:32:59 2012 From: penght at cn.fujitsu.com (Peng Haitao) Date: Mon, 16 Jan 2012 13:32:59 +0800 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID In-Reply-To: <4F069932.9070408@cn.fujitsu.com> References: <4F069932.9070408@cn.fujitsu.com> Message-ID: <4F13B68B.5070203@cn.fujitsu.com> Peng Haitao said the following on 2012-1-6 14:48: > [add list to the CC list] > > When HAVE_SETSID is not defined, rsyslogd will use ioctl() make itself to daemon, > but this can not make rsyslogd process become the process group leader of a new > process group. In RHEL6.1, the status is as follows: > # uname -a > Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux > # /etc/init.d/rsyslog restart > Shutting down system logger: [ OK ] > Starting system logger: [ OK ] > # ps axo pgrp,ppid,pid,comm | grep rsyslog > 6290 1 6301 rsyslogd > > When we send SIGTERM signal to 6290, rsyslogd will die:( So I think we should > call setpgid() before ioctl(). > hi, anyone can comment the patch? -- Best Regards, Peng > Signed-off-by: Peng Haitao > --- > tools/syslogd.c | 7 +++++++ > 1 files changed, 7 insertions(+), 0 deletions(-) > > diff --git a/tools/syslogd.c b/tools/syslogd.c > index d1224f2..ac954b7 100644 > --- a/tools/syslogd.c > +++ b/tools/syslogd.c > @@ -356,8 +356,15 @@ void untty(void) > #else > { > int i; > + pid_t pid; > > if(!Debug) { > + pid = getpid(); > + if (setpgid(pid, pid) < 0) { > + perror("setpgid"); > + exit(1); > + } > + > i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); > if (i >= 0) { > # if !defined(__hpux) From michael at maymann.org Mon Jan 16 08:57:26 2012 From: michael at maymann.org (Michael Maymann) Date: Mon, 16 Jan 2012 08:57:26 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: If I want DNS caching, should i use the new stable-6.2.0 or the older devel-6.3.6... ? Can see this feature mentioned here: http://rsyslog.com/features/ or here: http://rsyslog.com/project-status/ Thanks in advance :-) ! ~maymann 2012/1/14 > On Sat, 14 Jan 2012, Michael Maymann wrote: > > Hi David, >> >> thanks for this...this is super info...:-) ! >> If I have to create different logs per host, will this be the a valid >> configuration: >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >> *.* -?DynaFile_messages >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >> *.* -?DynaFile_secure >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >> *.* -?DynaFile_auth.log >> > > I believe so. > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to >> create every hosts dir upfront... ? >> > > it will create it for you (make sure it's running with the appropriate > permissions, if you have rsyslog configured to drop privileges, the lower > privileges need the ability to create the directories) > > > 2. Is DNS caching default enabled or do I have to enable this somewhere >> first...? >> > > I don't know, I haven't had a chance to look into that yet. > > David Lang > > >> Thanks in advance :-) ! >> ~maymann >> >> >> 2012/1/14 >> >> http://rsyslog.com/article60/ >>> >>> David Lang >>> >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> >>>> From: Michael Maymann >>>> To: rsyslog-users , david at lang.hm, >>>> Michael Maymann >>>> >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>> howto/links/examples >>>> >>>> Hi David, >>>> >>>> thanks for you kind reply...:-) ! >>>> --- >>>> This didn't seem to get through to the archives for some reason...: >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>>> January/thread.html >>>> >>> html >>>> > >>>> >>>> Hope I will not dobbel-post... >>>> --- >>>> I don't use syslog-relays, so this will not cause me any problems. >>>> Don't actually know what version we are running - can see this Monday >>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>> already then. >>>> I have to configure this into a already running live production system - >>>> our previous syslog-admin left...:-(. >>>> Could I perhaps ask you to be so kind as to give an configuration >>>> example >>>> of how this is done, if I ask really nicely... :-) ? >>>> >>>> Thanks in advance :-) ! >>>> ~maymann >>>> >>>> 2012/1/13 >>>> >>>> you need to be aware that doing the DNS queries is rather expensive >>>> >>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>> there >>>>> may now be a DNS cache that will drastically help) >>>>> >>>>> you would need to create a template with FROMHOST in it and use that as >>>>> the filename to write to (look for dynafile in the documentation) >>>>> >>>>> note that if you are relaying logs from one machine to another, only >>>>> the >>>>> first machine will see the true source in FROMHOST, machines after that >>>>> will only see the relay box. >>>>> >>>>> let me know if this doesn't give you enough clues to learn how to do >>>>> this. >>>>> >>>>> David Lang >>>>> >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>> >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>> >>>>> From: Michael Maymann >>>>>> Reply-To: rsyslog-users >>>>>> To: rsyslog at lists.adiscon.com >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>> howto/links/examples >>>>>> >>>>>> >>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>> from >>>>>> syslog-info hostname. >>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>> e.g.: >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>> [set_host_compat_list] >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>> HostCompatList >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>>>>> >>>>>> >>>>>> >>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>> >>>>>> Thanks in advance :-) ! >>>>>> ~maymann >>>>>> >>>>>> >>>>>> 2012/1/13 Michael Maymann >>>>>> >>>>>> Hi List, >>>>>> >>>>>> >>>>>>> I'm new to rsyslog/syslog in general. >>>>>>> >>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>>>> containing logs from this host if more logfiles per host are best >>>>>>> practice)... >>>>>>> >>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>> kind >>>>>>> of setup. >>>>>>> >>>>>>> >>>>>>> Thanks in advance :-) ! >>>>>>> >>>>>>> ~maymann >>>>>>> >>>>>>> ______________________________******_________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>>> >>>>>> > >>>>>> >>>>>> >>>>>> > >>>>>> >>>>>>> >>>>>>> http://www.rsyslog.com/******professional-services/ >>>>>> >>>>>> > >>>>>> >>>>>> >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>> ______________________________******_________________ >>>>>> >>>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>> >>>>> > >>>>> >>>>> >>>>> > >>>>> >>>>>> >>>>>> http://www.rsyslog.com/******professional-services/ >>>>> >>>>> > >>>>> >>>>> >>>>> > >>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >> From rgerhards at hq.adiscon.com Mon Jan 16 10:33:42 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 10:33:42 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> The cache is available since 6.3.1, so you need to go for the devel version. A good place to check those things is the ChangeLog itself, here is the current one: http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8575d085 a0fcf48f71339154813971;hb=HEAD Note that v6-devel is almost as stable as v6-stable except for the config read phase at startup. HTH Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > Sent: Monday, January 16, 2012 8:57 AM > To: david at lang.hm > Cc: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > howto/links/examples > > If I want DNS caching, should i use the new stable-6.2.0 or the older > devel-6.3.6... ? > Can see this feature mentioned here: > http://rsyslog.com/features/ > or here: > http://rsyslog.com/project-status/ > > > Thanks in advance :-) ! > ~maymann > > 2012/1/14 > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > Hi David, > >> > >> thanks for this...this is super info...:-) ! > >> If I have to create different logs per host, will this be the a > valid > >> configuration: > >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > >> *.* -?DynaFile_messages > >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > >> *.* -?DynaFile_secure > >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > >> *.* -?DynaFile_auth.log > >> > > > > I believe so. > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > have to > >> create every hosts dir upfront... ? > >> > > > > it will create it for you (make sure it's running with the > appropriate > > permissions, if you have rsyslog configured to drop privileges, the > lower > > privileges need the ability to create the directories) > > > > > > 2. Is DNS caching default enabled or do I have to enable this > somewhere > >> first...? > >> > > > > I don't know, I haven't had a chance to look into that yet. > > > > David Lang > > > > > >> Thanks in advance :-) ! > >> ~maymann > >> > >> > >> 2012/1/14 > >> > >> http://rsyslog.com/article60/ > >>> > >>> David Lang > >>> > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > >>> > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > >>> > >>>> From: Michael Maymann > >>>> To: rsyslog-users , david at lang.hm, > >>>> Michael Maymann > >>>> > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>>> howto/links/examples > >>>> > >>>> Hi David, > >>>> > >>>> thanks for you kind reply...:-) ! > >>>> --- > >>>> This didn't seem to get through to the archives for some > reason...: > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > >>>> > January/thread.html **January/thread.html> > >>>> January/thread.** > >>>> html January/thread.html> > >>>> > > >>>> > >>>> Hope I will not dobbel-post... > >>>> --- > >>>> I don't use syslog-relays, so this will not cause me any problems. > >>>> Don't actually know what version we are running - can see this > Monday > >>>> morning though... Thanks for this hint... will upgrade to 6.2 if > not > >>>> already then. > >>>> I have to configure this into a already running live production > system - > >>>> our previous syslog-admin left...:-(. > >>>> Could I perhaps ask you to be so kind as to give an configuration > >>>> example > >>>> of how this is done, if I ask really nicely... :-) ? > >>>> > >>>> Thanks in advance :-) ! > >>>> ~maymann > >>>> > >>>> 2012/1/13 > >>>> > >>>> you need to be aware that doing the DNS queries is rather > expensive > >>>> > >>>>> (although I think I saw a comment that in the very latest 6.2 > version > >>>>> there > >>>>> may now be a DNS cache that will drastically help) > >>>>> > >>>>> you would need to create a template with FROMHOST in it and use > that as > >>>>> the filename to write to (look for dynafile in the documentation) > >>>>> > >>>>> note that if you are relaying logs from one machine to another, > only > >>>>> the > >>>>> first machine will see the true source in FROMHOST, machines > after that > >>>>> will only see the relay box. > >>>>> > >>>>> let me know if this doesn't give you enough clues to learn how to > do > >>>>> this. > >>>>> > >>>>> David Lang > >>>>> > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > >>>>> > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > >>>>> > >>>>> From: Michael Maymann > >>>>>> Reply-To: rsyslog-users > >>>>>> To: rsyslog at lists.adiscon.com > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>>>>> howto/links/examples > >>>>>> > >>>>>> > >>>>>> Furthermore: would it be possible to validate FQDN from DNS and > not > >>>>>> from > >>>>>> syslog-info hostname. > >>>>>> We are getting a lot of weird logfiles as some applications are > not > >>>>>> including the hostname as the first parameter in the syslog- > entries, > >>>>>> e.g.: > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > >>>>>> [set_host_compat_list] > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData > >>>>>> HostCompatList > >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > >>>>>> > >>>>>> > >>>>>> > >>>>>> Would be nice to validate FQDN from sender DNS query... > >>>>>> > >>>>>> Thanks in advance :-) ! > >>>>>> ~maymann > >>>>>> > >>>>>> > >>>>>> 2012/1/13 Michael Maymann > >>>>>> > >>>>>> Hi List, > >>>>>> > >>>>>> > >>>>>>> I'm new to rsyslog/syslog in general. > >>>>>>> > >>>>>>> I would like to syslog from all my 100+ network devices. > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > FQDN-dir > >>>>>>> containing logs from this host if more logfiles per host are > best > >>>>>>> practice)... > >>>>>>> > >>>>>>> Can anyone give me an example of (or link to) best practice of > this > >>>>>>> kind > >>>>>>> of setup. > >>>>>>> > >>>>>>> > >>>>>>> Thanks in advance :-) ! > >>>>>>> > >>>>>>> ~maymann > >>>>>>> > >>>>>>> ______________________________******_________________ > >>>>>>> > >>>>>>> rsyslog mailing list > >>>>>> > http://lists.adiscon.net/******mailman/listinfo/rsyslog iscon.net/****mailman/listinfo/rsyslog> > >>>>>> > discon.net/**mailman/listinfo/rsyslog> > >>>>>> > > >>>>>> > discon.net/mailman/**listinfo/rsyslog> > >>>>>> > scon.net/mailman/listinfo/rsyslog> > >>>>>> > > >>>>>> > >>>>>>> > >>>>>>> http://www.rsyslog.com/******professional- > services/ > >>>>>> services/ > >>>>>> > > >>>>>> **services/ > >>>>>> services/ > >>>>>> > > >>>>>> > >>>>>>> > >>>>>>> > >>>>>> ______________________________******_________________ > >>>>>> > >>>>>> rsyslog mailing list > >>>>> > http://lists.adiscon.net/******mailman/listinfo/rsyslog iscon.net/****mailman/listinfo/rsyslog> > >>>>> > discon.net/**mailman/listinfo/rsyslog> > >>>>> > > >>>>> > discon.net/mailman/**listinfo/rsyslog> > >>>>> > scon.net/mailman/listinfo/rsyslog> > >>>>> > > >>>>> > >>>>>> > >>>>>> http://www.rsyslog.com/******professional- > services/ > >>>>> services/ > >>>>> > > >>>>> **services/ > >>>>> services/ > >>>>> > > >>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Mon Jan 16 10:47:57 2012 From: michael at maymann.org (Michael Maymann) Date: Mon, 16 Jan 2012 10:47:57 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, Thanks for your reply. How do I install 6.3.6 on RHEL6 easiest ? Thanks in advance :-)! ~maymann 2012/1/16 Rainer Gerhards > The cache is available since 6.3.1, so you need to go for the devel > version. > A good place to check those things is the ChangeLog itself, here is the > current one: > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8575d085 > a0fcf48f71339154813971 > ;hb=HEAD > > Note that v6-devel is almost as stable as v6-stable except for the config > read phase at startup. > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Monday, January 16, 2012 8:57 AM > > To: david at lang.hm > > Cc: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > howto/links/examples > > > > If I want DNS caching, should i use the new stable-6.2.0 or the older > > devel-6.3.6... ? > > Can see this feature mentioned here: > > http://rsyslog.com/features/ > > or here: > > http://rsyslog.com/project-status/ > > > > > > Thanks in advance :-) ! > > ~maymann > > > > 2012/1/14 > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > Hi David, > > >> > > >> thanks for this...this is super info...:-) ! > > >> If I have to create different logs per host, will this be the a > > valid > > >> configuration: > > >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > >> *.* -?DynaFile_messages > > >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > >> *.* -?DynaFile_secure > > >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > >> *.* -?DynaFile_auth.log > > >> > > > > > > I believe so. > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > have to > > >> create every hosts dir upfront... ? > > >> > > > > > > it will create it for you (make sure it's running with the > > appropriate > > > permissions, if you have rsyslog configured to drop privileges, the > > lower > > > privileges need the ability to create the directories) > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > somewhere > > >> first...? > > >> > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > David Lang > > > > > > > > >> Thanks in advance :-) ! > > >> ~maymann > > >> > > >> > > >> 2012/1/14 > > >> > > >> http://rsyslog.com/article60/ > > >>> > > >>> David Lang > > >>> > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > >>> > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > >>> > > >>>> From: Michael Maymann > > >>>> To: rsyslog-users , david at lang.hm, > > >>>> Michael Maymann > > >>>> > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>> howto/links/examples > > >>>> > > >>>> Hi David, > > >>>> > > >>>> thanks for you kind reply...:-) ! > > >>>> --- > > >>>> This didn't seem to get through to the archives for some > > reason...: > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > >>>> > > January/thread.html > **January/thread.html> > > >>>> > January/thread.** > > >>>> html > January/thread.html> > > >>>> > > > >>>> > > >>>> Hope I will not dobbel-post... > > >>>> --- > > >>>> I don't use syslog-relays, so this will not cause me any problems. > > >>>> Don't actually know what version we are running - can see this > > Monday > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 if > > not > > >>>> already then. > > >>>> I have to configure this into a already running live production > > system - > > >>>> our previous syslog-admin left...:-(. > > >>>> Could I perhaps ask you to be so kind as to give an configuration > > >>>> example > > >>>> of how this is done, if I ask really nicely... :-) ? > > >>>> > > >>>> Thanks in advance :-) ! > > >>>> ~maymann > > >>>> > > >>>> 2012/1/13 > > >>>> > > >>>> you need to be aware that doing the DNS queries is rather > > expensive > > >>>> > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > version > > >>>>> there > > >>>>> may now be a DNS cache that will drastically help) > > >>>>> > > >>>>> you would need to create a template with FROMHOST in it and use > > that as > > >>>>> the filename to write to (look for dynafile in the documentation) > > >>>>> > > >>>>> note that if you are relaying logs from one machine to another, > > only > > >>>>> the > > >>>>> first machine will see the true source in FROMHOST, machines > > after that > > >>>>> will only see the relay box. > > >>>>> > > >>>>> let me know if this doesn't give you enough clues to learn how to > > do > > >>>>> this. > > >>>>> > > >>>>> David Lang > > >>>>> > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > >>>>> > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > >>>>> > > >>>>> From: Michael Maymann > > >>>>>> Reply-To: rsyslog-users > > >>>>>> To: rsyslog at lists.adiscon.com > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>>>> howto/links/examples > > >>>>>> > > >>>>>> > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS and > > not > > >>>>>> from > > >>>>>> syslog-info hostname. > > >>>>>> We are getting a lot of weird logfiles as some applications are > > not > > >>>>>> including the hostname as the first parameter in the syslog- > > entries, > > >>>>>> e.g.: > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > >>>>>> [set_host_compat_list] > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData > > >>>>>> HostCompatList > > >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > >>>>>> > > >>>>>> Thanks in advance :-) ! > > >>>>>> ~maymann > > >>>>>> > > >>>>>> > > >>>>>> 2012/1/13 Michael Maymann > > >>>>>> > > >>>>>> Hi List, > > >>>>>> > > >>>>>> > > >>>>>>> I'm new to rsyslog/syslog in general. > > >>>>>>> > > >>>>>>> I would like to syslog from all my 100+ network devices. > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > FQDN-dir > > >>>>>>> containing logs from this host if more logfiles per host are > > best > > >>>>>>> practice)... > > >>>>>>> > > >>>>>>> Can anyone give me an example of (or link to) best practice of > > this > > >>>>>>> kind > > >>>>>>> of setup. > > >>>>>>> > > >>>>>>> > > >>>>>>> Thanks in advance :-) ! > > >>>>>>> > > >>>>>>> ~maymann > > >>>>>>> > > >>>>>>> ______________________________******_________________ > > >>>>>>> > > >>>>>>> rsyslog mailing list > > >>>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > **services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> ______________________________******_________________ > > >>>>>> > > >>>>>> rsyslog mailing list > > >>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > **services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From rgerhards at hq.adiscon.com Mon Jan 16 11:10:34 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 11:10:34 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > Sent: Monday, January 16, 2012 10:48 AM > To: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi Rainer, > > Thanks for your reply. > How do I install 6.3.6 on RHEL6 easiest ? I have no specific instructions. Just grab the sources and compile, I'd say ;) Note, however, that you need to install libestr and probably libee first. Rainer > > Thanks in advance :-)! > ~maymann > > 2012/1/16 Rainer Gerhards > > > The cache is available since 6.3.1, so you need to go for the devel > > version. > > A good place to check those things is the ChangeLog itself, here is > > the current one: > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 > > 575d085 > > a0fcf48f71339154813971 > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> > > ;hb=HEAD > > > > Note that v6-devel is almost as stable as v6-stable except for the > > config read phase at startup. > > > > HTH > > Rainer > > > > > -----Original Message----- > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > > Sent: Monday, January 16, 2012 8:57 AM > > > To: david at lang.hm > > > Cc: rsyslog-users > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > howto/links/examples > > > > > > If I want DNS caching, should i use the new stable-6.2.0 or the > > > older devel-6.3.6... ? > > > Can see this feature mentioned here: > > > http://rsyslog.com/features/ > > > or here: > > > http://rsyslog.com/project-status/ > > > > > > > > > Thanks in advance :-) ! > > > ~maymann > > > > > > 2012/1/14 > > > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > > > Hi David, > > > >> > > > >> thanks for this...this is super info...:-) ! > > > >> If I have to create different logs per host, will this be the a > > > valid > > > >> configuration: > > > >> $template > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > > >> *.* -?DynaFile_messages > > > >> $template > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > > >> *.* -?DynaFile_secure > > > >> $template > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > > >> *.* -?DynaFile_auth.log > > > >> > > > > > > > > I believe so. > > > > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > > have to > > > >> create every hosts dir upfront... ? > > > >> > > > > > > > > it will create it for you (make sure it's running with the > > > appropriate > > > > permissions, if you have rsyslog configured to drop privileges, > > > > the > > > lower > > > > privileges need the ability to create the directories) > > > > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > > somewhere > > > >> first...? > > > >> > > > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > > > David Lang > > > > > > > > > > > >> Thanks in advance :-) ! > > > >> ~maymann > > > >> > > > >> > > > >> 2012/1/14 > > > >> > > > >> http://rsyslog.com/article60/ > > > >>> > > > >>> David Lang > > > >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > > >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > > >>> > > > >>>> From: Michael Maymann > > > >>>> To: rsyslog-users , david at lang.hm, > > > >>>> Michael Maymann > > > >>>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > >>>> howto/links/examples > > > >>>> > > > >>>> Hi David, > > > >>>> > > > >>>> thanks for you kind reply...:-) ! > > > >>>> --- > > > >>>> This didn't seem to get through to the archives for some > > > reason...: > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > > >>>> > > > January/thread.html > > 2- > > > **January/thread.html> > > > >>>> > > January/thread.** > > > >>>> html > > January/thread.html> > > > >>>> > > > > >>>> > > > >>>> Hope I will not dobbel-post... > > > >>>> --- > > > >>>> I don't use syslog-relays, so this will not cause me any problems. > > > >>>> Don't actually know what version we are running - can see this > > > Monday > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 > > > >>>> if > > > not > > > >>>> already then. > > > >>>> I have to configure this into a already running live production > > > system - > > > >>>> our previous syslog-admin left...:-(. > > > >>>> Could I perhaps ask you to be so kind as to give an > > > >>>> configuration example of how this is done, if I ask really > > > >>>> nicely... :-) ? > > > >>>> > > > >>>> Thanks in advance :-) ! > > > >>>> ~maymann > > > >>>> > > > >>>> 2012/1/13 > > > >>>> > > > >>>> you need to be aware that doing the DNS queries is rather > > > expensive > > > >>>> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > > version > > > >>>>> there > > > >>>>> may now be a DNS cache that will drastically help) > > > >>>>> > > > >>>>> you would need to create a template with FROMHOST in it and > > > >>>>> use > > > that as > > > >>>>> the filename to write to (look for dynafile in the > > > >>>>> documentation) > > > >>>>> > > > >>>>> note that if you are relaying logs from one machine to > > > >>>>> another, > > > only > > > >>>>> the > > > >>>>> first machine will see the true source in FROMHOST, machines > > > after that > > > >>>>> will only see the relay box. > > > >>>>> > > > >>>>> let me know if this doesn't give you enough clues to learn how > > > >>>>> to > > > do > > > >>>>> this. > > > >>>>> > > > >>>>> David Lang > > > >>>>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > > >>>>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > > >>>>> > > > >>>>> From: Michael Maymann > > > >>>>>> Reply-To: rsyslog-users > > > >>>>>> To: rsyslog at lists.adiscon.com > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > >>>>>> howto/links/examples > > > >>>>>> > > > >>>>>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS > > > >>>>>> and > > > not > > > >>>>>> from > > > >>>>>> syslog-info hostname. > > > >>>>>> We are getting a lot of weird logfiles as some applications > > > >>>>>> are > > > not > > > >>>>>> including the hostname as the first parameter in the syslog- > > > entries, > > > >>>>>> e.g.: > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > > >>>>>> [set_host_compat_list] > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd > "SetUserData > > > >>>>>> HostCompatList text > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > > >>>>>> > > > >>>>>> Thanks in advance :-) ! > > > >>>>>> ~maymann > > > >>>>>> > > > >>>>>> > > > >>>>>> 2012/1/13 Michael Maymann > > > >>>>>> > > > >>>>>> Hi List, > > > >>>>>> > > > >>>>>> > > > >>>>>>> I'm new to rsyslog/syslog in general. > > > >>>>>>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > > FQDN-dir > > > >>>>>>> containing logs from this host if more logfiles per host are > > > best > > > >>>>>>> practice)... > > > >>>>>>> > > > >>>>>>> Can anyone give me an example of (or link to) best practice > > > >>>>>>> of > > > this > > > >>>>>>> kind > > > >>>>>>> of setup. > > > >>>>>>> > > > >>>>>>> > > > >>>>>>> Thanks in advance :-) ! > > > >>>>>>> > > > >>>>>>> ~maymann > > > >>>>>>> > > > >>>>>>> > ______________________________******_________________ > > > >>>>>>> > > > >>>>>>> rsyslog mailing list > > > >>>>>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > >>>>>> > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > >>>>>> > > > > >>>>>> > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > >>>>>> > > > > > adi > > > scon.net/mailman/listinfo/rsyslog> > > > >>>>>> > > > > >>>>>> > > > >>>>>>> > > > >>>>>>> http://www.rsyslog.com/******professional- > > > services/ > > > >>>>>> > > services/ > > > >>>>>> > > > > >>>>>> > > **services/ > > > >>>>>> > > services/ > > > >>>>>> > > > > >>>>>> > > > >>>>>>> > > > >>>>>>> > > > >>>>>> > ______________________________******_________________ > > > >>>>>> > > > >>>>>> rsyslog mailing list > > > >>>>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > >>>>> > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > >>>>> > > > > >>>>> > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > >>>>> > > > > > adi > > > scon.net/mailman/listinfo/rsyslog> > > > >>>>> > > > > >>>>> > > > >>>>>> > > > >>>>>> http://www.rsyslog.com/******professional- > > > services/ > > > >>>>> > > services/ > > > >>>>> > > > > >>>>> > > **services/ > > > >>>>> > > services/ > > > >>>>> > > > > >>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> > > > >>>>> > > > >>>> > > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From david at lang.hm Mon Jan 16 11:15:58 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 16 Jan 2012 02:15:58 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: I would say that 6.2.0 is probably better. sorry I can't give you more info on this. David Lang On Mon, 16 Jan 2012, Michael Maymann wrote: > If I want DNS caching, should i use the new stable-6.2.0 or the older > devel-6.3.6... ? > Can see this feature mentioned here: > http://rsyslog.com/features/ > or here: > http://rsyslog.com/project-status/ > > > Thanks in advance :-) ! > ~maymann > > 2012/1/14 > >> On Sat, 14 Jan 2012, Michael Maymann wrote: >> >> Hi David, >>> >>> thanks for this...this is super info...:-) ! >>> If I have to create different logs per host, will this be the a valid >>> configuration: >>> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >>> *.* -?DynaFile_messages >>> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >>> *.* -?DynaFile_secure >>> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >>> *.* -?DynaFile_auth.log >>> >> >> I believe so. >> >> >> 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have to >>> create every hosts dir upfront... ? >>> >> >> it will create it for you (make sure it's running with the appropriate >> permissions, if you have rsyslog configured to drop privileges, the lower >> privileges need the ability to create the directories) >> >> >> 2. Is DNS caching default enabled or do I have to enable this somewhere >>> first...? >>> >> >> I don't know, I haven't had a chance to look into that yet. >> >> David Lang >> >> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> >>> 2012/1/14 >>> >>> http://rsyslog.com/article60/ >>>> >>>> David Lang >>>> >>>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>> >>>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>>> >>>>> From: Michael Maymann >>>>> To: rsyslog-users , david at lang.hm, >>>>> Michael Maymann >>>>> >>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>> howto/links/examples >>>>> >>>>> Hi David, >>>>> >>>>> thanks for you kind reply...:-) ! >>>>> --- >>>>> This didn't seem to get through to the archives for some reason...: >>>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>>>> January/thread.html >>>>> >>>> html >>>>>> >>>>> >>>>> Hope I will not dobbel-post... >>>>> --- >>>>> I don't use syslog-relays, so this will not cause me any problems. >>>>> Don't actually know what version we are running - can see this Monday >>>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>>> already then. >>>>> I have to configure this into a already running live production system - >>>>> our previous syslog-admin left...:-(. >>>>> Could I perhaps ask you to be so kind as to give an configuration >>>>> example >>>>> of how this is done, if I ask really nicely... :-) ? >>>>> >>>>> Thanks in advance :-) ! >>>>> ~maymann >>>>> >>>>> 2012/1/13 >>>>> >>>>> you need to be aware that doing the DNS queries is rather expensive >>>>> >>>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>>> there >>>>>> may now be a DNS cache that will drastically help) >>>>>> >>>>>> you would need to create a template with FROMHOST in it and use that as >>>>>> the filename to write to (look for dynafile in the documentation) >>>>>> >>>>>> note that if you are relaying logs from one machine to another, only >>>>>> the >>>>>> first machine will see the true source in FROMHOST, machines after that >>>>>> will only see the relay box. >>>>>> >>>>>> let me know if this doesn't give you enough clues to learn how to do >>>>>> this. >>>>>> >>>>>> David Lang >>>>>> >>>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>>> >>>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>>> >>>>>> From: Michael Maymann >>>>>>> Reply-To: rsyslog-users >>>>>>> To: rsyslog at lists.adiscon.com >>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>> howto/links/examples >>>>>>> >>>>>>> >>>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>>> from >>>>>>> syslog-info hostname. >>>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>>> e.g.: >>>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>>> [set_host_compat_list] >>>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>>> HostCompatList >>>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>>>>>> >>>>>>> >>>>>>> >>>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>>> >>>>>>> Thanks in advance :-) ! >>>>>>> ~maymann >>>>>>> >>>>>>> >>>>>>> 2012/1/13 Michael Maymann >>>>>>> >>>>>>> Hi List, >>>>>>> >>>>>>> >>>>>>>> I'm new to rsyslog/syslog in general. >>>>>>>> >>>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>>> Preferably I would like a FQDN.log file for each host (or a FQDN-dir >>>>>>>> containing logs from this host if more logfiles per host are best >>>>>>>> practice)... >>>>>>>> >>>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>>> kind >>>>>>>> of setup. >>>>>>>> >>>>>>>> >>>>>>>> Thanks in advance :-) ! >>>>>>>> >>>>>>>> ~maymann >>>>>>>> >>>>>>>> ______________________________******_________________ >>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> http://www.rsyslog.com/******professional-services/ >>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> ______________________________******_________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/******mailman/listinfo/rsyslog >>>>>> >>>>>>> >>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>>> http://www.rsyslog.com/******professional-services/ >>>>>> >>>>>>> >>>>>> >>>>>> >>>>>>> >>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>> > From rgerhards at hq.adiscon.com Mon Jan 16 16:41:43 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 16:41:43 +0100 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID In-Reply-To: <4F13B68B.5070203@cn.fujitsu.com> References: <4F069932.9070408@cn.fujitsu.com> <4F13B68B.5070203@cn.fujitsu.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281752@GRFEXC.intern.adiscon.com> > -----Original Message----- > From: Peng Haitao [mailto:penght at cn.fujitsu.com] > Sent: Monday, January 16, 2012 6:33 AM > To: Rainer Gerhards > Cc: rsyslog-users > Subject: Re: [rsyslog] [PATCH] fix error of not define HAVE_SETSID > > > Peng Haitao said the following on 2012-1-6 14:48: > > [add list to the CC list] > > > > When HAVE_SETSID is not defined, rsyslogd will use ioctl() make itself > > to daemon, but this can not make rsyslogd process become the process > > group leader of a new process group. In RHEL6.1, the status is as follows: > > # uname -a > > Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May > > 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux # /etc/init.d/rsyslog > restart > > Shutting down system logger: [ OK ] > > Starting system logger: [ OK ] > > # ps axo pgrp,ppid,pid,comm | grep rsyslog > > 6290 1 6301 rsyslogd > > > > When we send SIGTERM signal to 6290, rsyslogd will die:( So I think we > > should call setpgid() before ioctl(). > > > > hi, anyone can comment the patch? Sorry, I am still busy, it's on my todo list and looks good on first impression. Bear with me a little bit more, I hope to be able to go back to real work soon... ;) Rainer > > -- > Best Regards, > Peng > > > Signed-off-by: Peng Haitao > > --- > > tools/syslogd.c | 7 +++++++ > > 1 files changed, 7 insertions(+), 0 deletions(-) > > > > diff --git a/tools/syslogd.c b/tools/syslogd.c index d1224f2..ac954b7 > > 100644 > > --- a/tools/syslogd.c > > +++ b/tools/syslogd.c > > @@ -356,8 +356,15 @@ void untty(void) > > #else > > { > > int i; > > + pid_t pid; > > > > if(!Debug) { > > + pid = getpid(); > > + if (setpgid(pid, pid) < 0) { > > + perror("setpgid"); > > + exit(1); > > + } > > + > > i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); > > if (i >= 0) { > > # if !defined(__hpux) > From rgerhards at hq.adiscon.com Mon Jan 16 17:15:08 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Mon, 16 Jan 2012 17:15:08 +0100 Subject: [rsyslog] [PATCH] fix error of not define HAVE_SETSID In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281752@GRFEXC.intern.adiscon.com> References: <4F069932.9070408@cn.fujitsu.com> <4F13B68B.5070203@cn.fujitsu.com> <9B6E2A8877C38245BFB15CC491A11DA7281752@GRFEXC.intern.adiscon.com> Message-ID: <9B6E2A8877C38245BFB15CC491A11DA7281754@GRFEXC.intern.adiscon.com> I should have waited with my reply ;) I am currently merging things and it seems to go rather smooth. So I also merged your patch: http://git.adiscon.com/?p=rsyslog.git;a=commitdiff;h=167192666ba8905b83210ab7 c5f00cc6be9f7147 It goes into v5-devel. Thanks! Rainer > -----Original Message----- > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards > Sent: Monday, January 16, 2012 4:42 PM > To: Peng Haitao > Cc: rsyslog-users > Subject: Re: [rsyslog] [PATCH] fix error of not define HAVE_SETSID > > > > > -----Original Message----- > > From: Peng Haitao [mailto:penght at cn.fujitsu.com] > > Sent: Monday, January 16, 2012 6:33 AM > > To: Rainer Gerhards > > Cc: rsyslog-users > > Subject: Re: [rsyslog] [PATCH] fix error of not define HAVE_SETSID > > > > > > Peng Haitao said the following on 2012-1-6 14:48: > > > [add list to the CC list] > > > > > > When HAVE_SETSID is not defined, rsyslogd will use ioctl() make > > > itself to daemon, but this can not make rsyslogd process become the > > > process group leader of a new process group. In RHEL6.1, the status > > > is as > follows: > > > # uname -a > > > Linux RHEL6U1GA-Intel64-199 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue > > > May > > > 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux # > > > /etc/init.d/rsyslog > > restart > > > Shutting down system logger: [ OK ] > > > Starting system logger: [ OK ] > > > # ps axo pgrp,ppid,pid,comm | grep rsyslog > > > 6290 1 6301 rsyslogd > > > > > > When we send SIGTERM signal to 6290, rsyslogd will die:( So I think > > > we should call setpgid() before ioctl(). > > > > > > > hi, anyone can comment the patch? > > Sorry, I am still busy, it's on my todo list and looks good on first impression. > Bear with me a little bit more, I hope to be able to go back to real work > soon... ;) > > Rainer > > > > -- > > Best Regards, > > Peng > > > > > Signed-off-by: Peng Haitao > > > --- > > > tools/syslogd.c | 7 +++++++ > > > 1 files changed, 7 insertions(+), 0 deletions(-) > > > > > > diff --git a/tools/syslogd.c b/tools/syslogd.c index > > > d1224f2..ac954b7 > > > 100644 > > > --- a/tools/syslogd.c > > > +++ b/tools/syslogd.c > > > @@ -356,8 +356,15 @@ void untty(void) #else { > > > int i; > > > + pid_t pid; > > > > > > if(!Debug) { > > > + pid = getpid(); > > > + if (setpgid(pid, pid) < 0) { > > > + perror("setpgid"); > > > + exit(1); > > > + } > > > + > > > i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); > > > if (i >= 0) { > > > # if !defined(__hpux) > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ From michael at maymann.org Mon Jan 16 20:07:44 2012 From: michael at maymann.org (Michael Maymann) Date: Mon, 16 Jan 2012 20:07:44 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Hi, 6.2.0 doesn't have DNS cache, does it... I would like to use stable, but would really need the DNS caching feature - so would have to do devel (especially if this is nearly as stable - as Rainer describes): http://lists.adiscon.net/pipermail/rsyslog/2012-January/014244.html Thanks in advance :-) ! ~maymann 2012/1/16 > I would say that 6.2.0 is probably better. > > sorry I can't give you more info on this. > > David Lang > > > On Mon, 16 Jan 2012, Michael Maymann wrote: > > If I want DNS caching, should i use the new stable-6.2.0 or the older >> devel-6.3.6... ? >> Can see this feature mentioned here: >> http://rsyslog.com/features/ >> or here: >> http://rsyslog.com/project-**status/ >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/14 >> >> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> >>> Hi David, >>> >>>> >>>> thanks for this...this is super info...:-) ! >>>> If I have to create different logs per host, will this be the a valid >>>> configuration: >>>> $template DynaFile_messages,?/logfiles_****on_nfs/%HOSTNAME%/messages? >>>> >>>> *.* -?DynaFile_messages >>>> $template DynaFile_secure,?/logfiles_on_****nfs/%HOSTNAME%/secure? >>>> *.* -?DynaFile_secure >>>> $template DynaFile_auth.log,?/logfiles_****on_nfs/%HOSTNAME%/auth.log? >>>> *.* -?DynaFile_auth.log >>>> >>>> >>> I believe so. >>> >>> >>> 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have >>> to >>> >>>> create every hosts dir upfront... ? >>>> >>>> >>> it will create it for you (make sure it's running with the appropriate >>> permissions, if you have rsyslog configured to drop privileges, the lower >>> privileges need the ability to create the directories) >>> >>> >>> 2. Is DNS caching default enabled or do I have to enable this somewhere >>> >>>> first...? >>>> >>>> >>> I don't know, I haven't had a chance to look into that yet. >>> >>> David Lang >>> >>> >>> Thanks in advance :-) ! >>>> ~maymann >>>> >>>> >>>> 2012/1/14 >>>> >>>> http://rsyslog.com/article60/ >>>> >>>>> >>>>> David Lang >>>>> >>>>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>>> >>>>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>>>> >>>>> From: Michael Maymann >>>>>> To: rsyslog-users , david at lang.hm, >>>>>> Michael Maymann >>>>>> >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>> howto/links/examples >>>>>> >>>>>> Hi David, >>>>>> >>>>>> thanks for you kind reply...:-) ! >>>>>> --- >>>>>> This didn't seem to get through to the archives for some reason...: >>>>>> http://lists.adiscon.net/******pipermail/rsyslog/2012-**** >>>>>> January/thread.html>>>>> rsyslog/2012-**January/thread.**html >>>>>> > >>>>>> >>>>> January/thread.** >>>>>> >>>>>> html>>>>> January/thread.html >>>>>> > >>>>>> >>>>>>> >>>>>>> >>>>>> Hope I will not dobbel-post... >>>>>> --- >>>>>> I don't use syslog-relays, so this will not cause me any problems. >>>>>> Don't actually know what version we are running - can see this Monday >>>>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>>>> already then. >>>>>> I have to configure this into a already running live production >>>>>> system - >>>>>> our previous syslog-admin left...:-(. >>>>>> Could I perhaps ask you to be so kind as to give an configuration >>>>>> example >>>>>> of how this is done, if I ask really nicely... :-) ? >>>>>> >>>>>> Thanks in advance :-) ! >>>>>> ~maymann >>>>>> >>>>>> 2012/1/13 >>>>>> >>>>>> you need to be aware that doing the DNS queries is rather expensive >>>>>> >>>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>>>> there >>>>>>> may now be a DNS cache that will drastically help) >>>>>>> >>>>>>> you would need to create a template with FROMHOST in it and use that >>>>>>> as >>>>>>> the filename to write to (look for dynafile in the documentation) >>>>>>> >>>>>>> note that if you are relaying logs from one machine to another, only >>>>>>> the >>>>>>> first machine will see the true source in FROMHOST, machines after >>>>>>> that >>>>>>> will only see the relay box. >>>>>>> >>>>>>> let me know if this doesn't give you enough clues to learn how to do >>>>>>> this. >>>>>>> >>>>>>> David Lang >>>>>>> >>>>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>>>> >>>>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>>>> >>>>>>> From: Michael Maymann >>>>>>> >>>>>>>> Reply-To: rsyslog-users >>>>>>>> To: rsyslog at lists.adiscon.com >>>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>>> howto/links/examples >>>>>>>> >>>>>>>> >>>>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>>>> from >>>>>>>> syslog-info hostname. >>>>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>>>> e.g.: >>>>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>>>> [set_host_compat_list] >>>>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>>>> HostCompatList >>>>>>>> text /tmp/hostCompatList"'#012#********01212/16/11 09:47:40 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>>>> >>>>>>>> Thanks in advance :-) ! >>>>>>>> ~maymann >>>>>>>> >>>>>>>> >>>>>>>> 2012/1/13 Michael Maymann >>>>>>>> >>>>>>>> Hi List, >>>>>>>> >>>>>>>> >>>>>>>> I'm new to rsyslog/syslog in general. >>>>>>>>> >>>>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>>>> Preferably I would like a FQDN.log file for each host (or a >>>>>>>>> FQDN-dir >>>>>>>>> containing logs from this host if more logfiles per host are best >>>>>>>>> practice)... >>>>>>>>> >>>>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>>>> kind >>>>>>>>> of setup. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks in advance :-) ! >>>>>>>>> >>>>>>>>> ~maymann >>>>>>>>> >>>>>>>>> ______________________________********_________________ >>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> >>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>>> >>>>>>>>> > >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> ______________________________********_________________ >>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>>> >>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>> >>>>>>>> > >>>>>>>> >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>> >>>>>>> > >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>> >> From david at lang.hm Mon Jan 16 22:37:51 2012 From: david at lang.hm (david at lang.hm) Date: Mon, 16 Jan 2012 13:37:51 -0800 (PST) Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: Message-ID: Rainer knows this stuff far better than I do, so go with what he says. David Lang On Mon, 16 Jan 2012, Michael Maymann wrote: > Date: Mon, 16 Jan 2012 20:07:44 +0100 > From: Michael Maymann > To: david at lang.hm > Cc: rsyslog-users > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples > > Hi, > > 6.2.0 doesn't have DNS cache, does it... > I would like to use stable, but would really need the DNS caching feature - > so would have to do devel (especially if this is nearly as stable - as > Rainer describes): > http://lists.adiscon.net/pipermail/rsyslog/2012-January/014244.html > > Thanks in advance :-) ! > ~maymann > > 2012/1/16 > >> I would say that 6.2.0 is probably better. >> >> sorry I can't give you more info on this. >> >> David Lang >> >> >> On Mon, 16 Jan 2012, Michael Maymann wrote: >> >> If I want DNS caching, should i use the new stable-6.2.0 or the older >>> devel-6.3.6... ? >>> Can see this feature mentioned here: >>> http://rsyslog.com/features/ >>> or here: >>> http://rsyslog.com/project-**status/ >>> >>> >>> Thanks in advance :-) ! >>> ~maymann >>> >>> 2012/1/14 >>> >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>> >>>> Hi David, >>>> >>>>> >>>>> thanks for this...this is super info...:-) ! >>>>> If I have to create different logs per host, will this be the a valid >>>>> configuration: >>>>> $template DynaFile_messages,?/logfiles_****on_nfs/%HOSTNAME%/messages? >>>>> >>>>> *.* -?DynaFile_messages >>>>> $template DynaFile_secure,?/logfiles_on_****nfs/%HOSTNAME%/secure? >>>>> *.* -?DynaFile_secure >>>>> $template DynaFile_auth.log,?/logfiles_****on_nfs/%HOSTNAME%/auth.log? >>>>> *.* -?DynaFile_auth.log >>>>> >>>>> >>>> I believe so. >>>> >>>> >>>> 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I have >>>> to >>>> >>>>> create every hosts dir upfront... ? >>>>> >>>>> >>>> it will create it for you (make sure it's running with the appropriate >>>> permissions, if you have rsyslog configured to drop privileges, the lower >>>> privileges need the ability to create the directories) >>>> >>>> >>>> 2. Is DNS caching default enabled or do I have to enable this somewhere >>>> >>>>> first...? >>>>> >>>>> >>>> I don't know, I haven't had a chance to look into that yet. >>>> >>>> David Lang >>>> >>>> >>>> Thanks in advance :-) ! >>>>> ~maymann >>>>> >>>>> >>>>> 2012/1/14 >>>>> >>>>> http://rsyslog.com/article60/ >>>>> >>>>>> >>>>>> David Lang >>>>>> >>>>>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>>>>> >>>>>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>>>>> >>>>>> From: Michael Maymann >>>>>>> To: rsyslog-users , david at lang.hm, >>>>>>> Michael Maymann >>>>>>> >>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>> howto/links/examples >>>>>>> >>>>>>> Hi David, >>>>>>> >>>>>>> thanks for you kind reply...:-) ! >>>>>>> --- >>>>>>> This didn't seem to get through to the archives for some reason...: >>>>>>> http://lists.adiscon.net/******pipermail/rsyslog/2012-**** >>>>>>> January/thread.html>>>>>> rsyslog/2012-**January/thread.**html >>>>>>>> >>>>>>> >>>>>> January/thread.** >>>>>>> >>>>>>> html>>>>>> January/thread.html >>>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>> Hope I will not dobbel-post... >>>>>>> --- >>>>>>> I don't use syslog-relays, so this will not cause me any problems. >>>>>>> Don't actually know what version we are running - can see this Monday >>>>>>> morning though... Thanks for this hint... will upgrade to 6.2 if not >>>>>>> already then. >>>>>>> I have to configure this into a already running live production >>>>>>> system - >>>>>>> our previous syslog-admin left...:-(. >>>>>>> Could I perhaps ask you to be so kind as to give an configuration >>>>>>> example >>>>>>> of how this is done, if I ask really nicely... :-) ? >>>>>>> >>>>>>> Thanks in advance :-) ! >>>>>>> ~maymann >>>>>>> >>>>>>> 2012/1/13 >>>>>>> >>>>>>> you need to be aware that doing the DNS queries is rather expensive >>>>>>> >>>>>>> (although I think I saw a comment that in the very latest 6.2 version >>>>>>>> there >>>>>>>> may now be a DNS cache that will drastically help) >>>>>>>> >>>>>>>> you would need to create a template with FROMHOST in it and use that >>>>>>>> as >>>>>>>> the filename to write to (look for dynafile in the documentation) >>>>>>>> >>>>>>>> note that if you are relaying logs from one machine to another, only >>>>>>>> the >>>>>>>> first machine will see the true source in FROMHOST, machines after >>>>>>>> that >>>>>>>> will only see the relay box. >>>>>>>> >>>>>>>> let me know if this doesn't give you enough clues to learn how to do >>>>>>>> this. >>>>>>>> >>>>>>>> David Lang >>>>>>>> >>>>>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>>>>>>> >>>>>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>>>>>>> >>>>>>>> From: Michael Maymann >>>>>>>> >>>>>>>>> Reply-To: rsyslog-users >>>>>>>>> To: rsyslog at lists.adiscon.com >>>>>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>>>>>>>> howto/links/examples >>>>>>>>> >>>>>>>>> >>>>>>>>> Furthermore: would it be possible to validate FQDN from DNS and not >>>>>>>>> from >>>>>>>>> syslog-info hostname. >>>>>>>>> We are getting a lot of weird logfiles as some applications are not >>>>>>>>> including the hostname as the first parameter in the syslog-entries, >>>>>>>>> e.g.: >>>>>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>>>>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>>>>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>>>>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>>>>>>>> [set_host_compat_list] >>>>>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>>>>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData >>>>>>>>> HostCompatList >>>>>>>>> text /tmp/hostCompatList"'#012#********01212/16/11 09:47:40 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Would be nice to validate FQDN from sender DNS query... >>>>>>>>> >>>>>>>>> Thanks in advance :-) ! >>>>>>>>> ~maymann >>>>>>>>> >>>>>>>>> >>>>>>>>> 2012/1/13 Michael Maymann >>>>>>>>> >>>>>>>>> Hi List, >>>>>>>>> >>>>>>>>> >>>>>>>>> I'm new to rsyslog/syslog in general. >>>>>>>>>> >>>>>>>>>> I would like to syslog from all my 100+ network devices. >>>>>>>>>> Preferably I would like a FQDN.log file for each host (or a >>>>>>>>>> FQDN-dir >>>>>>>>>> containing logs from this host if more logfiles per host are best >>>>>>>>>> practice)... >>>>>>>>>> >>>>>>>>>> Can anyone give me an example of (or link to) best practice of this >>>>>>>>>> kind >>>>>>>>>> of setup. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks in advance :-) ! >>>>>>>>>> >>>>>>>>>> ~maymann >>>>>>>>>> >>>>>>>>>> ______________________________********_________________ >>>>>>>>>> >>>>>>>>>> rsyslog mailing list >>>>>>>>>> >>>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> ______________________________********_________________ >>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> >>>>>>>> http://lists.adiscon.net/********mailman/listinfo/rsyslog >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> http://www.rsyslog.com/********professional-services/ >>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>> >>> > From michael at maymann.org Tue Jan 17 08:33:39 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 17 Jan 2012 08:33:39 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, do I have to enable DNS cache somewhere, or is this feature on by default ? Thanks in advance :-) ! ~maymann 2012/1/16 Rainer Gerhards > The cache is available since 6.3.1, so you need to go for the devel > version. > A good place to check those things is the ChangeLog itself, here is the > current one: > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8575d085 > a0fcf48f71339154813971 > ;hb=HEAD > > Note that v6-devel is almost as stable as v6-stable except for the config > read phase at startup. > > HTH > Rainer > > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Monday, January 16, 2012 8:57 AM > > To: david at lang.hm > > Cc: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > howto/links/examples > > > > If I want DNS caching, should i use the new stable-6.2.0 or the older > > devel-6.3.6... ? > > Can see this feature mentioned here: > > http://rsyslog.com/features/ > > or here: > > http://rsyslog.com/project-status/ > > > > > > Thanks in advance :-) ! > > ~maymann > > > > 2012/1/14 > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > Hi David, > > >> > > >> thanks for this...this is super info...:-) ! > > >> If I have to create different logs per host, will this be the a > > valid > > >> configuration: > > >> $template DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > >> *.* -?DynaFile_messages > > >> $template DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > >> *.* -?DynaFile_secure > > >> $template DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > >> *.* -?DynaFile_auth.log > > >> > > > > > > I believe so. > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > have to > > >> create every hosts dir upfront... ? > > >> > > > > > > it will create it for you (make sure it's running with the > > appropriate > > > permissions, if you have rsyslog configured to drop privileges, the > > lower > > > privileges need the ability to create the directories) > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > somewhere > > >> first...? > > >> > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > David Lang > > > > > > > > >> Thanks in advance :-) ! > > >> ~maymann > > >> > > >> > > >> 2012/1/14 > > >> > > >> http://rsyslog.com/article60/ > > >>> > > >>> David Lang > > >>> > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > >>> > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > >>> > > >>>> From: Michael Maymann > > >>>> To: rsyslog-users , david at lang.hm, > > >>>> Michael Maymann > > >>>> > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>> howto/links/examples > > >>>> > > >>>> Hi David, > > >>>> > > >>>> thanks for you kind reply...:-) ! > > >>>> --- > > >>>> This didn't seem to get through to the archives for some > > reason...: > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > >>>> > > January/thread.html > **January/thread.html> > > >>>> > January/thread.** > > >>>> html > January/thread.html> > > >>>> > > > >>>> > > >>>> Hope I will not dobbel-post... > > >>>> --- > > >>>> I don't use syslog-relays, so this will not cause me any problems. > > >>>> Don't actually know what version we are running - can see this > > Monday > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 if > > not > > >>>> already then. > > >>>> I have to configure this into a already running live production > > system - > > >>>> our previous syslog-admin left...:-(. > > >>>> Could I perhaps ask you to be so kind as to give an configuration > > >>>> example > > >>>> of how this is done, if I ask really nicely... :-) ? > > >>>> > > >>>> Thanks in advance :-) ! > > >>>> ~maymann > > >>>> > > >>>> 2012/1/13 > > >>>> > > >>>> you need to be aware that doing the DNS queries is rather > > expensive > > >>>> > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > version > > >>>>> there > > >>>>> may now be a DNS cache that will drastically help) > > >>>>> > > >>>>> you would need to create a template with FROMHOST in it and use > > that as > > >>>>> the filename to write to (look for dynafile in the documentation) > > >>>>> > > >>>>> note that if you are relaying logs from one machine to another, > > only > > >>>>> the > > >>>>> first machine will see the true source in FROMHOST, machines > > after that > > >>>>> will only see the relay box. > > >>>>> > > >>>>> let me know if this doesn't give you enough clues to learn how to > > do > > >>>>> this. > > >>>>> > > >>>>> David Lang > > >>>>> > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > >>>>> > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > >>>>> > > >>>>> From: Michael Maymann > > >>>>>> Reply-To: rsyslog-users > > >>>>>> To: rsyslog at lists.adiscon.com > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > >>>>>> howto/links/examples > > >>>>>> > > >>>>>> > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS and > > not > > >>>>>> from > > >>>>>> syslog-info hostname. > > >>>>>> We are getting a lot of weird logfiles as some applications are > > not > > >>>>>> including the hostname as the first parameter in the syslog- > > entries, > > >>>>>> e.g.: > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > >>>>>> [set_host_compat_list] > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd "SetUserData > > >>>>>> HostCompatList > > >>>>>> text /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > >>>>>> > > >>>>>> Thanks in advance :-) ! > > >>>>>> ~maymann > > >>>>>> > > >>>>>> > > >>>>>> 2012/1/13 Michael Maymann > > >>>>>> > > >>>>>> Hi List, > > >>>>>> > > >>>>>> > > >>>>>>> I'm new to rsyslog/syslog in general. > > >>>>>>> > > >>>>>>> I would like to syslog from all my 100+ network devices. > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > FQDN-dir > > >>>>>>> containing logs from this host if more logfiles per host are > > best > > >>>>>>> practice)... > > >>>>>>> > > >>>>>>> Can anyone give me an example of (or link to) best practice of > > this > > >>>>>>> kind > > >>>>>>> of setup. > > >>>>>>> > > >>>>>>> > > >>>>>>> Thanks in advance :-) ! > > >>>>>>> > > >>>>>>> ~maymann > > >>>>>>> > > >>>>>>> ______________________________******_________________ > > >>>>>>> > > >>>>>>> rsyslog mailing list > > >>>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > **services/ > > >>>>>> > services/ > > >>>>>> > > > >>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> ______________________________******_________________ > > >>>>>> > > >>>>>> rsyslog mailing list > > >>>>> > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > iscon.net/****mailman/listinfo/rsyslog> > > >>>>> > > > discon.net/**mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > > discon.net/mailman/**listinfo/rsyslog> > > >>>>> > > > scon.net/mailman/listinfo/rsyslog> > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> http://www.rsyslog.com/******professional- > > services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > **services/ > > >>>>> > services/ > > >>>>> > > > >>>>> > > >>>>>> > > >>>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From michael at maymann.org Tue Jan 17 10:36:46 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 17 Jan 2012 10:36:46 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> Message-ID: Hi Rainer, yes, rsyslog-6.3.6 needs libestr and libee...: I installed libestr just fine: [root at MyServer pkgconfig]# locate libestr /root/libestr-0.1.2 /root/libestr-0.1.2.tar.gz /root/libestr-0.1.2/AUTHORS /root/libestr-0.1.2/COPYING /root/libestr-0.1.2/ChangeLog /root/libestr-0.1.2/INSTALL /root/libestr-0.1.2/Makefile /root/libestr-0.1.2/Makefile.am /root/libestr-0.1.2/Makefile.in /root/libestr-0.1.2/NEWS /root/libestr-0.1.2/README /root/libestr-0.1.2/aclocal.m4 /root/libestr-0.1.2/compile /root/libestr-0.1.2/config.guess /root/libestr-0.1.2/config.h /root/libestr-0.1.2/config.h.in /root/libestr-0.1.2/config.log /root/libestr-0.1.2/config.status /root/libestr-0.1.2/config.sub /root/libestr-0.1.2/configure /root/libestr-0.1.2/configure.ac /root/libestr-0.1.2/depcomp /root/libestr-0.1.2/include /root/libestr-0.1.2/install-sh /root/libestr-0.1.2/libestr.pc /root/libestr-0.1.2/libestr.pc.in /root/libestr-0.1.2/libtool /root/libestr-0.1.2/ltmain.sh /root/libestr-0.1.2/m4 /root/libestr-0.1.2/missing /root/libestr-0.1.2/src /root/libestr-0.1.2/stamp-h1 /root/libestr-0.1.2/include/Makefile /root/libestr-0.1.2/include/Makefile.am /root/libestr-0.1.2/include/Makefile.in /root/libestr-0.1.2/include/libestr.h /root/libestr-0.1.2/m4/libtool.m4 /root/libestr-0.1.2/m4/ltoptions.m4 /root/libestr-0.1.2/m4/ltsugar.m4 /root/libestr-0.1.2/m4/ltversion.m4 /root/libestr-0.1.2/m4/lt~obsolete.m4 /root/libestr-0.1.2/src/.deps /root/libestr-0.1.2/src/.libs /root/libestr-0.1.2/src/Makefile /root/libestr-0.1.2/src/Makefile.am /root/libestr-0.1.2/src/Makefile.in /root/libestr-0.1.2/src/libestr.c /root/libestr-0.1.2/src/libestr.la /root/libestr-0.1.2/src/libestr_la-libestr.lo /root/libestr-0.1.2/src/libestr_la-libestr.o /root/libestr-0.1.2/src/libestr_la-string.lo /root/libestr-0.1.2/src/libestr_la-string.o /root/libestr-0.1.2/src/string.c /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo /root/libestr-0.1.2/src/.libs/libestr.a /root/libestr-0.1.2/src/.libs/libestr.la /root/libestr-0.1.2/src/.libs/libestr.lai /root/libestr-0.1.2/src/.libs/libestr.so /root/libestr-0.1.2/src/.libs/libestr.so.0 /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o /root/libestr-0.1.2/src/.libs/libestr_la-string.o /usr/local/include/libestr.h /usr/local/lib/libestr.a /usr/local/lib/libestr.la /usr/local/lib/libestr.so /usr/local/lib/libestr.so.0 /usr/local/lib/libestr.so.0.0.0 /usr/local/lib/pkgconfig/libestr.pc I try to install libee: [root at MyServer libee-0.3.2]# ./configure LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking whether gcc and cc understand -c and -o together... yes checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking how to print strings... printf checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1966080 checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... yes checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for ar... ar checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... no checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking for stdlib.h... (cached) yes checking for GNU libc compatible malloc... yes checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.9.0... yes checking for LIBESTR... yes configure: creating ./config.status config.status: creating Makefile config.status: creating libee.pc config.status: creating src/Makefile config.status: creating include/Makefile config.status: creating include/libee/Makefile config.status: creating tests/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands config.status: executing libtool commands ***************************************************** libee will be compiled with the following settings: Debug mode enabled: no Testbench enabled: yes [root at MyServer libee-0.3.2]# echo $? 0 [root at MyServer libee-0.3.2]# make make all-recursive make[1]: Entering directory `/root/libee-0.3.2' Making all in tests make[2]: Entering directory `/root/libee-0.3.2/tests' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/libee-0.3.2/tests' Making all in include make[2]: Entering directory `/root/libee-0.3.2/include' Making all in libee make[3]: Entering directory `/root/libee-0.3.2/include/libee' make[3]: Nothing to be done for `all'. make[3]: Leaving directory `/root/libee-0.3.2/include/libee' make[3]: Entering directory `/root/libee-0.3.2/include' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/root/libee-0.3.2/include' make[2]: Leaving directory `/root/libee-0.3.2/include' Making all in src make[2]: Entering directory `/root/libee-0.3.2/src' CC libee_la-ctx.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-tag.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-event.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-value.lo value.c: In function 'ee_newValue': value.c:37: warning: unused parameter 'ctx' gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-tagbucket.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-field.lo field.c: In function 'ee_getFieldValueAsStr': field.c:181: warning: 'str' may be used uninitialized in this function gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-fieldbucket.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-primitivetype.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-int_dec.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-apache_dec.lo apache_dec.c: In function 'ee_newApache': apache_dec.c:37: warning: unused parameter 'ctx' apache_dec.c: In function 'ee_apacheAddName': apache_dec.c:71: warning: unused parameter 'ctx' apache_dec.c: In function 'processLn': apache_dec.c:205: warning: unused variable 'value' gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-syslog_enc.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-json_enc.lo gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-csv_enc.lo csv_enc.c: In function 'ee_AddName': csv_enc.c:66: warning: unused parameter 'ctx' gcc: /usr/local/include: linker input file unused because linking not done CC libee_la-xml_enc.lo xml_enc.c: In function 'ee_addValue_XML': xml_enc.c:60: warning: unused variable 'j' xml_enc.c:59: warning: unused variable 'numbuf' xml_enc.c: At top level: xml_enc.c:40: warning: 'hexdigit' defined but not used gcc: /usr/local/include: linker input file unused because linking not done CCLD libee.la CC convert-convert.o gcc: /usr/local/include: linker input file unused because linking not done CCLD convert /usr/local/lib: file not recognized: Is a directory collect2: ld returned 1 exit status make[2]: *** [convert] Error 1 make[2]: Leaving directory `/root/libee-0.3.2/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/libee-0.3.2' make: *** [all] Error 2 [root at MyServer libee-0.3.2]# echo $? 2 It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know how to solve this ? Thanks in advance :-) ! ~maymann 2012/1/16 Rainer Gerhards > > -----Original Message----- > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > Sent: Monday, January 16, 2012 10:48 AM > > To: rsyslog-users > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > howto/links/examples > > > > Hi Rainer, > > > > Thanks for your reply. > > How do I install 6.3.6 on RHEL6 easiest ? > > I have no specific instructions. Just grab the sources and compile, I'd say > ;) Note, however, that you need to install libestr and probably libee > first. > > Rainer > > > > Thanks in advance :-)! > > ~maymann > > > > 2012/1/16 Rainer Gerhards > > > > > The cache is available since 6.3.1, so you need to go for the devel > > > version. > > > A good place to check those things is the ChangeLog itself, here is > > > the current one: > > > > > > > > > http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 > > > 575d085 > > > a0fcf48f71339154813971 > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> > > > ;hb=HEAD > > > > > > Note that v6-devel is almost as stable as v6-stable except for the > > > config read phase at startup. > > > > > > HTH > > > Rainer > > > > > > > -----Original Message----- > > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > > > > Sent: Monday, January 16, 2012 8:57 AM > > > > To: david at lang.hm > > > > Cc: rsyslog-users > > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > > howto/links/examples > > > > > > > > If I want DNS caching, should i use the new stable-6.2.0 or the > > > > older devel-6.3.6... ? > > > > Can see this feature mentioned here: > > > > http://rsyslog.com/features/ > > > > or here: > > > > http://rsyslog.com/project-status/ > > > > > > > > > > > > Thanks in advance :-) ! > > > > ~maymann > > > > > > > > 2012/1/14 > > > > > > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > > > > > > > Hi David, > > > > >> > > > > >> thanks for this...this is super info...:-) ! > > > > >> If I have to create different logs per host, will this be the a > > > > valid > > > > >> configuration: > > > > >> $template > > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > > > > >> *.* -?DynaFile_messages > > > > >> $template > > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > > > > >> *.* -?DynaFile_secure > > > > >> $template > > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > > > > >> *.* -?DynaFile_auth.log > > > > >> > > > > > > > > > > I believe so. > > > > > > > > > > > > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I > > > > have to > > > > >> create every hosts dir upfront... ? > > > > >> > > > > > > > > > > it will create it for you (make sure it's running with the > > > > appropriate > > > > > permissions, if you have rsyslog configured to drop privileges, > > > > > the > > > > lower > > > > > privileges need the ability to create the directories) > > > > > > > > > > > > > > > 2. Is DNS caching default enabled or do I have to enable this > > > > somewhere > > > > >> first...? > > > > >> > > > > > > > > > > I don't know, I haven't had a chance to look into that yet. > > > > > > > > > > David Lang > > > > > > > > > > > > > > >> Thanks in advance :-) ! > > > > >> ~maymann > > > > >> > > > > >> > > > > >> 2012/1/14 > > > > >> > > > > >> http://rsyslog.com/article60/ > > > > >>> > > > > >>> David Lang > > > > >>> > > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > > > > >>> > > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > > > > >>> > > > > >>>> From: Michael Maymann > > > > >>>> To: rsyslog-users , david at lang.hm, > > > > >>>> Michael Maymann > > > > >>>> > > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > > >>>> howto/links/examples > > > > >>>> > > > > >>>> Hi David, > > > > >>>> > > > > >>>> thanks for you kind reply...:-) ! > > > > >>>> --- > > > > >>>> This didn't seem to get through to the archives for some > > > > reason...: > > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > > > > >>>> > > > > January/thread.html > > > 2- > > > > **January/thread.html> > > > > >>>> > > > January/thread.** > > > > >>>> html > > > January/thread.html> > > > > >>>> > > > > > >>>> > > > > >>>> Hope I will not dobbel-post... > > > > >>>> --- > > > > >>>> I don't use syslog-relays, so this will not cause me any > problems. > > > > >>>> Don't actually know what version we are running - can see this > > > > Monday > > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 > > > > >>>> if > > > > not > > > > >>>> already then. > > > > >>>> I have to configure this into a already running live production > > > > system - > > > > >>>> our previous syslog-admin left...:-(. > > > > >>>> Could I perhaps ask you to be so kind as to give an > > > > >>>> configuration example of how this is done, if I ask really > > > > >>>> nicely... :-) ? > > > > >>>> > > > > >>>> Thanks in advance :-) ! > > > > >>>> ~maymann > > > > >>>> > > > > >>>> 2012/1/13 > > > > >>>> > > > > >>>> you need to be aware that doing the DNS queries is rather > > > > expensive > > > > >>>> > > > > >>>>> (although I think I saw a comment that in the very latest 6.2 > > > > version > > > > >>>>> there > > > > >>>>> may now be a DNS cache that will drastically help) > > > > >>>>> > > > > >>>>> you would need to create a template with FROMHOST in it and > > > > >>>>> use > > > > that as > > > > >>>>> the filename to write to (look for dynafile in the > > > > >>>>> documentation) > > > > >>>>> > > > > >>>>> note that if you are relaying logs from one machine to > > > > >>>>> another, > > > > only > > > > >>>>> the > > > > >>>>> first machine will see the true source in FROMHOST, machines > > > > after that > > > > >>>>> will only see the relay box. > > > > >>>>> > > > > >>>>> let me know if this doesn't give you enough clues to learn how > > > > >>>>> to > > > > do > > > > >>>>> this. > > > > >>>>> > > > > >>>>> David Lang > > > > >>>>> > > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > > > > >>>>> > > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > > > > >>>>> > > > > >>>>> From: Michael Maymann > > > > >>>>>> Reply-To: rsyslog-users > > > > >>>>>> To: rsyslog at lists.adiscon.com > > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > > > > >>>>>> howto/links/examples > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS > > > > >>>>>> and > > > > not > > > > >>>>>> from > > > > >>>>>> syslog-info hostname. > > > > >>>>>> We are getting a lot of weird logfiles as some applications > > > > >>>>>> are > > > > not > > > > >>>>>> including the hostname as the first parameter in the syslog- > > > > entries, > > > > >>>>>> e.g.: > > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > > > > >>>>>> [set_host_compat_list] > > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is > > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd > > "SetUserData > > > > >>>>>> HostCompatList text > > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> Would be nice to validate FQDN from sender DNS query... > > > > >>>>>> > > > > >>>>>> Thanks in advance :-) ! > > > > >>>>>> ~maymann > > > > >>>>>> > > > > >>>>>> > > > > >>>>>> 2012/1/13 Michael Maymann > > > > >>>>>> > > > > >>>>>> Hi List, > > > > >>>>>> > > > > >>>>>> > > > > >>>>>>> I'm new to rsyslog/syslog in general. > > > > >>>>>>> > > > > >>>>>>> I would like to syslog from all my 100+ network devices. > > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a > > > > FQDN-dir > > > > >>>>>>> containing logs from this host if more logfiles per host are > > > > best > > > > >>>>>>> practice)... > > > > >>>>>>> > > > > >>>>>>> Can anyone give me an example of (or link to) best practice > > > > >>>>>>> of > > > > this > > > > >>>>>>> kind > > > > >>>>>>> of setup. > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>>> Thanks in advance :-) ! > > > > >>>>>>> > > > > >>>>>>> ~maymann > > > > >>>>>>> > > > > >>>>>>> > > ______________________________******_________________ > > > > >>>>>>> > > > > >>>>>>> rsyslog mailing list > > > > >>>>>> > > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > > >>>>>> > > > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > > >>>>>> > > > > > >>>>>> > > > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > > >>>>>> > > > > > > > adi > > > > scon.net/mailman/listinfo/rsyslog> > > > > >>>>>> > > > > > >>>>>> > > > > >>>>>>> > > > > >>>>>>> http://www.rsyslog.com/******professional- > > > > services/ > > > > >>>>>> > > > services/ > > > > >>>>>> > > > > > >>>>>> > > > **services/ > > > > >>>>>> > > > services/ > > > > >>>>>> > > > > > >>>>>> > > > > >>>>>>> > > > > >>>>>>> > > > > >>>>>> > > ______________________________******_________________ > > > > >>>>>> > > > > >>>>>> rsyslog mailing list > > > > >>>>> > > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog > > > .ad iscon.net/****mailman/listinfo/rsyslog> > > > > >>>>> > > > > > > > s.a discon.net/**mailman/listinfo/rsyslog> > > > > >>>>> > > > > > >>>>> > > > > > > > s.a discon.net/mailman/**listinfo/rsyslog> > > > > >>>>> > > > > > > > adi > > > > scon.net/mailman/listinfo/rsyslog> > > > > >>>>> > > > > > >>>>> > > > > >>>>>> > > > > >>>>>> http://www.rsyslog.com/******professional- > > > > services/ > > > > >>>>> > > > services/ > > > > >>>>> > > > > > >>>>> > > > **services/ > > > > >>>>> > > > services/ > > > > >>>>> > > > > > >>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>>> > > > > >>>>> > > > > >>>> > > > > >> > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com/professional-services/ > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com/professional-services/ > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > From michael at maymann.org Tue Jan 17 12:09:19 2012 From: michael at maymann.org (Michael Maymann) Date: Tue, 17 Jan 2012 12:09:19 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> Message-ID: Tried to delete all installed and follow this guide: http://www.liblognorm.com/help/first-steps-using-liblognorm/ same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib --includedir=/usr/include: ... checking for LIBESTR... configure: error: Package requirements (libestr >= 0.0.0) were not met: No package 'libestr' found Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. Alternatively, you may set the environment variables LIBESTR_CFLAGS and LIBESTR_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. Thanks in advance :-) ! ~maymann 2012/1/17 Michael Maymann > Hi Rainer, > > yes, rsyslog-6.3.6 needs libestr and libee...: > I installed libestr just fine: > [root at MyServer pkgconfig]# locate libestr > /root/libestr-0.1.2 > /root/libestr-0.1.2.tar.gz > /root/libestr-0.1.2/AUTHORS > /root/libestr-0.1.2/COPYING > /root/libestr-0.1.2/ChangeLog > /root/libestr-0.1.2/INSTALL > /root/libestr-0.1.2/Makefile > /root/libestr-0.1.2/Makefile.am > /root/libestr-0.1.2/Makefile.in > /root/libestr-0.1.2/NEWS > /root/libestr-0.1.2/README > /root/libestr-0.1.2/aclocal.m4 > /root/libestr-0.1.2/compile > /root/libestr-0.1.2/config.guess > /root/libestr-0.1.2/config.h > /root/libestr-0.1.2/config.h.in > /root/libestr-0.1.2/config.log > /root/libestr-0.1.2/config.status > /root/libestr-0.1.2/config.sub > /root/libestr-0.1.2/configure > /root/libestr-0.1.2/configure.ac > /root/libestr-0.1.2/depcomp > /root/libestr-0.1.2/include > /root/libestr-0.1.2/install-sh > /root/libestr-0.1.2/libestr.pc > /root/libestr-0.1.2/libestr.pc.in > /root/libestr-0.1.2/libtool > /root/libestr-0.1.2/ltmain.sh > /root/libestr-0.1.2/m4 > /root/libestr-0.1.2/missing > /root/libestr-0.1.2/src > /root/libestr-0.1.2/stamp-h1 > /root/libestr-0.1.2/include/Makefile > /root/libestr-0.1.2/include/Makefile.am > /root/libestr-0.1.2/include/Makefile.in > /root/libestr-0.1.2/include/libestr.h > /root/libestr-0.1.2/m4/libtool.m4 > /root/libestr-0.1.2/m4/ltoptions.m4 > /root/libestr-0.1.2/m4/ltsugar.m4 > /root/libestr-0.1.2/m4/ltversion.m4 > /root/libestr-0.1.2/m4/lt~obsolete.m4 > /root/libestr-0.1.2/src/.deps > /root/libestr-0.1.2/src/.libs > /root/libestr-0.1.2/src/Makefile > /root/libestr-0.1.2/src/Makefile.am > /root/libestr-0.1.2/src/Makefile.in > /root/libestr-0.1.2/src/libestr.c > /root/libestr-0.1.2/src/libestr.la > /root/libestr-0.1.2/src/libestr_la-libestr.lo > /root/libestr-0.1.2/src/libestr_la-libestr.o > /root/libestr-0.1.2/src/libestr_la-string.lo > /root/libestr-0.1.2/src/libestr_la-string.o > /root/libestr-0.1.2/src/string.c > /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo > /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo > /root/libestr-0.1.2/src/.libs/libestr.a > /root/libestr-0.1.2/src/.libs/libestr.la > /root/libestr-0.1.2/src/.libs/libestr.lai > /root/libestr-0.1.2/src/.libs/libestr.so > /root/libestr-0.1.2/src/.libs/libestr.so.0 > /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 > /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o > /root/libestr-0.1.2/src/.libs/libestr_la-string.o > /usr/local/include/libestr.h > /usr/local/lib/libestr.a > /usr/local/lib/libestr.la > /usr/local/lib/libestr.so > /usr/local/lib/libestr.so.0 > /usr/local/lib/libestr.so.0.0.0 > /usr/local/lib/pkgconfig/libestr.pc > > I try to install libee: > [root at MyServer libee-0.3.2]# ./configure > LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib > checking for a BSD-compatible install... /usr/bin/install -c > checking whether build environment is sane... yes > checking for a thread-safe mkdir -p... /bin/mkdir -p > checking for gawk... gawk > checking whether make sets $(MAKE)... yes > checking for gcc... gcc > checking whether the C compiler works... yes > checking for C compiler default output file name... a.out > checking for suffix of executables... > checking whether we are cross compiling... no > checking for suffix of object files... o > checking whether we are using the GNU C compiler... yes > checking whether gcc accepts -g... yes > checking for gcc option to accept ISO C89... none needed > checking for style of include used by make... GNU > checking dependency style of gcc... gcc3 > checking whether gcc and cc understand -c and -o together... yes > checking build system type... x86_64-unknown-linux-gnu > checking host system type... x86_64-unknown-linux-gnu > checking how to print strings... printf > checking for a sed that does not truncate output... /bin/sed > checking for grep that handles long lines and -e... /bin/grep > checking for egrep... /bin/grep -E > checking for fgrep... /bin/grep -F > checking for ld used by gcc... /usr/bin/ld > checking if the linker (/usr/bin/ld) is GNU ld... yes > checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B > checking the name lister (/usr/bin/nm -B) interface... BSD nm > checking whether ln -s works... yes > checking the maximum length of command line arguments... 1966080 > checking whether the shell understands some XSI constructs... yes > checking whether the shell understands "+="... yes > checking for /usr/bin/ld option to reload object files... -r > checking for objdump... objdump > checking how to recognize dependent libraries... pass_all > checking for ar... ar > checking for strip... strip > checking for ranlib... ranlib > checking command to parse /usr/bin/nm -B output from gcc object... ok > checking how to run the C preprocessor... gcc -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking for dlfcn.h... yes > checking for objdir... .libs > checking if gcc supports -fno-rtti -fno-exceptions... no > checking for gcc option to produce PIC... -fPIC -DPIC > checking if gcc PIC flag -fPIC -DPIC works... yes > checking if gcc static flag -static works... no > checking if gcc supports -c -o file.o... yes > checking if gcc supports -c -o file.o... (cached) yes > checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports > shared libraries... yes > checking whether -lc should be explicitly linked in... no > checking dynamic linker characteristics... GNU/Linux ld.so > checking how to hardcode library paths into programs... immediate > checking whether stripping libraries is possible... yes > checking if libtool supports shared libraries... yes > checking whether to build shared libraries... yes > checking whether to build static libraries... yes > checking for stdlib.h... (cached) yes > checking for GNU libc compatible malloc... yes > checking for pkg-config... /usr/bin/pkg-config > checking pkg-config is at least version 0.9.0... yes > checking for LIBESTR... yes > configure: creating ./config.status > config.status: creating Makefile > config.status: creating libee.pc > config.status: creating src/Makefile > config.status: creating include/Makefile > config.status: creating include/libee/Makefile > config.status: creating tests/Makefile > config.status: creating config.h > config.status: config.h is unchanged > config.status: executing depfiles commands > config.status: executing libtool commands > ***************************************************** > libee will be compiled with the following settings: > > Debug mode enabled: no > Testbench enabled: yes > [root at MyServer libee-0.3.2]# echo $? > 0 > [root at MyServer libee-0.3.2]# make > make all-recursive > make[1]: Entering directory `/root/libee-0.3.2' > Making all in tests > make[2]: Entering directory `/root/libee-0.3.2/tests' > make[2]: Nothing to be done for `all'. > make[2]: Leaving directory `/root/libee-0.3.2/tests' > Making all in include > make[2]: Entering directory `/root/libee-0.3.2/include' > Making all in libee > make[3]: Entering directory `/root/libee-0.3.2/include/libee' > make[3]: Nothing to be done for `all'. > make[3]: Leaving directory `/root/libee-0.3.2/include/libee' > make[3]: Entering directory `/root/libee-0.3.2/include' > make[3]: Nothing to be done for `all-am'. > make[3]: Leaving directory `/root/libee-0.3.2/include' > make[2]: Leaving directory `/root/libee-0.3.2/include' > Making all in src > make[2]: Entering directory `/root/libee-0.3.2/src' > CC libee_la-ctx.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-tag.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-event.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-value.lo > value.c: In function 'ee_newValue': > value.c:37: warning: unused parameter 'ctx' > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-tagbucket.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-field.lo > field.c: In function 'ee_getFieldValueAsStr': > field.c:181: warning: 'str' may be used uninitialized in this function > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-fieldbucket.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-primitivetype.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-int_dec.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-apache_dec.lo > apache_dec.c: In function 'ee_newApache': > apache_dec.c:37: warning: unused parameter 'ctx' > apache_dec.c: In function 'ee_apacheAddName': > apache_dec.c:71: warning: unused parameter 'ctx' > apache_dec.c: In function 'processLn': > apache_dec.c:205: warning: unused variable 'value' > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-syslog_enc.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-json_enc.lo > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-csv_enc.lo > csv_enc.c: In function 'ee_AddName': > csv_enc.c:66: warning: unused parameter 'ctx' > gcc: /usr/local/include: linker input file unused because linking not done > CC libee_la-xml_enc.lo > xml_enc.c: In function 'ee_addValue_XML': > xml_enc.c:60: warning: unused variable 'j' > xml_enc.c:59: warning: unused variable 'numbuf' > xml_enc.c: At top level: > xml_enc.c:40: warning: 'hexdigit' defined but not used > gcc: /usr/local/include: linker input file unused because linking not done > CCLD libee.la > CC convert-convert.o > gcc: /usr/local/include: linker input file unused because linking not done > CCLD convert > /usr/local/lib: file not recognized: Is a directory > collect2: ld returned 1 exit status > make[2]: *** [convert] Error 1 > make[2]: Leaving directory `/root/libee-0.3.2/src' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/root/libee-0.3.2' > make: *** [all] Error 2 > [root at MyServer libee-0.3.2]# echo $? > 2 > > It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know > how to solve this ? > > > > Thanks in advance :-) ! > ~maymann > > 2012/1/16 Rainer Gerhards > >> > -----Original Message----- >> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >> > Sent: Monday, January 16, 2012 10:48 AM >> > To: rsyslog-users >> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> howto/links/examples >> > >> > Hi Rainer, >> > >> > Thanks for your reply. >> > How do I install 6.3.6 on RHEL6 easiest ? >> >> I have no specific instructions. Just grab the sources and compile, I'd >> say >> ;) Note, however, that you need to install libestr and probably libee >> first. >> >> Rainer >> > >> > Thanks in advance :-)! >> > ~maymann >> > >> > 2012/1/16 Rainer Gerhards >> > >> > > The cache is available since 6.3.1, so you need to go for the devel >> > > version. >> > > A good place to check those things is the ChangeLog itself, here is >> > > the current one: >> > > >> > > >> > > >> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 >> > > 575d085 >> > > a0fcf48f71339154813971< >> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= >> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> >> > > ;hb=HEAD >> > > >> > > Note that v6-devel is almost as stable as v6-stable except for the >> > > config read phase at startup. >> > > >> > > HTH >> > > Rainer >> > > >> > > > -----Original Message----- >> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >> > > > Sent: Monday, January 16, 2012 8:57 AM >> > > > To: david at lang.hm >> > > > Cc: rsyslog-users >> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> > > > howto/links/examples >> > > > >> > > > If I want DNS caching, should i use the new stable-6.2.0 or the >> > > > older devel-6.3.6... ? >> > > > Can see this feature mentioned here: >> > > > http://rsyslog.com/features/ >> > > > or here: >> > > > http://rsyslog.com/project-status/ >> > > > >> > > > >> > > > Thanks in advance :-) ! >> > > > ~maymann >> > > > >> > > > 2012/1/14 >> > > > >> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: >> > > > > >> > > > > Hi David, >> > > > >> >> > > > >> thanks for this...this is super info...:-) ! >> > > > >> If I have to create different logs per host, will this be the a >> > > > valid >> > > > >> configuration: >> > > > >> $template >> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >> > > > >> *.* -?DynaFile_messages >> > > > >> $template >> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >> > > > >> *.* -?DynaFile_secure >> > > > >> $template >> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >> > > > >> *.* -?DynaFile_auth.log >> > > > >> >> > > > > >> > > > > I believe so. >> > > > > >> > > > > >> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do I >> > > > have to >> > > > >> create every hosts dir upfront... ? >> > > > >> >> > > > > >> > > > > it will create it for you (make sure it's running with the >> > > > appropriate >> > > > > permissions, if you have rsyslog configured to drop privileges, >> > > > > the >> > > > lower >> > > > > privileges need the ability to create the directories) >> > > > > >> > > > > >> > > > > 2. Is DNS caching default enabled or do I have to enable this >> > > > somewhere >> > > > >> first...? >> > > > >> >> > > > > >> > > > > I don't know, I haven't had a chance to look into that yet. >> > > > > >> > > > > David Lang >> > > > > >> > > > > >> > > > >> Thanks in advance :-) ! >> > > > >> ~maymann >> > > > >> >> > > > >> >> > > > >> 2012/1/14 >> > > > >> >> > > > >> http://rsyslog.com/article60/ >> > > > >>> >> > > > >>> David Lang >> > > > >>> >> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >> > > > >>> >> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >> > > > >>> >> > > > >>>> From: Michael Maymann >> > > > >>>> To: rsyslog-users , david at lang.hm, >> > > > >>>> Michael Maymann >> > > > >>>> >> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> > > > >>>> howto/links/examples >> > > > >>>> >> > > > >>>> Hi David, >> > > > >>>> >> > > > >>>> thanks for you kind reply...:-) ! >> > > > >>>> --- >> > > > >>>> This didn't seem to get through to the archives for some >> > > > reason...: >> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >> > > > >>>> >> > > > January/thread.html< >> http://lists.adiscon.net/**pipermail/rsyslog/201 >> > > > 2- >> > > > **January/thread.html> >> > > > >>>> > > > > January/thread.** >> > > > >>>> html> > > > January/thread.html> >> > > > >>>> > >> > > > >>>> >> > > > >>>> Hope I will not dobbel-post... >> > > > >>>> --- >> > > > >>>> I don't use syslog-relays, so this will not cause me any >> problems. >> > > > >>>> Don't actually know what version we are running - can see this >> > > > Monday >> > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 >> > > > >>>> if >> > > > not >> > > > >>>> already then. >> > > > >>>> I have to configure this into a already running live production >> > > > system - >> > > > >>>> our previous syslog-admin left...:-(. >> > > > >>>> Could I perhaps ask you to be so kind as to give an >> > > > >>>> configuration example of how this is done, if I ask really >> > > > >>>> nicely... :-) ? >> > > > >>>> >> > > > >>>> Thanks in advance :-) ! >> > > > >>>> ~maymann >> > > > >>>> >> > > > >>>> 2012/1/13 >> > > > >>>> >> > > > >>>> you need to be aware that doing the DNS queries is rather >> > > > expensive >> > > > >>>> >> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 >> > > > version >> > > > >>>>> there >> > > > >>>>> may now be a DNS cache that will drastically help) >> > > > >>>>> >> > > > >>>>> you would need to create a template with FROMHOST in it and >> > > > >>>>> use >> > > > that as >> > > > >>>>> the filename to write to (look for dynafile in the >> > > > >>>>> documentation) >> > > > >>>>> >> > > > >>>>> note that if you are relaying logs from one machine to >> > > > >>>>> another, >> > > > only >> > > > >>>>> the >> > > > >>>>> first machine will see the true source in FROMHOST, machines >> > > > after that >> > > > >>>>> will only see the relay box. >> > > > >>>>> >> > > > >>>>> let me know if this doesn't give you enough clues to learn how >> > > > >>>>> to >> > > > do >> > > > >>>>> this. >> > > > >>>>> >> > > > >>>>> David Lang >> > > > >>>>> >> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >> > > > >>>>> >> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >> > > > >>>>> >> > > > >>>>> From: Michael Maymann >> > > > >>>>>> Reply-To: rsyslog-users >> > > > >>>>>> To: rsyslog at lists.adiscon.com >> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >> > > > >>>>>> howto/links/examples >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS >> > > > >>>>>> and >> > > > not >> > > > >>>>>> from >> > > > >>>>>> syslog-info hostname. >> > > > >>>>>> We are getting a lot of weird logfiles as some applications >> > > > >>>>>> are >> > > > not >> > > > >>>>>> including the hostname as the first parameter in the syslog- >> > > > entries, >> > > > >>>>>> e.g.: >> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >> > > > >>>>>> [set_host_compat_list] >> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd >> > "SetUserData >> > > > >>>>>> HostCompatList text >> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... >> > > > >>>>>> >> > > > >>>>>> Thanks in advance :-) ! >> > > > >>>>>> ~maymann >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>> 2012/1/13 Michael Maymann >> > > > >>>>>> >> > > > >>>>>> Hi List, >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>>>> I'm new to rsyslog/syslog in general. >> > > > >>>>>>> >> > > > >>>>>>> I would like to syslog from all my 100+ network devices. >> > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a >> > > > FQDN-dir >> > > > >>>>>>> containing logs from this host if more logfiles per host are >> > > > best >> > > > >>>>>>> practice)... >> > > > >>>>>>> >> > > > >>>>>>> Can anyone give me an example of (or link to) best practice >> > > > >>>>>>> of >> > > > this >> > > > >>>>>>> kind >> > > > >>>>>>> of setup. >> > > > >>>>>>> >> > > > >>>>>>> >> > > > >>>>>>> Thanks in advance :-) ! >> > > > >>>>>>> >> > > > >>>>>>> ~maymann >> > > > >>>>>>> >> > > > >>>>>>> >> > ______________________________******_________________ >> > > > >>>>>>> >> > > > >>>>>>> rsyslog mailing list >> > > > >>>>>> >> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >> http://lists >> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >> > > > >>>>>> >> > > > > http://list >> > > > s.a discon.net/**mailman/listinfo/rsyslog> >> > > > >>>>>> > >> > > > >>>>>> >> > > > > http://list >> > > > s.a discon.net/mailman/**listinfo/rsyslog> >> > > > >>>>>> >> > > > > . >> > > > adi >> > > > scon.net/mailman/listinfo/rsyslog> >> > > > >>>>>> > >> > > > >>>>>> >> > > > >>>>>>> >> > > > >>>>>>> http://www.rsyslog.com/******professional- >> > > > services/ >> > > > >>>>>> > > > > services/ >> > > > >>>>>> > >> > > > >>>>>> > > > > **services/ >> > > > >>>>>> > > > > services/ >> > > > >>>>>> > >> > > > >>>>>> >> > > > >>>>>>> >> > > > >>>>>>> >> > > > >>>>>> >> > ______________________________******_________________ >> > > > >>>>>> >> > > > >>>>>> rsyslog mailing list >> > > > >>>>> >> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >> http://lists >> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >> > > > >>>>> >> > > > > http://list >> > > > s.a discon.net/**mailman/listinfo/rsyslog> >> > > > >>>>> > >> > > > >>>>> >> > > > > http://list >> > > > s.a discon.net/mailman/**listinfo/rsyslog> >> > > > >>>>> >> > > > > . >> > > > adi >> > > > scon.net/mailman/listinfo/rsyslog> >> > > > >>>>> > >> > > > >>>>> >> > > > >>>>>> >> > > > >>>>>> http://www.rsyslog.com/******professional- >> > > > services/ >> > > > >>>>> > > > > services/ >> > > > >>>>> > >> > > > >>>>> > > > > **services/ >> > > > >>>>> > > > > services/ >> > > > >>>>> > >> > > > >>>>> >> > > > >>>>>> >> > > > >>>>>> >> > > > >>>>> >> > > > >>>>> >> > > > >>>> >> > > > >> >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> > > From michael at maymann.org Wed Jan 18 20:36:07 2012 From: michael at maymann.org (Michael Maymann) Date: Wed, 18 Jan 2012 20:36:07 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: References: <9B6E2A8877C38245BFB15CC491A11DA7281745@GRFEXC.intern.adiscon.com> <9B6E2A8877C38245BFB15CC491A11DA7281747@GRFEXC.intern.adiscon.com> Message-ID: The --libdir=/usr/lib64 option did the trick...:-): cd libestr ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd libee/ ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd rsyslog-6.3.6 ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr [root at MyServer rsyslog-6.3.6]# echo $? 0 [root at MyServer rsyslog-6.3.6]# make make all-recursive make[1]: Entering directory `/root/rsyslog-6.3.6' Making all in doc make[2]: Entering directory `/root/rsyslog-6.3.6/doc' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/rsyslog-6.3.6/doc' Making all in runtime make[2]: Entering directory `/root/rsyslog-6.3.6/runtime' CC librsyslog_la-rsyslog.lo In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:28:18: error: zlib.h: No such file or directory In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:32: error: expected ')' before 'strm' zlibw.h:33: error: expected ';' before 'int' In file included from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: stream.h:123: error: expected specifier-qualifier-list before 'Bytef' make[2]: *** [librsyslog_la-rsyslog.lo] Error 1 make[2]: Leaving directory `/root/rsyslog-6.3.6/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog-6.3.6' make: *** [all] Error 2 So both libestr and linee in installed and ./configure in rsyslog dir ran without errors, but now make is causing me problems..., anyone who know how to solve this ? Thanks in advance :-) ! ~maymann 2012/1/17 Michael Maymann > Tried to delete all installed and follow this guide: > http://www.liblognorm.com/help/first-steps-using-liblognorm/ > same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib > --includedir=/usr/include: > ... > checking for LIBESTR... configure: error: Package requirements (libestr >= > 0.0.0) were not met: > > No package 'libestr' found > > Consider adjusting the PKG_CONFIG_PATH environment variable if you > installed software in a non-standard prefix. > > Alternatively, you may set the environment variables LIBESTR_CFLAGS > and LIBESTR_LIBS to avoid the need to call pkg-config. > See the pkg-config man page for more details. > > > Thanks in advance :-) ! > ~maymann > > > 2012/1/17 Michael Maymann > >> Hi Rainer, >> >> yes, rsyslog-6.3.6 needs libestr and libee...: >> I installed libestr just fine: >> [root at MyServer pkgconfig]# locate libestr >> /root/libestr-0.1.2 >> /root/libestr-0.1.2.tar.gz >> /root/libestr-0.1.2/AUTHORS >> /root/libestr-0.1.2/COPYING >> /root/libestr-0.1.2/ChangeLog >> /root/libestr-0.1.2/INSTALL >> /root/libestr-0.1.2/Makefile >> /root/libestr-0.1.2/Makefile.am >> /root/libestr-0.1.2/Makefile.in >> /root/libestr-0.1.2/NEWS >> /root/libestr-0.1.2/README >> /root/libestr-0.1.2/aclocal.m4 >> /root/libestr-0.1.2/compile >> /root/libestr-0.1.2/config.guess >> /root/libestr-0.1.2/config.h >> /root/libestr-0.1.2/config.h.in >> /root/libestr-0.1.2/config.log >> /root/libestr-0.1.2/config.status >> /root/libestr-0.1.2/config.sub >> /root/libestr-0.1.2/configure >> /root/libestr-0.1.2/configure.ac >> /root/libestr-0.1.2/depcomp >> /root/libestr-0.1.2/include >> /root/libestr-0.1.2/install-sh >> /root/libestr-0.1.2/libestr.pc >> /root/libestr-0.1.2/libestr.pc.in >> /root/libestr-0.1.2/libtool >> /root/libestr-0.1.2/ltmain.sh >> /root/libestr-0.1.2/m4 >> /root/libestr-0.1.2/missing >> /root/libestr-0.1.2/src >> /root/libestr-0.1.2/stamp-h1 >> /root/libestr-0.1.2/include/Makefile >> /root/libestr-0.1.2/include/Makefile.am >> /root/libestr-0.1.2/include/Makefile.in >> /root/libestr-0.1.2/include/libestr.h >> /root/libestr-0.1.2/m4/libtool.m4 >> /root/libestr-0.1.2/m4/ltoptions.m4 >> /root/libestr-0.1.2/m4/ltsugar.m4 >> /root/libestr-0.1.2/m4/ltversion.m4 >> /root/libestr-0.1.2/m4/lt~obsolete.m4 >> /root/libestr-0.1.2/src/.deps >> /root/libestr-0.1.2/src/.libs >> /root/libestr-0.1.2/src/Makefile >> /root/libestr-0.1.2/src/Makefile.am >> /root/libestr-0.1.2/src/Makefile.in >> /root/libestr-0.1.2/src/libestr.c >> /root/libestr-0.1.2/src/libestr.la >> /root/libestr-0.1.2/src/libestr_la-libestr.lo >> /root/libestr-0.1.2/src/libestr_la-libestr.o >> /root/libestr-0.1.2/src/libestr_la-string.lo >> /root/libestr-0.1.2/src/libestr_la-string.o >> /root/libestr-0.1.2/src/string.c >> /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo >> /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo >> /root/libestr-0.1.2/src/.libs/libestr.a >> /root/libestr-0.1.2/src/.libs/libestr.la >> /root/libestr-0.1.2/src/.libs/libestr.lai >> /root/libestr-0.1.2/src/.libs/libestr.so >> /root/libestr-0.1.2/src/.libs/libestr.so.0 >> /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 >> /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o >> /root/libestr-0.1.2/src/.libs/libestr_la-string.o >> /usr/local/include/libestr.h >> /usr/local/lib/libestr.a >> /usr/local/lib/libestr.la >> /usr/local/lib/libestr.so >> /usr/local/lib/libestr.so.0 >> /usr/local/lib/libestr.so.0.0.0 >> /usr/local/lib/pkgconfig/libestr.pc >> >> I try to install libee: >> [root at MyServer libee-0.3.2]# ./configure >> LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib >> checking for a BSD-compatible install... /usr/bin/install -c >> checking whether build environment is sane... yes >> checking for a thread-safe mkdir -p... /bin/mkdir -p >> checking for gawk... gawk >> checking whether make sets $(MAKE)... yes >> checking for gcc... gcc >> checking whether the C compiler works... yes >> checking for C compiler default output file name... a.out >> checking for suffix of executables... >> checking whether we are cross compiling... no >> checking for suffix of object files... o >> checking whether we are using the GNU C compiler... yes >> checking whether gcc accepts -g... yes >> checking for gcc option to accept ISO C89... none needed >> checking for style of include used by make... GNU >> checking dependency style of gcc... gcc3 >> checking whether gcc and cc understand -c and -o together... yes >> checking build system type... x86_64-unknown-linux-gnu >> checking host system type... x86_64-unknown-linux-gnu >> checking how to print strings... printf >> checking for a sed that does not truncate output... /bin/sed >> checking for grep that handles long lines and -e... /bin/grep >> checking for egrep... /bin/grep -E >> checking for fgrep... /bin/grep -F >> checking for ld used by gcc... /usr/bin/ld >> checking if the linker (/usr/bin/ld) is GNU ld... yes >> checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B >> checking the name lister (/usr/bin/nm -B) interface... BSD nm >> checking whether ln -s works... yes >> checking the maximum length of command line arguments... 1966080 >> checking whether the shell understands some XSI constructs... yes >> checking whether the shell understands "+="... yes >> checking for /usr/bin/ld option to reload object files... -r >> checking for objdump... objdump >> checking how to recognize dependent libraries... pass_all >> checking for ar... ar >> checking for strip... strip >> checking for ranlib... ranlib >> checking command to parse /usr/bin/nm -B output from gcc object... ok >> checking how to run the C preprocessor... gcc -E >> checking for ANSI C header files... yes >> checking for sys/types.h... yes >> checking for sys/stat.h... yes >> checking for stdlib.h... yes >> checking for string.h... yes >> checking for memory.h... yes >> checking for strings.h... yes >> checking for inttypes.h... yes >> checking for stdint.h... yes >> checking for unistd.h... yes >> checking for dlfcn.h... yes >> checking for objdir... .libs >> checking if gcc supports -fno-rtti -fno-exceptions... no >> checking for gcc option to produce PIC... -fPIC -DPIC >> checking if gcc PIC flag -fPIC -DPIC works... yes >> checking if gcc static flag -static works... no >> checking if gcc supports -c -o file.o... yes >> checking if gcc supports -c -o file.o... (cached) yes >> checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports >> shared libraries... yes >> checking whether -lc should be explicitly linked in... no >> checking dynamic linker characteristics... GNU/Linux ld.so >> checking how to hardcode library paths into programs... immediate >> checking whether stripping libraries is possible... yes >> checking if libtool supports shared libraries... yes >> checking whether to build shared libraries... yes >> checking whether to build static libraries... yes >> checking for stdlib.h... (cached) yes >> checking for GNU libc compatible malloc... yes >> checking for pkg-config... /usr/bin/pkg-config >> checking pkg-config is at least version 0.9.0... yes >> checking for LIBESTR... yes >> configure: creating ./config.status >> config.status: creating Makefile >> config.status: creating libee.pc >> config.status: creating src/Makefile >> config.status: creating include/Makefile >> config.status: creating include/libee/Makefile >> config.status: creating tests/Makefile >> config.status: creating config.h >> config.status: config.h is unchanged >> config.status: executing depfiles commands >> config.status: executing libtool commands >> ***************************************************** >> libee will be compiled with the following settings: >> >> Debug mode enabled: no >> Testbench enabled: yes >> [root at MyServer libee-0.3.2]# echo $? >> 0 >> [root at MyServer libee-0.3.2]# make >> make all-recursive >> make[1]: Entering directory `/root/libee-0.3.2' >> Making all in tests >> make[2]: Entering directory `/root/libee-0.3.2/tests' >> make[2]: Nothing to be done for `all'. >> make[2]: Leaving directory `/root/libee-0.3.2/tests' >> Making all in include >> make[2]: Entering directory `/root/libee-0.3.2/include' >> Making all in libee >> make[3]: Entering directory `/root/libee-0.3.2/include/libee' >> make[3]: Nothing to be done for `all'. >> make[3]: Leaving directory `/root/libee-0.3.2/include/libee' >> make[3]: Entering directory `/root/libee-0.3.2/include' >> make[3]: Nothing to be done for `all-am'. >> make[3]: Leaving directory `/root/libee-0.3.2/include' >> make[2]: Leaving directory `/root/libee-0.3.2/include' >> Making all in src >> make[2]: Entering directory `/root/libee-0.3.2/src' >> CC libee_la-ctx.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tag.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-event.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-value.lo >> value.c: In function 'ee_newValue': >> value.c:37: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tagbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-field.lo >> field.c: In function 'ee_getFieldValueAsStr': >> field.c:181: warning: 'str' may be used uninitialized in this function >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-fieldbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-primitivetype.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-int_dec.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-apache_dec.lo >> apache_dec.c: In function 'ee_newApache': >> apache_dec.c:37: warning: unused parameter 'ctx' >> apache_dec.c: In function 'ee_apacheAddName': >> apache_dec.c:71: warning: unused parameter 'ctx' >> apache_dec.c: In function 'processLn': >> apache_dec.c:205: warning: unused variable 'value' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-syslog_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-json_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-csv_enc.lo >> csv_enc.c: In function 'ee_AddName': >> csv_enc.c:66: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-xml_enc.lo >> xml_enc.c: In function 'ee_addValue_XML': >> xml_enc.c:60: warning: unused variable 'j' >> xml_enc.c:59: warning: unused variable 'numbuf' >> xml_enc.c: At top level: >> xml_enc.c:40: warning: 'hexdigit' defined but not used >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD libee.la >> CC convert-convert.o >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD convert >> /usr/local/lib: file not recognized: Is a directory >> collect2: ld returned 1 exit status >> make[2]: *** [convert] Error 1 >> make[2]: Leaving directory `/root/libee-0.3.2/src' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/root/libee-0.3.2' >> make: *** [all] Error 2 >> [root at MyServer libee-0.3.2]# echo $? >> 2 >> >> It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know >> how to solve this ? >> >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/16 Rainer Gerhards >> >>> > -----Original Message----- >>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > Sent: Monday, January 16, 2012 10:48 AM >>> > To: rsyslog-users >>> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> howto/links/examples >>> > >>> > Hi Rainer, >>> > >>> > Thanks for your reply. >>> > How do I install 6.3.6 on RHEL6 easiest ? >>> >>> I have no specific instructions. Just grab the sources and compile, I'd >>> say >>> ;) Note, however, that you need to install libestr and probably libee >>> first. >>> >>> Rainer >>> > >>> > Thanks in advance :-)! >>> > ~maymann >>> > >>> > 2012/1/16 Rainer Gerhards >>> > >>> > > The cache is available since 6.3.1, so you need to go for the devel >>> > > version. >>> > > A good place to check those things is the ChangeLog itself, here is >>> > > the current one: >>> > > >>> > > >>> > > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 >>> > > 575d085 >>> > > a0fcf48f71339154813971< >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= >>> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> >>> > > ;hb=HEAD >>> > > >>> > > Note that v6-devel is almost as stable as v6-stable except for the >>> > > config read phase at startup. >>> > > >>> > > HTH >>> > > Rainer >>> > > >>> > > > -----Original Message----- >>> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > > > Sent: Monday, January 16, 2012 8:57 AM >>> > > > To: david at lang.hm >>> > > > Cc: rsyslog-users >>> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > howto/links/examples >>> > > > >>> > > > If I want DNS caching, should i use the new stable-6.2.0 or the >>> > > > older devel-6.3.6... ? >>> > > > Can see this feature mentioned here: >>> > > > http://rsyslog.com/features/ >>> > > > or here: >>> > > > http://rsyslog.com/project-status/ >>> > > > >>> > > > >>> > > > Thanks in advance :-) ! >>> > > > ~maymann >>> > > > >>> > > > 2012/1/14 >>> > > > >>> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > > >>> > > > > Hi David, >>> > > > >> >>> > > > >> thanks for this...this is super info...:-) ! >>> > > > >> If I have to create different logs per host, will this be the a >>> > > > valid >>> > > > >> configuration: >>> > > > >> $template >>> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >>> > > > >> *.* -?DynaFile_messages >>> > > > >> $template >>> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >>> > > > >> *.* -?DynaFile_secure >>> > > > >> $template >>> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >>> > > > >> *.* -?DynaFile_auth.log >>> > > > >> >>> > > > > >>> > > > > I believe so. >>> > > > > >>> > > > > >>> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do >>> I >>> > > > have to >>> > > > >> create every hosts dir upfront... ? >>> > > > >> >>> > > > > >>> > > > > it will create it for you (make sure it's running with the >>> > > > appropriate >>> > > > > permissions, if you have rsyslog configured to drop privileges, >>> > > > > the >>> > > > lower >>> > > > > privileges need the ability to create the directories) >>> > > > > >>> > > > > >>> > > > > 2. Is DNS caching default enabled or do I have to enable this >>> > > > somewhere >>> > > > >> first...? >>> > > > >> >>> > > > > >>> > > > > I don't know, I haven't had a chance to look into that yet. >>> > > > > >>> > > > > David Lang >>> > > > > >>> > > > > >>> > > > >> Thanks in advance :-) ! >>> > > > >> ~maymann >>> > > > >> >>> > > > >> >>> > > > >> 2012/1/14 >>> > > > >> >>> > > > >> http://rsyslog.com/article60/ >>> > > > >>> >>> > > > >>> David Lang >>> > > > >>> >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > >>> >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> > > > >>> >>> > > > >>>> From: Michael Maymann >>> > > > >>>> To: rsyslog-users , david at lang.hm, >>> > > > >>>> Michael Maymann >>> > > > >>>> >>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>> howto/links/examples >>> > > > >>>> >>> > > > >>>> Hi David, >>> > > > >>>> >>> > > > >>>> thanks for you kind reply...:-) ! >>> > > > >>>> --- >>> > > > >>>> This didn't seem to get through to the archives for some >>> > > > reason...: >>> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>> > > > >>>> >>> > > > January/thread.html< >>> http://lists.adiscon.net/**pipermail/rsyslog/201 >>> > > > 2- >>> > > > **January/thread.html> >>> > > > >>>> >> > > > January/thread.** >>> > > > >>>> html>> > > > January/thread.html> >>> > > > >>>> > >>> > > > >>>> >>> > > > >>>> Hope I will not dobbel-post... >>> > > > >>>> --- >>> > > > >>>> I don't use syslog-relays, so this will not cause me any >>> problems. >>> > > > >>>> Don't actually know what version we are running - can see this >>> > > > Monday >>> > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 >>> > > > >>>> if >>> > > > not >>> > > > >>>> already then. >>> > > > >>>> I have to configure this into a already running live >>> production >>> > > > system - >>> > > > >>>> our previous syslog-admin left...:-(. >>> > > > >>>> Could I perhaps ask you to be so kind as to give an >>> > > > >>>> configuration example of how this is done, if I ask really >>> > > > >>>> nicely... :-) ? >>> > > > >>>> >>> > > > >>>> Thanks in advance :-) ! >>> > > > >>>> ~maymann >>> > > > >>>> >>> > > > >>>> 2012/1/13 >>> > > > >>>> >>> > > > >>>> you need to be aware that doing the DNS queries is rather >>> > > > expensive >>> > > > >>>> >>> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 >>> > > > version >>> > > > >>>>> there >>> > > > >>>>> may now be a DNS cache that will drastically help) >>> > > > >>>>> >>> > > > >>>>> you would need to create a template with FROMHOST in it and >>> > > > >>>>> use >>> > > > that as >>> > > > >>>>> the filename to write to (look for dynafile in the >>> > > > >>>>> documentation) >>> > > > >>>>> >>> > > > >>>>> note that if you are relaying logs from one machine to >>> > > > >>>>> another, >>> > > > only >>> > > > >>>>> the >>> > > > >>>>> first machine will see the true source in FROMHOST, machines >>> > > > after that >>> > > > >>>>> will only see the relay box. >>> > > > >>>>> >>> > > > >>>>> let me know if this doesn't give you enough clues to learn >>> how >>> > > > >>>>> to >>> > > > do >>> > > > >>>>> this. >>> > > > >>>>> >>> > > > >>>>> David Lang >>> > > > >>>>> >>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>> > > > >>>>> >>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> > > > >>>>> >>> > > > >>>>> From: Michael Maymann >>> > > > >>>>>> Reply-To: rsyslog-users >>> > > > >>>>>> To: rsyslog at lists.adiscon.com >>> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>>>> howto/links/examples >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS >>> > > > >>>>>> and >>> > > > not >>> > > > >>>>>> from >>> > > > >>>>>> syslog-info hostname. >>> > > > >>>>>> We are getting a lot of weird logfiles as some applications >>> > > > >>>>>> are >>> > > > not >>> > > > >>>>>> including the hostname as the first parameter in the syslog- >>> > > > entries, >>> > > > >>>>>> e.g.: >>> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>> > > > >>>>>> [set_host_compat_list] >>> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd >>> > "SetUserData >>> > > > >>>>>> HostCompatList text >>> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... >>> > > > >>>>>> >>> > > > >>>>>> Thanks in advance :-) ! >>> > > > >>>>>> ~maymann >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> 2012/1/13 Michael Maymann >>> > > > >>>>>> >>> > > > >>>>>> Hi List, >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>>> I'm new to rsyslog/syslog in general. >>> > > > >>>>>>> >>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. >>> > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a >>> > > > FQDN-dir >>> > > > >>>>>>> containing logs from this host if more logfiles per host >>> are >>> > > > best >>> > > > >>>>>>> practice)... >>> > > > >>>>>>> >>> > > > >>>>>>> Can anyone give me an example of (or link to) best practice >>> > > > >>>>>>> of >>> > > > this >>> > > > >>>>>>> kind >>> > > > >>>>>>> of setup. >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> Thanks in advance :-) ! >>> > > > >>>>>>> >>> > > > >>>>>>> ~maymann >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>>> >>> > > > >>>>>>> rsyslog mailing list >>> > > > >>>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >> > > > **services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>> >>> > > > >>>>>> rsyslog mailing list >>> > > > >>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >> > > > **services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>> >>> > > > >>>>> >>> > > > >>>> >>> > > > >> >>> > > > _______________________________________________ >>> > > > rsyslog mailing list >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > > http://www.rsyslog.com/professional-services/ >>> > > _______________________________________________ >>> > > rsyslog mailing list >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > http://www.rsyslog.com/professional-services/ >>> > > >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com/professional-services/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> >> >> > From rgerhards at hq.adiscon.com Wed Jan 18 20:38:52 2012 From: rgerhards at hq.adiscon.com (Rainer Gerhards) Date: Wed, 18 Jan 2012 20:38:52 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples Message-ID: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Zlib-dev is missing, looks like configure did not catch that. Rainer Michael Maymann hat geschrieben:The --libdir=/usr/lib64 option did the trick...:-): cd libestr ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd libee/ ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr make make install cd .. cd rsyslog-6.3.6 ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr [root at MyServer rsyslog-6.3.6]# echo $? 0 [root at MyServer rsyslog-6.3.6]# make make all-recursive make[1]: Entering directory `/root/rsyslog-6.3.6' Making all in doc make[2]: Entering directory `/root/rsyslog-6.3.6/doc' make[2]: Nothing to be done for `all'. make[2]: Leaving directory `/root/rsyslog-6.3.6/doc' Making all in runtime make[2]: Entering directory `/root/rsyslog-6.3.6/runtime' CC librsyslog_la-rsyslog.lo In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:28:18: error: zlib.h: No such file or directory In file included from stream.h:72, from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: zlibw.h:32: error: expected ')' before 'strm' zlibw.h:33: error: expected ';' before 'int' In file included from obj.h:50, from rsyslog.h:474, from rsyslog.c:63: stream.h:123: error: expected specifier-qualifier-list before 'Bytef' make[2]: *** [librsyslog_la-rsyslog.lo] Error 1 make[2]: Leaving directory `/root/rsyslog-6.3.6/runtime' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/rsyslog-6.3.6' make: *** [all] Error 2 So both libestr and linee in installed and ./configure in rsyslog dir ran without errors, but now make is causing me problems..., anyone who know how to solve this ? Thanks in advance :-) ! ~maymann 2012/1/17 Michael Maymann > Tried to delete all installed and follow this guide: > http://www.liblognorm.com/help/first-steps-using-liblognorm/ > same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib > --includedir=/usr/include: > ... > checking for LIBESTR... configure: error: Package requirements (libestr >= > 0.0.0) were not met: > > No package 'libestr' found > > Consider adjusting the PKG_CONFIG_PATH environment variable if you > installed software in a non-standard prefix. > > Alternatively, you may set the environment variables LIBESTR_CFLAGS > and LIBESTR_LIBS to avoid the need to call pkg-config. > See the pkg-config man page for more details. > > > Thanks in advance :-) ! > ~maymann > > > 2012/1/17 Michael Maymann > >> Hi Rainer, >> >> yes, rsyslog-6.3.6 needs libestr and libee...: >> I installed libestr just fine: >> [root at MyServer pkgconfig]# locate libestr >> /root/libestr-0.1.2 >> /root/libestr-0.1.2.tar.gz >> /root/libestr-0.1.2/AUTHORS >> /root/libestr-0.1.2/COPYING >> /root/libestr-0.1.2/ChangeLog >> /root/libestr-0.1.2/INSTALL >> /root/libestr-0.1.2/Makefile >> /root/libestr-0.1.2/Makefile.am >> /root/libestr-0.1.2/Makefile.in >> /root/libestr-0.1.2/NEWS >> /root/libestr-0.1.2/README >> /root/libestr-0.1.2/aclocal.m4 >> /root/libestr-0.1.2/compile >> /root/libestr-0.1.2/config.guess >> /root/libestr-0.1.2/config.h >> /root/libestr-0.1.2/config.h.in >> /root/libestr-0.1.2/config.log >> /root/libestr-0.1.2/config.status >> /root/libestr-0.1.2/config.sub >> /root/libestr-0.1.2/configure >> /root/libestr-0.1.2/configure.ac >> /root/libestr-0.1.2/depcomp >> /root/libestr-0.1.2/include >> /root/libestr-0.1.2/install-sh >> /root/libestr-0.1.2/libestr.pc >> /root/libestr-0.1.2/libestr.pc.in >> /root/libestr-0.1.2/libtool >> /root/libestr-0.1.2/ltmain.sh >> /root/libestr-0.1.2/m4 >> /root/libestr-0.1.2/missing >> /root/libestr-0.1.2/src >> /root/libestr-0.1.2/stamp-h1 >> /root/libestr-0.1.2/include/Makefile >> /root/libestr-0.1.2/include/Makefile.am >> /root/libestr-0.1.2/include/Makefile.in >> /root/libestr-0.1.2/include/libestr.h >> /root/libestr-0.1.2/m4/libtool.m4 >> /root/libestr-0.1.2/m4/ltoptions.m4 >> /root/libestr-0.1.2/m4/ltsugar.m4 >> /root/libestr-0.1.2/m4/ltversion.m4 >> /root/libestr-0.1.2/m4/lt~obsolete.m4 >> /root/libestr-0.1.2/src/.deps >> /root/libestr-0.1.2/src/.libs >> /root/libestr-0.1.2/src/Makefile >> /root/libestr-0.1.2/src/Makefile.am >> /root/libestr-0.1.2/src/Makefile.in >> /root/libestr-0.1.2/src/libestr.c >> /root/libestr-0.1.2/src/libestr.la >> /root/libestr-0.1.2/src/libestr_la-libestr.lo >> /root/libestr-0.1.2/src/libestr_la-libestr.o >> /root/libestr-0.1.2/src/libestr_la-string.lo >> /root/libestr-0.1.2/src/libestr_la-string.o >> /root/libestr-0.1.2/src/string.c >> /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo >> /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo >> /root/libestr-0.1.2/src/.libs/libestr.a >> /root/libestr-0.1.2/src/.libs/libestr.la >> /root/libestr-0.1.2/src/.libs/libestr.lai >> /root/libestr-0.1.2/src/.libs/libestr.so >> /root/libestr-0.1.2/src/.libs/libestr.so.0 >> /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 >> /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o >> /root/libestr-0.1.2/src/.libs/libestr_la-string.o >> /usr/local/include/libestr.h >> /usr/local/lib/libestr.a >> /usr/local/lib/libestr.la >> /usr/local/lib/libestr.so >> /usr/local/lib/libestr.so.0 >> /usr/local/lib/libestr.so.0.0.0 >> /usr/local/lib/pkgconfig/libestr.pc >> >> I try to install libee: >> [root at MyServer libee-0.3.2]# ./configure >> LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib >> checking for a BSD-compatible install... /usr/bin/install -c >> checking whether build environment is sane... yes >> checking for a thread-safe mkdir -p... /bin/mkdir -p >> checking for gawk... gawk >> checking whether make sets $(MAKE)... yes >> checking for gcc... gcc >> checking whether the C compiler works... yes >> checking for C compiler default output file name... a.out >> checking for suffix of executables... >> checking whether we are cross compiling... no >> checking for suffix of object files... o >> checking whether we are using the GNU C compiler... yes >> checking whether gcc accepts -g... yes >> checking for gcc option to accept ISO C89... none needed >> checking for style of include used by make... GNU >> checking dependency style of gcc... gcc3 >> checking whether gcc and cc understand -c and -o together... yes >> checking build system type... x86_64-unknown-linux-gnu >> checking host system type... x86_64-unknown-linux-gnu >> checking how to print strings... printf >> checking for a sed that does not truncate output... /bin/sed >> checking for grep that handles long lines and -e... /bin/grep >> checking for egrep... /bin/grep -E >> checking for fgrep... /bin/grep -F >> checking for ld used by gcc... /usr/bin/ld >> checking if the linker (/usr/bin/ld) is GNU ld... yes >> checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B >> checking the name lister (/usr/bin/nm -B) interface... BSD nm >> checking whether ln -s works... yes >> checking the maximum length of command line arguments... 1966080 >> checking whether the shell understands some XSI constructs... yes >> checking whether the shell understands "+="... yes >> checking for /usr/bin/ld option to reload object files... -r >> checking for objdump... objdump >> checking how to recognize dependent libraries... pass_all >> checking for ar... ar >> checking for strip... strip >> checking for ranlib... ranlib >> checking command to parse /usr/bin/nm -B output from gcc object... ok >> checking how to run the C preprocessor... gcc -E >> checking for ANSI C header files... yes >> checking for sys/types.h... yes >> checking for sys/stat.h... yes >> checking for stdlib.h... yes >> checking for string.h... yes >> checking for memory.h... yes >> checking for strings.h... yes >> checking for inttypes.h... yes >> checking for stdint.h... yes >> checking for unistd.h... yes >> checking for dlfcn.h... yes >> checking for objdir... .libs >> checking if gcc supports -fno-rtti -fno-exceptions... no >> checking for gcc option to produce PIC... -fPIC -DPIC >> checking if gcc PIC flag -fPIC -DPIC works... yes >> checking if gcc static flag -static works... no >> checking if gcc supports -c -o file.o... yes >> checking if gcc supports -c -o file.o... (cached) yes >> checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports >> shared libraries... yes >> checking whether -lc should be explicitly linked in... no >> checking dynamic linker characteristics... GNU/Linux ld.so >> checking how to hardcode library paths into programs... immediate >> checking whether stripping libraries is possible... yes >> checking if libtool supports shared libraries... yes >> checking whether to build shared libraries... yes >> checking whether to build static libraries... yes >> checking for stdlib.h... (cached) yes >> checking for GNU libc compatible malloc... yes >> checking for pkg-config... /usr/bin/pkg-config >> checking pkg-config is at least version 0.9.0... yes >> checking for LIBESTR... yes >> configure: creating ./config.status >> config.status: creating Makefile >> config.status: creating libee.pc >> config.status: creating src/Makefile >> config.status: creating include/Makefile >> config.status: creating include/libee/Makefile >> config.status: creating tests/Makefile >> config.status: creating config.h >> config.status: config.h is unchanged >> config.status: executing depfiles commands >> config.status: executing libtool commands >> ***************************************************** >> libee will be compiled with the following settings: >> >> Debug mode enabled: no >> Testbench enabled: yes >> [root at MyServer libee-0.3.2]# echo $? >> 0 >> [root at MyServer libee-0.3.2]# make >> make all-recursive >> make[1]: Entering directory `/root/libee-0.3.2' >> Making all in tests >> make[2]: Entering directory `/root/libee-0.3.2/tests' >> make[2]: Nothing to be done for `all'. >> make[2]: Leaving directory `/root/libee-0.3.2/tests' >> Making all in include >> make[2]: Entering directory `/root/libee-0.3.2/include' >> Making all in libee >> make[3]: Entering directory `/root/libee-0.3.2/include/libee' >> make[3]: Nothing to be done for `all'. >> make[3]: Leaving directory `/root/libee-0.3.2/include/libee' >> make[3]: Entering directory `/root/libee-0.3.2/include' >> make[3]: Nothing to be done for `all-am'. >> make[3]: Leaving directory `/root/libee-0.3.2/include' >> make[2]: Leaving directory `/root/libee-0.3.2/include' >> Making all in src >> make[2]: Entering directory `/root/libee-0.3.2/src' >> CC libee_la-ctx.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tag.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-event.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-value.lo >> value.c: In function 'ee_newValue': >> value.c:37: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-tagbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-field.lo >> field.c: In function 'ee_getFieldValueAsStr': >> field.c:181: warning: 'str' may be used uninitialized in this function >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-fieldbucket.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-primitivetype.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-int_dec.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-apache_dec.lo >> apache_dec.c: In function 'ee_newApache': >> apache_dec.c:37: warning: unused parameter 'ctx' >> apache_dec.c: In function 'ee_apacheAddName': >> apache_dec.c:71: warning: unused parameter 'ctx' >> apache_dec.c: In function 'processLn': >> apache_dec.c:205: warning: unused variable 'value' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-syslog_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-json_enc.lo >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-csv_enc.lo >> csv_enc.c: In function 'ee_AddName': >> csv_enc.c:66: warning: unused parameter 'ctx' >> gcc: /usr/local/include: linker input file unused because linking not done >> CC libee_la-xml_enc.lo >> xml_enc.c: In function 'ee_addValue_XML': >> xml_enc.c:60: warning: unused variable 'j' >> xml_enc.c:59: warning: unused variable 'numbuf' >> xml_enc.c: At top level: >> xml_enc.c:40: warning: 'hexdigit' defined but not used >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD libee.la >> CC convert-convert.o >> gcc: /usr/local/include: linker input file unused because linking not done >> CCLD convert >> /usr/local/lib: file not recognized: Is a directory >> collect2: ld returned 1 exit status >> make[2]: *** [convert] Error 1 >> make[2]: Leaving directory `/root/libee-0.3.2/src' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/root/libee-0.3.2' >> make: *** [all] Error 2 >> [root at MyServer libee-0.3.2]# echo $? >> 2 >> >> It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know >> how to solve this ? >> >> >> >> Thanks in advance :-) ! >> ~maymann >> >> 2012/1/16 Rainer Gerhards >> >>> > -----Original Message----- >>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > Sent: Monday, January 16, 2012 10:48 AM >>> > To: rsyslog-users >>> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> howto/links/examples >>> > >>> > Hi Rainer, >>> > >>> > Thanks for your reply. >>> > How do I install 6.3.6 on RHEL6 easiest ? >>> >>> I have no specific instructions. Just grab the sources and compile, I'd >>> say >>> ;) Note, however, that you need to install libestr and probably libee >>> first. >>> >>> Rainer >>> > >>> > Thanks in advance :-)! >>> > ~maymann >>> > >>> > 2012/1/16 Rainer Gerhards >>> > >>> > > The cache is available since 6.3.1, so you need to go for the devel >>> > > version. >>> > > A good place to check those things is the ChangeLog itself, here is >>> > > the current one: >>> > > >>> > > >>> > > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 >>> > > 575d085 >>> > > a0fcf48f71339154813971< >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= >>> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> >>> > > ;hb=HEAD >>> > > >>> > > Note that v6-devel is almost as stable as v6-stable except for the >>> > > config read phase at startup. >>> > > >>> > > HTH >>> > > Rainer >>> > > >>> > > > -----Original Message----- >>> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- >>> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann >>> > > > Sent: Monday, January 16, 2012 8:57 AM >>> > > > To: david at lang.hm >>> > > > Cc: rsyslog-users >>> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > howto/links/examples >>> > > > >>> > > > If I want DNS caching, should i use the new stable-6.2.0 or the >>> > > > older devel-6.3.6... ? >>> > > > Can see this feature mentioned here: >>> > > > http://rsyslog.com/features/ >>> > > > or here: >>> > > > http://rsyslog.com/project-status/ >>> > > > >>> > > > >>> > > > Thanks in advance :-) ! >>> > > > ~maymann >>> > > > >>> > > > 2012/1/14 >>> > > > >>> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > > >>> > > > > Hi David, >>> > > > >> >>> > > > >> thanks for this...this is super info...:-) ! >>> > > > >> If I have to create different logs per host, will this be the a >>> > > > valid >>> > > > >> configuration: >>> > > > >> $template >>> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? >>> > > > >> *.* -?DynaFile_messages >>> > > > >> $template >>> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? >>> > > > >> *.* -?DynaFile_secure >>> > > > >> $template >>> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? >>> > > > >> *.* -?DynaFile_auth.log >>> > > > >> >>> > > > > >>> > > > > I believe so. >>> > > > > >>> > > > > >>> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or do >>> I >>> > > > have to >>> > > > >> create every hosts dir upfront... ? >>> > > > >> >>> > > > > >>> > > > > it will create it for you (make sure it's running with the >>> > > > appropriate >>> > > > > permissions, if you have rsyslog configured to drop privileges, >>> > > > > the >>> > > > lower >>> > > > > privileges need the ability to create the directories) >>> > > > > >>> > > > > >>> > > > > 2. Is DNS caching default enabled or do I have to enable this >>> > > > somewhere >>> > > > >> first...? >>> > > > >> >>> > > > > >>> > > > > I don't know, I haven't had a chance to look into that yet. >>> > > > > >>> > > > > David Lang >>> > > > > >>> > > > > >>> > > > >> Thanks in advance :-) ! >>> > > > >> ~maymann >>> > > > >> >>> > > > >> >>> > > > >> 2012/1/14 >>> > > > >> >>> > > > >> http://rsyslog.com/article60/ >>> > > > >>> >>> > > > >>> David Lang >>> > > > >>> >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: >>> > > > >>> >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 >>> > > > >>> >>> > > > >>>> From: Michael Maymann >>> > > > >>>> To: rsyslog-users , david at lang.hm, >>> > > > >>>> Michael Maymann >>> > > > >>>> >>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>> howto/links/examples >>> > > > >>>> >>> > > > >>>> Hi David, >>> > > > >>>> >>> > > > >>>> thanks for you kind reply...:-) ! >>> > > > >>>> --- >>> > > > >>>> This didn't seem to get through to the archives for some >>> > > > reason...: >>> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** >>> > > > >>>> >>> > > > January/thread.html< >>> http://lists.adiscon.net/**pipermail/rsyslog/201 >>> > > > 2- >>> > > > **January/thread.html> >>> > > > >>>> >> > > > January/thread.** >>> > > > >>>> html>> > > > January/thread.html> >>> > > > >>>> > >>> > > > >>>> >>> > > > >>>> Hope I will not dobbel-post... >>> > > > >>>> --- >>> > > > >>>> I don't use syslog-relays, so this will not cause me any >>> problems. >>> > > > >>>> Don't actually know what version we are running - can see this >>> > > > Monday >>> > > > >>>> morning though... Thanks for this hint... will upgrade to 6.2 >>> > > > >>>> if >>> > > > not >>> > > > >>>> already then. >>> > > > >>>> I have to configure this into a already running live >>> production >>> > > > system - >>> > > > >>>> our previous syslog-admin left...:-(. >>> > > > >>>> Could I perhaps ask you to be so kind as to give an >>> > > > >>>> configuration example of how this is done, if I ask really >>> > > > >>>> nicely... :-) ? >>> > > > >>>> >>> > > > >>>> Thanks in advance :-) ! >>> > > > >>>> ~maymann >>> > > > >>>> >>> > > > >>>> 2012/1/13 >>> > > > >>>> >>> > > > >>>> you need to be aware that doing the DNS queries is rather >>> > > > expensive >>> > > > >>>> >>> > > > >>>>> (although I think I saw a comment that in the very latest 6.2 >>> > > > version >>> > > > >>>>> there >>> > > > >>>>> may now be a DNS cache that will drastically help) >>> > > > >>>>> >>> > > > >>>>> you would need to create a template with FROMHOST in it and >>> > > > >>>>> use >>> > > > that as >>> > > > >>>>> the filename to write to (look for dynafile in the >>> > > > >>>>> documentation) >>> > > > >>>>> >>> > > > >>>>> note that if you are relaying logs from one machine to >>> > > > >>>>> another, >>> > > > only >>> > > > >>>>> the >>> > > > >>>>> first machine will see the true source in FROMHOST, machines >>> > > > after that >>> > > > >>>>> will only see the relay box. >>> > > > >>>>> >>> > > > >>>>> let me know if this doesn't give you enough clues to learn >>> how >>> > > > >>>>> to >>> > > > do >>> > > > >>>>> this. >>> > > > >>>>> >>> > > > >>>>> David Lang >>> > > > >>>>> >>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: >>> > > > >>>>> >>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 >>> > > > >>>>> >>> > > > >>>>> From: Michael Maymann >>> > > > >>>>>> Reply-To: rsyslog-users >>> > > > >>>>>> To: rsyslog at lists.adiscon.com >>> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir >>> > > > >>>>>> howto/links/examples >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from DNS >>> > > > >>>>>> and >>> > > > not >>> > > > >>>>>> from >>> > > > >>>>>> syslog-info hostname. >>> > > > >>>>>> We are getting a lot of weird logfiles as some applications >>> > > > >>>>>> are >>> > > > not >>> > > > >>>>>> including the hostname as the first parameter in the syslog- >>> > > > entries, >>> > > > >>>>>> e.g.: >>> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 >>> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 >>> > > > >>>>>> [set_host_compat_list] >>> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command is >>> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd >>> > "SetUserData >>> > > > >>>>>> HostCompatList text >>> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... >>> > > > >>>>>> >>> > > > >>>>>> Thanks in advance :-) ! >>> > > > >>>>>> ~maymann >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>> 2012/1/13 Michael Maymann >>> > > > >>>>>> >>> > > > >>>>>> Hi List, >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>>>> I'm new to rsyslog/syslog in general. >>> > > > >>>>>>> >>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. >>> > > > >>>>>>> Preferably I would like a FQDN.log file for each host (or a >>> > > > FQDN-dir >>> > > > >>>>>>> containing logs from this host if more logfiles per host >>> are >>> > > > best >>> > > > >>>>>>> practice)... >>> > > > >>>>>>> >>> > > > >>>>>>> Can anyone give me an example of (or link to) best practice >>> > > > >>>>>>> of >>> > > > this >>> > > > >>>>>>> kind >>> > > > >>>>>>> of setup. >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> Thanks in advance :-) ! >>> > > > >>>>>>> >>> > > > >>>>>>> ~maymann >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>>> >>> > > > >>>>>>> rsyslog mailing list >>> > > > >>>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >> > > > **services/ >>> > > > >>>>>> >> > > > services/ >>> > > > >>>>>> > >>> > > > >>>>>> >>> > > > >>>>>>> >>> > > > >>>>>>> >>> > > > >>>>>> >>> > ______________________________******_________________ >>> > > > >>>>>> >>> > > > >>>>>> rsyslog mailing list >>> > > > >>>>> >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< >>> http://lists >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >> http://list >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> >>> > > > >>>>> >>> > > > >> http://lists. >>> > > > adi >>> > > > scon.net/mailman/listinfo/rsyslog> >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> http://www.rsyslog.com/******professional- >>> > > > services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >> > > > **services/ >>> > > > >>>>> >> > > > services/ >>> > > > >>>>> > >>> > > > >>>>> >>> > > > >>>>>> >>> > > > >>>>>> >>> > > > >>>>> >>> > > > >>>>> >>> > > > >>>> >>> > > > >> >>> > > > _______________________________________________ >>> > > > rsyslog mailing list >>> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > > http://www.rsyslog.com/professional-services/ >>> > > _______________________________________________ >>> > > rsyslog mailing list >>> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > > http://www.rsyslog.com/professional-services/ >>> > > >>> > _______________________________________________ >>> > rsyslog mailing list >>> > http://lists.adiscon.net/mailman/listinfo/rsyslog >>> > http://www.rsyslog.com/professional-services/ >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> >> >> > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ From michael at maymann.org Thu Jan 19 11:24:13 2012 From: michael at maymann.org (Michael Maymann) Date: Thu, 19 Jan 2012 11:24:13 +0100 Subject: [rsyslog] Syslogging FQDN logfile/logdir howto/links/examples In-Reply-To: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> References: <00d501ccd618$c744d54b$100013ac@intern.adiscon.com> Message-ID: Hi, Got it installed...: yum install zlib-devel.x86_64 yum install byacc.x86_64 did the trick for me... Thanks for your help Rainer...:-) ! Trying to look through the man-page and googling a bit... 1. do I have to enable DNS cache somewhere, or is this feature on by default ? 2. Is there some v6.3.6 version config examples somewhere. I would need to: a. generally have logfiles in a /NFS_logdir/FQDN/equial_to_local_logfile_name (e.g. messages) b. have maillog from mailcluster=host001 (host001a+host001b) being merged in a single /NFS_logdir/host001/maillog file Thanks in advance :-) ! ~maymann 2012/1/18 Rainer Gerhards > Zlib-dev is missing, looks like configure did not catch that. > > Rainer > > Michael Maymann hat geschrieben:The > --libdir=/usr/lib64 option did the trick...:-): > > cd libestr > > ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr > > make > > make install > > cd .. > > cd libee/ > > ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr > > make > > make install > > cd .. > > cd rsyslog-6.3.6 > > ./configure --libdir=/usr/lib64 --includedir=/usr/include --prefix=/usr > > [root at MyServer rsyslog-6.3.6]# echo $? > > 0 > > [root at MyServer rsyslog-6.3.6]# make > > make all-recursive > > make[1]: Entering directory `/root/rsyslog-6.3.6' > > Making all in doc > > make[2]: Entering directory `/root/rsyslog-6.3.6/doc' > > make[2]: Nothing to be done for `all'. > > make[2]: Leaving directory `/root/rsyslog-6.3.6/doc' > > Making all in runtime > > make[2]: Entering directory `/root/rsyslog-6.3.6/runtime' > > CC librsyslog_la-rsyslog.lo > > In file included from stream.h:72, > > from obj.h:50, > > from rsyslog.h:474, > > from rsyslog.c:63: > > zlibw.h:28:18: error: zlib.h: No such file or directory > > In file included from stream.h:72, > > from obj.h:50, > > from rsyslog.h:474, > > from rsyslog.c:63: > > zlibw.h:32: error: expected ')' before 'strm' > > zlibw.h:33: error: expected ';' before 'int' > > In file included from obj.h:50, > > from rsyslog.h:474, > > from rsyslog.c:63: > > stream.h:123: error: expected specifier-qualifier-list before 'Bytef' > > make[2]: *** [librsyslog_la-rsyslog.lo] Error 1 > > make[2]: Leaving directory `/root/rsyslog-6.3.6/runtime' > > make[1]: *** [all-recursive] Error 1 > > make[1]: Leaving directory `/root/rsyslog-6.3.6' > make: *** [all] Error 2 > > So both libestr and linee in installed and ./configure in rsyslog dir ran > without errors, but now make is causing me problems..., anyone who know how > to solve this ? > > > Thanks in advance :-) ! > ~maymann > > 2012/1/17 Michael Maymann > > > Tried to delete all installed and follow this guide: > > http://www.liblognorm.com/help/first-steps-using-liblognorm/ > > same problem, when [root at MyServer libee]# ./configure --libdir=/usr/lib > > --includedir=/usr/include: > > ... > > checking for LIBESTR... configure: error: Package requirements (libestr > >= > > 0.0.0) were not met: > > > > No package 'libestr' found > > > > Consider adjusting the PKG_CONFIG_PATH environment variable if you > > installed software in a non-standard prefix. > > > > Alternatively, you may set the environment variables LIBESTR_CFLAGS > > and LIBESTR_LIBS to avoid the need to call pkg-config. > > See the pkg-config man page for more details. > > > > > > Thanks in advance :-) ! > > ~maymann > > > > > > 2012/1/17 Michael Maymann > > > >> Hi Rainer, > >> > >> yes, rsyslog-6.3.6 needs libestr and libee...: > >> I installed libestr just fine: > >> [root at MyServer pkgconfig]# locate libestr > >> /root/libestr-0.1.2 > >> /root/libestr-0.1.2.tar.gz > >> /root/libestr-0.1.2/AUTHORS > >> /root/libestr-0.1.2/COPYING > >> /root/libestr-0.1.2/ChangeLog > >> /root/libestr-0.1.2/INSTALL > >> /root/libestr-0.1.2/Makefile > >> /root/libestr-0.1.2/Makefile.am > >> /root/libestr-0.1.2/Makefile.in > >> /root/libestr-0.1.2/NEWS > >> /root/libestr-0.1.2/README > >> /root/libestr-0.1.2/aclocal.m4 > >> /root/libestr-0.1.2/compile > >> /root/libestr-0.1.2/config.guess > >> /root/libestr-0.1.2/config.h > >> /root/libestr-0.1.2/config.h.in > >> /root/libestr-0.1.2/config.log > >> /root/libestr-0.1.2/config.status > >> /root/libestr-0.1.2/config.sub > >> /root/libestr-0.1.2/configure > >> /root/libestr-0.1.2/configure.ac > >> /root/libestr-0.1.2/depcomp > >> /root/libestr-0.1.2/include > >> /root/libestr-0.1.2/install-sh > >> /root/libestr-0.1.2/libestr.pc > >> /root/libestr-0.1.2/libestr.pc.in > >> /root/libestr-0.1.2/libtool > >> /root/libestr-0.1.2/ltmain.sh > >> /root/libestr-0.1.2/m4 > >> /root/libestr-0.1.2/missing > >> /root/libestr-0.1.2/src > >> /root/libestr-0.1.2/stamp-h1 > >> /root/libestr-0.1.2/include/Makefile > >> /root/libestr-0.1.2/include/Makefile.am > >> /root/libestr-0.1.2/include/Makefile.in > >> /root/libestr-0.1.2/include/libestr.h > >> /root/libestr-0.1.2/m4/libtool.m4 > >> /root/libestr-0.1.2/m4/ltoptions.m4 > >> /root/libestr-0.1.2/m4/ltsugar.m4 > >> /root/libestr-0.1.2/m4/ltversion.m4 > >> /root/libestr-0.1.2/m4/lt~obsolete.m4 > >> /root/libestr-0.1.2/src/.deps > >> /root/libestr-0.1.2/src/.libs > >> /root/libestr-0.1.2/src/Makefile > >> /root/libestr-0.1.2/src/Makefile.am > >> /root/libestr-0.1.2/src/Makefile.in > >> /root/libestr-0.1.2/src/libestr.c > >> /root/libestr-0.1.2/src/libestr.la > >> /root/libestr-0.1.2/src/libestr_la-libestr.lo > >> /root/libestr-0.1.2/src/libestr_la-libestr.o > >> /root/libestr-0.1.2/src/libestr_la-string.lo > >> /root/libestr-0.1.2/src/libestr_la-string.o > >> /root/libestr-0.1.2/src/string.c > >> /root/libestr-0.1.2/src/.deps/libestr_la-libestr.Plo > >> /root/libestr-0.1.2/src/.deps/libestr_la-string.Plo > >> /root/libestr-0.1.2/src/.libs/libestr.a > >> /root/libestr-0.1.2/src/.libs/libestr.la > >> /root/libestr-0.1.2/src/.libs/libestr.lai > >> /root/libestr-0.1.2/src/.libs/libestr.so > >> /root/libestr-0.1.2/src/.libs/libestr.so.0 > >> /root/libestr-0.1.2/src/.libs/libestr.so.0.0.0 > >> /root/libestr-0.1.2/src/.libs/libestr_la-libestr.o > >> /root/libestr-0.1.2/src/.libs/libestr_la-string.o > >> /usr/local/include/libestr.h > >> /usr/local/lib/libestr.a > >> /usr/local/lib/libestr.la > >> /usr/local/lib/libestr.so > >> /usr/local/lib/libestr.so.0 > >> /usr/local/lib/libestr.so.0.0.0 > >> /usr/local/lib/pkgconfig/libestr.pc > >> > >> I try to install libee: > >> [root at MyServer libee-0.3.2]# ./configure > >> LIBESTR_CFLAGS=/usr/local/include LIBESTR_LIBS=/usr/local/lib > >> checking for a BSD-compatible install... /usr/bin/install -c > >> checking whether build environment is sane... yes > >> checking for a thread-safe mkdir -p... /bin/mkdir -p > >> checking for gawk... gawk > >> checking whether make sets $(MAKE)... yes > >> checking for gcc... gcc > >> checking whether the C compiler works... yes > >> checking for C compiler default output file name... a.out > >> checking for suffix of executables... > >> checking whether we are cross compiling... no > >> checking for suffix of object files... o > >> checking whether we are using the GNU C compiler... yes > >> checking whether gcc accepts -g... yes > >> checking for gcc option to accept ISO C89... none needed > >> checking for style of include used by make... GNU > >> checking dependency style of gcc... gcc3 > >> checking whether gcc and cc understand -c and -o together... yes > >> checking build system type... x86_64-unknown-linux-gnu > >> checking host system type... x86_64-unknown-linux-gnu > >> checking how to print strings... printf > >> checking for a sed that does not truncate output... /bin/sed > >> checking for grep that handles long lines and -e... /bin/grep > >> checking for egrep... /bin/grep -E > >> checking for fgrep... /bin/grep -F > >> checking for ld used by gcc... /usr/bin/ld > >> checking if the linker (/usr/bin/ld) is GNU ld... yes > >> checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B > >> checking the name lister (/usr/bin/nm -B) interface... BSD nm > >> checking whether ln -s works... yes > >> checking the maximum length of command line arguments... 1966080 > >> checking whether the shell understands some XSI constructs... yes > >> checking whether the shell understands "+="... yes > >> checking for /usr/bin/ld option to reload object files... -r > >> checking for objdump... objdump > >> checking how to recognize dependent libraries... pass_all > >> checking for ar... ar > >> checking for strip... strip > >> checking for ranlib... ranlib > >> checking command to parse /usr/bin/nm -B output from gcc object... ok > >> checking how to run the C preprocessor... gcc -E > >> checking for ANSI C header files... yes > >> checking for sys/types.h... yes > >> checking for sys/stat.h... yes > >> checking for stdlib.h... yes > >> checking for string.h... yes > >> checking for memory.h... yes > >> checking for strings.h... yes > >> checking for inttypes.h... yes > >> checking for stdint.h... yes > >> checking for unistd.h... yes > >> checking for dlfcn.h... yes > >> checking for objdir... .libs > >> checking if gcc supports -fno-rtti -fno-exceptions... no > >> checking for gcc option to produce PIC... -fPIC -DPIC > >> checking if gcc PIC flag -fPIC -DPIC works... yes > >> checking if gcc static flag -static works... no > >> checking if gcc supports -c -o file.o... yes > >> checking if gcc supports -c -o file.o... (cached) yes > >> checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports > >> shared libraries... yes > >> checking whether -lc should be explicitly linked in... no > >> checking dynamic linker characteristics... GNU/Linux ld.so > >> checking how to hardcode library paths into programs... immediate > >> checking whether stripping libraries is possible... yes > >> checking if libtool supports shared libraries... yes > >> checking whether to build shared libraries... yes > >> checking whether to build static libraries... yes > >> checking for stdlib.h... (cached) yes > >> checking for GNU libc compatible malloc... yes > >> checking for pkg-config... /usr/bin/pkg-config > >> checking pkg-config is at least version 0.9.0... yes > >> checking for LIBESTR... yes > >> configure: creating ./config.status > >> config.status: creating Makefile > >> config.status: creating libee.pc > >> config.status: creating src/Makefile > >> config.status: creating include/Makefile > >> config.status: creating include/libee/Makefile > >> config.status: creating tests/Makefile > >> config.status: creating config.h > >> config.status: config.h is unchanged > >> config.status: executing depfiles commands > >> config.status: executing libtool commands > >> ***************************************************** > >> libee will be compiled with the following settings: > >> > >> Debug mode enabled: no > >> Testbench enabled: yes > >> [root at MyServer libee-0.3.2]# echo $? > >> 0 > >> [root at MyServer libee-0.3.2]# make > >> make all-recursive > >> make[1]: Entering directory `/root/libee-0.3.2' > >> Making all in tests > >> make[2]: Entering directory `/root/libee-0.3.2/tests' > >> make[2]: Nothing to be done for `all'. > >> make[2]: Leaving directory `/root/libee-0.3.2/tests' > >> Making all in include > >> make[2]: Entering directory `/root/libee-0.3.2/include' > >> Making all in libee > >> make[3]: Entering directory `/root/libee-0.3.2/include/libee' > >> make[3]: Nothing to be done for `all'. > >> make[3]: Leaving directory `/root/libee-0.3.2/include/libee' > >> make[3]: Entering directory `/root/libee-0.3.2/include' > >> make[3]: Nothing to be done for `all-am'. > >> make[3]: Leaving directory `/root/libee-0.3.2/include' > >> make[2]: Leaving directory `/root/libee-0.3.2/include' > >> Making all in src > >> make[2]: Entering directory `/root/libee-0.3.2/src' > >> CC libee_la-ctx.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-tag.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-event.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-value.lo > >> value.c: In function 'ee_newValue': > >> value.c:37: warning: unused parameter 'ctx' > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-tagbucket.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-field.lo > >> field.c: In function 'ee_getFieldValueAsStr': > >> field.c:181: warning: 'str' may be used uninitialized in this function > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-fieldbucket.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-primitivetype.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-int_dec.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-apache_dec.lo > >> apache_dec.c: In function 'ee_newApache': > >> apache_dec.c:37: warning: unused parameter 'ctx' > >> apache_dec.c: In function 'ee_apacheAddName': > >> apache_dec.c:71: warning: unused parameter 'ctx' > >> apache_dec.c: In function 'processLn': > >> apache_dec.c:205: warning: unused variable 'value' > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-syslog_enc.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-json_enc.lo > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-csv_enc.lo > >> csv_enc.c: In function 'ee_AddName': > >> csv_enc.c:66: warning: unused parameter 'ctx' > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CC libee_la-xml_enc.lo > >> xml_enc.c: In function 'ee_addValue_XML': > >> xml_enc.c:60: warning: unused variable 'j' > >> xml_enc.c:59: warning: unused variable 'numbuf' > >> xml_enc.c: At top level: > >> xml_enc.c:40: warning: 'hexdigit' defined but not used > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CCLD libee.la > >> CC convert-convert.o > >> gcc: /usr/local/include: linker input file unused because linking not > done > >> CCLD convert > >> /usr/local/lib: file not recognized: Is a directory > >> collect2: ld returned 1 exit status > >> make[2]: *** [convert] Error 1 > >> make[2]: Leaving directory `/root/libee-0.3.2/src' > >> make[1]: *** [all-recursive] Error 1 > >> make[1]: Leaving directory `/root/libee-0.3.2' > >> make: *** [all] Error 2 > >> [root at MyServer libee-0.3.2]# echo $? > >> 2 > >> > >> It must be my LIBESTR_CFLAGS and LIBESTR_LIBS being wrong - do you know > >> how to solve this ? > >> > >> > >> > >> Thanks in advance :-) ! > >> ~maymann > >> > >> 2012/1/16 Rainer Gerhards > >> > >>> > -----Original Message----- > >>> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > >>> > Sent: Monday, January 16, 2012 10:48 AM > >>> > To: rsyslog-users > >>> > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> howto/links/examples > >>> > > >>> > Hi Rainer, > >>> > > >>> > Thanks for your reply. > >>> > How do I install 6.3.6 on RHEL6 easiest ? > >>> > >>> I have no specific instructions. Just grab the sources and compile, I'd > >>> say > >>> ;) Note, however, that you need to install libestr and probably libee > >>> first. > >>> > >>> Rainer > >>> > > >>> > Thanks in advance :-)! > >>> > ~maymann > >>> > > >>> > 2012/1/16 Rainer Gerhards > >>> > > >>> > > The cache is available since 6.3.1, so you need to go for the devel > >>> > > version. > >>> > > A good place to check those things is the ChangeLog itself, here is > >>> > > the current one: > >>> > > > >>> > > > >>> > > > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f=ChangeLog;h=b42a8004ed8 > >>> > > 575d085 > >>> > > a0fcf48f71339154813971< > >>> http://git.adiscon.com/?p=rsyslog.git;a=blob;f= > >>> > > ChangeLog;h=b42a8004ed8575d085%0Aa0fcf48f71339154813971> > >>> > > ;hb=HEAD > >>> > > > >>> > > Note that v6-devel is almost as stable as v6-stable except for the > >>> > > config read phase at startup. > >>> > > > >>> > > HTH > >>> > > Rainer > >>> > > > >>> > > > -----Original Message----- > >>> > > > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog- > >>> > > > bounces at lists.adiscon.com] On Behalf Of Michael Maymann > >>> > > > Sent: Monday, January 16, 2012 8:57 AM > >>> > > > To: david at lang.hm > >>> > > > Cc: rsyslog-users > >>> > > > Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> > > > howto/links/examples > >>> > > > > >>> > > > If I want DNS caching, should i use the new stable-6.2.0 or the > >>> > > > older devel-6.3.6... ? > >>> > > > Can see this feature mentioned here: > >>> > > > http://rsyslog.com/features/ > >>> > > > or here: > >>> > > > http://rsyslog.com/project-status/ > >>> > > > > >>> > > > > >>> > > > Thanks in advance :-) ! > >>> > > > ~maymann > >>> > > > > >>> > > > 2012/1/14 > >>> > > > > >>> > > > > On Sat, 14 Jan 2012, Michael Maymann wrote: > >>> > > > > > >>> > > > > Hi David, > >>> > > > >> > >>> > > > >> thanks for this...this is super info...:-) ! > >>> > > > >> If I have to create different logs per host, will this be the > a > >>> > > > valid > >>> > > > >> configuration: > >>> > > > >> $template > >>> > DynaFile_messages,?/logfiles_**on_nfs/%HOSTNAME%/messages? > >>> > > > >> *.* -?DynaFile_messages > >>> > > > >> $template > >>> > DynaFile_secure,?/logfiles_on_**nfs/%HOSTNAME%/secure? > >>> > > > >> *.* -?DynaFile_secure > >>> > > > >> $template > >>> > DynaFile_auth.log,?/logfiles_**on_nfs/%HOSTNAME%/auth.log? > >>> > > > >> *.* -?DynaFile_auth.log > >>> > > > >> > >>> > > > > > >>> > > > > I believe so. > >>> > > > > > >>> > > > > > >>> > > > > 1. Will rsyslog automatically create the %HOSTNAME% dir's or > do > >>> I > >>> > > > have to > >>> > > > >> create every hosts dir upfront... ? > >>> > > > >> > >>> > > > > > >>> > > > > it will create it for you (make sure it's running with the > >>> > > > appropriate > >>> > > > > permissions, if you have rsyslog configured to drop privileges, > >>> > > > > the > >>> > > > lower > >>> > > > > privileges need the ability to create the directories) > >>> > > > > > >>> > > > > > >>> > > > > 2. Is DNS caching default enabled or do I have to enable this > >>> > > > somewhere > >>> > > > >> first...? > >>> > > > >> > >>> > > > > > >>> > > > > I don't know, I haven't had a chance to look into that yet. > >>> > > > > > >>> > > > > David Lang > >>> > > > > > >>> > > > > > >>> > > > >> Thanks in advance :-) ! > >>> > > > >> ~maymann > >>> > > > >> > >>> > > > >> > >>> > > > >> 2012/1/14 > >>> > > > >> > >>> > > > >> http://rsyslog.com/article60/ > >>> > > > >>> > >>> > > > >>> David Lang > >>> > > > >>> > >>> > > > >>> On Sat, 14 Jan 2012, Michael Maymann wrote: > >>> > > > >>> > >>> > > > >>> Date: Sat, 14 Jan 2012 07:23:57 +0100 > >>> > > > >>> > >>> > > > >>>> From: Michael Maymann > >>> > > > >>>> To: rsyslog-users , > david at lang.hm, > >>> > > > >>>> Michael Maymann > >>> > > > >>>> > >>> > > > >>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> > > > >>>> howto/links/examples > >>> > > > >>>> > >>> > > > >>>> Hi David, > >>> > > > >>>> > >>> > > > >>>> thanks for you kind reply...:-) ! > >>> > > > >>>> --- > >>> > > > >>>> This didn't seem to get through to the archives for some > >>> > > > reason...: > >>> > > > >>>> http://lists.adiscon.net/****pipermail/rsyslog/2012-**** > >>> > > > >>>> > >>> > > > January/thread.html< > >>> http://lists.adiscon.net/**pipermail/rsyslog/201 > >>> > > > 2- > >>> > > > **January/thread.html> > >>> > > > >>>> >>> > > > January/thread.** > >>> > > > >>>> html >>> > > > January/thread.html> > >>> > > > >>>> > > >>> > > > >>>> > >>> > > > >>>> Hope I will not dobbel-post... > >>> > > > >>>> --- > >>> > > > >>>> I don't use syslog-relays, so this will not cause me any > >>> problems. > >>> > > > >>>> Don't actually know what version we are running - can see > this > >>> > > > Monday > >>> > > > >>>> morning though... Thanks for this hint... will upgrade to > 6.2 > >>> > > > >>>> if > >>> > > > not > >>> > > > >>>> already then. > >>> > > > >>>> I have to configure this into a already running live > >>> production > >>> > > > system - > >>> > > > >>>> our previous syslog-admin left...:-(. > >>> > > > >>>> Could I perhaps ask you to be so kind as to give an > >>> > > > >>>> configuration example of how this is done, if I ask really > >>> > > > >>>> nicely... :-) ? > >>> > > > >>>> > >>> > > > >>>> Thanks in advance :-) ! > >>> > > > >>>> ~maymann > >>> > > > >>>> > >>> > > > >>>> 2012/1/13 > >>> > > > >>>> > >>> > > > >>>> you need to be aware that doing the DNS queries is rather > >>> > > > expensive > >>> > > > >>>> > >>> > > > >>>>> (although I think I saw a comment that in the very latest > 6.2 > >>> > > > version > >>> > > > >>>>> there > >>> > > > >>>>> may now be a DNS cache that will drastically help) > >>> > > > >>>>> > >>> > > > >>>>> you would need to create a template with FROMHOST in it and > >>> > > > >>>>> use > >>> > > > that as > >>> > > > >>>>> the filename to write to (look for dynafile in the > >>> > > > >>>>> documentation) > >>> > > > >>>>> > >>> > > > >>>>> note that if you are relaying logs from one machine to > >>> > > > >>>>> another, > >>> > > > only > >>> > > > >>>>> the > >>> > > > >>>>> first machine will see the true source in FROMHOST, > machines > >>> > > > after that > >>> > > > >>>>> will only see the relay box. > >>> > > > >>>>> > >>> > > > >>>>> let me know if this doesn't give you enough clues to learn > >>> how > >>> > > > >>>>> to > >>> > > > do > >>> > > > >>>>> this. > >>> > > > >>>>> > >>> > > > >>>>> David Lang > >>> > > > >>>>> > >>> > > > >>>>> On Fri, 13 Jan 2012, Michael Maymann wrote: > >>> > > > >>>>> > >>> > > > >>>>> Date: Fri, 13 Jan 2012 14:43:06 +0100 > >>> > > > >>>>> > >>> > > > >>>>> From: Michael Maymann > >>> > > > >>>>>> Reply-To: rsyslog-users > >>> > > > >>>>>> To: rsyslog at lists.adiscon.com > >>> > > > >>>>>> Subject: Re: [rsyslog] Syslogging FQDN logfile/logdir > >>> > > > >>>>>> howto/links/examples > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> Furthermore: would it be possible to validate FQDN from > DNS > >>> > > > >>>>>> and > >>> > > > not > >>> > > > >>>>>> from > >>> > > > >>>>>> syslog-info hostname. > >>> > > > >>>>>> We are getting a lot of weird logfiles as some > applications > >>> > > > >>>>>> are > >>> > > > not > >>> > > > >>>>>> including the hostname as the first parameter in the > syslog- > >>> > > > entries, > >>> > > > >>>>>> e.g.: > >>> > > > >>>>>> Dec 16 11:47:40 x002 |grep FAILED#012#01212/16/11 09:47:10 > >>> > > > >>>>>> [issue_cmd ] STATUS: 1#012#01212/16/11 09:47:10 > >>> > > > >>>>>> [issue_cmd ] RESULT:#012#01212/16/11 09:47:10 > >>> > > > >>>>>> [issue_cmd ] #012#01212/16/11 09:47:10 > >>> > > > >>>>>> [set_host_compat_list] > >>> > > > >>>>>> #012#01212/16/11 09:47:10 [issue_cli_cmd ] command > is > >>> > > > >>>>>> '/opt/vmware/aam/bin/ftcli -domain vmware -cmd > >>> > "SetUserData > >>> > > > >>>>>> HostCompatList text > >>> > > > >>>>>> /tmp/hostCompatList"'#012#******01212/16/11 09:47:40 > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> Would be nice to validate FQDN from sender DNS query... > >>> > > > >>>>>> > >>> > > > >>>>>> Thanks in advance :-) ! > >>> > > > >>>>>> ~maymann > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>> 2012/1/13 Michael Maymann > >>> > > > >>>>>> > >>> > > > >>>>>> Hi List, > >>> > > > >>>>>> > >>> > > > >>>>>> > >>> > > > >>>>>>> I'm new to rsyslog/syslog in general. > >>> > > > >>>>>>> > >>> > > > >>>>>>> I would like to syslog from all my 100+ network devices. > >>> > > > >>>>>>> Preferably I would like a FQDN.log file for each host > (or a > >>> > > > FQDN-dir > >>> > > > >>>>>>> containing logs from this host if more logfiles per host > >>> are > >>> > > > best > >>> > > > >>>>>>> practice)... > >>> > > > >>>>>>> > >>> > > > >>>>>>> Can anyone give me an example of (or link to) best > practice > >>> > > > >>>>>>> of > >>> > > > this > >>> > > > >>>>>>> kind > >>> > > > >>>>>>> of setup. > >>> > > > >>>>>>> > >>> > > > >>>>>>> > >>> > > > >>>>>>> Thanks in advance :-) ! > >>> > > > >>>>>>> > >>> > > > >>>>>>> ~maymann > >>> > > > >>>>>>> > >>> > > > >>>>>>> > >>> > ______________________________******_________________ > >>> > > > >>>>>>> > >>> > > > >>>>>>> rsyslog mailing list > >>> > > > >>>>>> > >>> > > > http://lists.adiscon.net/******mailman/listinfo/rsyslog< > >>> http://lists > >>> > > > .ad iscon.net/****mailman/listinfo/rsyslog> > >>> > > > >>>>>> > >>> > > > >>> http://list > >>> > > > s.a discon.net/**mailman/listinfo/rsyslog> > >>> > > > >>>>>> > > >>> > > > >>>>>> > >>> > > > >>> http://list > >>> > > > s.a discon.net/mailman/**listinfo/rsyslog> > >>> > > > >>>>>> > >>> > > > >>> http://lists. > >>> > > > adi > >>> > > > scon.net/mailman/listinfo/rsyslog> > >>> > > > >>>>>> > > >>> > > > >>>>>> > >>> > > > >>>>>>> > >>> > > > >>>>>>> http://www.rsyslog.com/******professional- > >>> > > > services/ > >>> > > > >>>>>> >>> > > > services/<