[rsyslog-notify] Forum Thread: Re: TSV data into mongodb - (Mode 'reply')

[noreply at adiscon.com]

User: toddaa 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=25568#p25568

Message: 
----------
dlang,

I turned on some debug and found this in the output:

[code:1wzxguvu]2596.319297687:7f032ce67700: mmnormalize
generated: {"originalmsg":
"INFO\tstream\tplay\t2015-05-12\t10:56:27\t409963998\tclientip\t-\t3633\t3497\t0.081\t081808\t1\t0\t0\t0\t133581414\t2360.859\t081808\t-\trtmp\trtmp://host/app/\thttps://host1/script/player.swf\tapp\t1431442560",
"unparsed-data":
"INFO\tstream\tplay\t2015-05-12\t10:56:27\t409963998\tclientip\t-\t3633\t3497\t0.081\t081808\t1\t0\t0\t0\t133581414\t2360.859\t081808\t-\trtmp\trtmp://host/app/\thttps://host1/script/player.swf\tapp\t1431442560"}[/code:1wzxguvu]

So it looks like mmnormalize is working...maybe...but I end up with only
the variables "originalmsg" and "unparsed-data".  If I change my template
to one of these variables I get the output in the custom file I am writing,
but its same as the original %msg% variable.  Below is my config.  I've
stripped out all the mongodb stuff just to tackle one thing at a time.  I
think if I can get it to write to a file first then getting to mongodb
should be fairly easy.  I think my problem is still the mmnormalize rule
and learning how to use each variable that is created.

[code:1wzxguvu]# rsyslog configuration file
# note that most of this config file uses old-style format,
# because it is well-known AND quite suitable for simple cases
# like we have with the default config. For more advanced
# things, RainerScript configuration is suggested.

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see
http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

module(load="imuxsock") # provides support for local system logging
(e.g. via logger command)
module(load="imklog")   # provides kernel logging support (previously done
by rklogd)
#module(load"immark")  # provides --MARK-- message capability

# Provides UDP syslog reception
# for parameters see http://www.rsyslog.com/doc/imudp.html
#module(load="imudp") # needs to be done just once
#input(type="imudp" port="514")

# Provides TCP syslog reception
# for parameters see http://www.rsyslog.com/doc/imtcp.html
module(load="imtcp") # needs to be done just once
input(type="imtcp" port="514")

module(load="mmnormalize")

#### GLOBAL DIRECTIVES ####
# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is
usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

$EscapeControlCharactersOnReceive off
$mmnormalizeRuleBase /etc/rsyslog.mmnormalize.rb
local1.* :mmnormalize:

$template cee,"msg is '%msg%', %$!originalmsg%\n"
local1.*     /var/log/rsyslog_test.log;cee

$DebugFile /var/log/rsyslog_debug.log
$DebugLevel 2

#### RULES ####
#
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none               
/var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                 
/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                
:omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                         
/var/log/spooler

# Save boot messages also to boot.log
local7.*                                               
/var/log/boot.log[/code:1wzxguvu]