[rsyslog] Any good guides for generating JSON formatted log messages on rsyslog client nodes for processing on rsyslog receiver node?

David Lang david at lang.hm
Tue Sep 19 10:55:21 CEST 2017


On Mon, 18 Sep 2017, rsyslog-users-lists.adiscon.net at whyaskwhy.org wrote:

> On 9/18/2017 12:02 PM, David Lang wrote:
>> to send in JSON you need to create a custom template
>> 
>> I use something like (typed from memory, may be errors)
>> $template structured,"<%pri%>%timestamp% %hostname% %syslogtag% %$!%\n"
>
> Thanks for that. I found many examples where the templates were defined with 
> the new syntax and using the list type. Other than how the template is put 
> together piece by piece, is there an advantage of one over the other?

listingall the individual fields means that you need to know what they are :-)

including $! means you include all variables under that.

>> getting things into different variables so that the result looks reasonable 
>> takes a little more effort.
>
> Different variables on the receiving side?

No, I'm still talking about the sending side. If things aren't parsed out to 
different $! variables on the sending side, you don't have any structureto send.

>> If what you are getting is json in the msg field to start with, you can use 
>> mmjsonparse (but you may need to set the cee cookie to "" to parse things 
>> correctly)
>
> If I am sourcing the local log socket on the rsyslog client nodes, I assume 
> that they're in the older syslog format? In that case, I have to use a 
> template like the one you provided to send the content over in a format that 
> rsyslog expects?

you have two issues

1. on the sending side, you need to take the data you are given and parse it 
out

then

2. you can send the parsed data as json to the other machine.

> One part I'm not yet following is the use of either $! or all-json in the 
> templates for sending messages to a remote node. Is that object (for lack of 
> another word) populated using a template or mmjsonparse?

templates are only used when delivering messages, not to parse messages, so 
mmjsonparse of mmnormalize can extract data from the message you have been given 
into the $! address space. Once it's parsed, you then use a template to 
configure how things are output.

>> If you are getting anything else, then you need to use mmnormalize to parse 
>> the message.
>
> So mmnormalize to convert syslog messages into JSON and then use the template 
> you provided to ship in syslog format with the $! representing the parsed 
> JSON data as the trailing portion of the message?

yes.

David Lang

> Just trying to make sure I understand how $! is constructed.
>
>> on the sending system, log locally using RSYSLOG_DebugFormat and you will 
>> be able to see what is in $! (and what else is known about the message)
> Thanks. That is actually one way that I was able to see that at one point 
> during my testing that $! contained a string value for msg instead of JSON as 
> expected.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
> LIKE THAT.
>


More information about the rsyslog mailing list