[rsyslog-notify] Forum Thread: Break up huge logs into chunks before shipping them? - (Mode 'post')

[noreply at adiscon.com]

User: ZillaG 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=27181#p27181

Message: 
----------
I'm using rsyslog as my shipper to a centralized server using the
Elasticsearch/Logstash/Kibana (ELK) stack. I send the messages in a JSON
format.

"Almost" all is working well. One problem is Logstash has issues receiving
HUGE messages, e.g., stack traces. One idea is to chop these huge messages
into manageable chunks BEFORE shipping them out. Can I do this in rsyslog?
I have this configuration file for example, where the startmsg.regex line
delineates the log event nicely. How can I alter this to say, "after a
certain word or line count, send the log?"

[code:3m49kmex]input(type="imfile"
    File="/tmp/elk/pe.log*"
    Facility="local5"
    Tag="pe-log:"
    Severity="info"
   
startmsg.regex="^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}"
    escapeLF="off"
)

if $programname == 'pe-log:' then {
    action(
        type="omfwd"
        Target="10.22.123.123"
        Port="5514"
        Protocol="udp"
        template="jsonLogTemplate"
    )
    stop
}
[/code:3m49kmex]