[rsyslog] Out of memory

David Lang david at lang.hm
Wed Jan 18 12:29:19 CET 2017


On Wed, 18 Jan 2017, mostolog--- via rsyslog wrote:

> *Update*
>
> In the log below you may notice suddenly relp queue grow to 35k events 
> (that's when the forwarding relay was restarted). And -at the end- it drops 
> to 0, as the relay stop forwarding.
>
> Why rsyslog has't been able to reduce queue size from ~35k for quite a long 
> time? Is 10k the aprox. maximum ingest with our current config?

when you give us your config, please include the name= portion so we can see 
what actions match up with what in impstats

Ok, here the maxrss is increasing

at 10:37, the queue is small and maxrss is still low (8092)

at 10:42, you got a huge burst of new traffic (104878-1889=102989 messages in 
this 5 min period), but the number processed is low. see the submitted counts 
that only processed 70614-1889=68725 messages, so 35001 in the queue with meory 
usage climging to 75992

at 10:47 you had received an additional 196015-104878=91137 messages, but 
processed 162117-70614=91503 messages, gaining slightly with the queue 
dropping to 33978 (with memory usage climbing to 104684)

the fact that memory use climbed so much with the queue size dropping is odd, 
but that could have been a burst from something processing messages

at 10:52 you had received an additonal 299439-196015=103424 messages.<but 
processed 265461-162118=103343 messages, loosing a little ground with the queue 
size at 35002 (with memory usage climbing to 135732)

at 10:57 the pattern continues, with lots of messages being received and 
processed (no significant change to the queue size), but memory use continuing 
to climb.

at 11:32 the number of new messages drops significantly and that lets rsyslog 
deliver everything in the queue to catch up.

what happens after that point?

it's worth noting that the queue sizes are instananious sizes, not average 
sizes, so if you have a lot of messages that are delivered just before the stats 
run happens, it will show a lot of messages in the queue. your stats times only 
vary by a few seconds over the time of the run, so it's impossible to say if 
the ~35K queue is due to a burst of what. You could try changing the stats time 
to something much smaller to see what happens, or change it to something that's 
not an even number of minutes so that if something is generating logs every X 
minutes, you won't stay in sync with them (try something like 100 seconds for 
example)

But overall, it does look like there is a memory leak somewhere, can you run a 
copy of rsyslog somewhere that will allow you to tinker with the config 
significantly? change the output to go to a file instead of ES (using the same 
template that you are using in ES would be good), and then see what happens. If 
the memory leak stops, it's an omelasticsearch issue, if not, we can try 
tinkering with the other actions and see what makes a difference.

David Lang

>
> Wed Jan 18 09:47:57 2017: global: origin=dynstats
> ...
> Wed Jan 18 10:32:57 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 10:32:57 2017: relp: origin=core.queue size=0 enqueued=1729 full=0 
> discarded.full=0 discarded.nf=0 maxqsize=47
> Wed Jan 18 10:32:57 2017: main Q: origin=core.queue size=12 enqueued=129 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 10:37:57 2017: global: origin=dynstats
> Wed Jan 18 10:37:57 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:37:57 2017: omelasticsearch: origin=omelasticsearch 
> submitted=1889 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 10:37:57 2017: json: origin=core.action processed=1889 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:37:57 2017: norm: origin=core.action processed=1889 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:37:57 2017: elastic: origin=core.action processed=1889 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:37:57 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:37:57 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:37:57 2017: imrelp[20514]: origin=imrelp submitted=1889
> Wed Jan 18 10:37:57 2017: resource-usage: origin=impstats utime=1036000 
> stime=876000 maxrss=8092 minflt=1500 majflt=0 inblock=10 oublock=45 
> nvcsw=11377 nivcsw=129
> Wed Jan 18 10:37:57 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 10:37:57 2017: relp: origin=core.queue size=0 enqueued=1889 full=0 
> discarded.full=0 discarded.nf=0 maxqsize=47
> Wed Jan 18 10:37:57 2017: main Q: origin=core.queue size=12 enqueued=142 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 10:42:58 2017: global: origin=dynstats
> Wed Jan 18 10:42:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:42:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=70614 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 10:42:58 2017: json: origin=core.action processed=70901 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:42:58 2017: norm: origin=core.action processed=70901 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:42:58 2017: elastic: origin=core.action processed=70901 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:42:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:42:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:42:58 2017: imrelp[20514]: origin=imrelp submitted=104878
> Wed Jan 18 10:42:58 2017: resource-usage: origin=impstats utime=15748000 
> stime=11172000 maxrss=75992 minflt=18537 majflt=0 inblock=10 oublock=48 
> nvcsw=219548 nivcsw=7017
> Wed Jan 18 10:42:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 10:42:58 2017: relp: origin=core.queue size=35001 enqueued=104879 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35003
> Wed Jan 18 10:42:58 2017: main Q: origin=core.queue size=12 enqueued=155 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 10:47:58 2017: global: origin=dynstats
> Wed Jan 18 10:47:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:47:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=162118 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 10:47:58 2017: json: origin=core.action processed=163061 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:47:58 2017: norm: origin=core.action processed=163061 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:47:58 2017: elastic: origin=core.action processed=163061 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:47:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:47:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:47:58 2017: imrelp[20514]: origin=imrelp submitted=196015
> Wed Jan 18 10:47:58 2017: resource-usage: origin=impstats utime=37292000 
> stime=24932000 maxrss=104684 minflt=25667 majflt=0 inblock=10 oublock=51 
> nvcsw=496418 nivcsw=14202
> Wed Jan 18 10:47:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 10:47:58 2017: relp: origin=core.queue size=33978 enqueued=196016 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 10:47:58 2017: main Q: origin=core.queue size=3 enqueued=168 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 10:52:58 2017: global: origin=dynstats
> Wed Jan 18 10:52:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:52:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=265391 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 10:52:58 2017: json: origin=core.action processed=265461 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:52:58 2017: norm: origin=core.action processed=265461 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:52:58 2017: elastic: origin=core.action processed=265461 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:52:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:52:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:52:58 2017: imrelp[20514]: origin=imrelp submitted=299439
> Wed Jan 18 10:52:58 2017: resource-usage: origin=impstats utime=60212000 
> stime=40316000 maxrss=135732 minflt=33431 majflt=0 inblock=10 oublock=54 
> nvcsw=808532 nivcsw=22916
> Wed Jan 18 10:52:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 10:52:58 2017: relp: origin=core.queue size=35002 enqueued=299440 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 10:52:58 2017: main Q: origin=core.queue size=2 enqueued=181 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 10:57:58 2017: global: origin=dynstats
> Wed Jan 18 10:57:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:57:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=370571 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 10:57:58 2017: json: origin=core.action processed=370933 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:57:58 2017: norm: origin=core.action processed=370933 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:57:58 2017: elastic: origin=core.action processed=370933 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:57:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:57:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 10:57:58 2017: imrelp[20514]: origin=imrelp submitted=404910
> Wed Jan 18 10:57:58 2017: resource-usage: origin=impstats utime=83596000 
> stime=55228000 maxrss=167020 minflt=41286 majflt=0 inblock=10 oublock=57 
> nvcsw=1124817 nivcsw=33282
> Wed Jan 18 10:57:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 10:57:58 2017: relp: origin=core.queue size=35001 enqueued=404911 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 10:57:58 2017: main Q: origin=core.queue size=12 enqueued=194 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:02:58 2017: global: origin=dynstats
> Wed Jan 18 11:02:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:02:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=471328 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:02:58 2017: json: origin=core.action processed=472309 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:02:58 2017: norm: origin=core.action processed=472309 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:02:58 2017: elastic: origin=core.action processed=472309 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:02:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:02:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:02:58 2017: imrelp[20514]: origin=imrelp submitted=505264
> Wed Jan 18 11:02:58 2017: resource-usage: origin=impstats utime=106828000 
> stime=69648000 maxrss=196836 minflt=48810 majflt=0 inblock=10 oublock=60 
> nvcsw=1429398 nivcsw=41644
> Wed Jan 18 11:02:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:02:58 2017: relp: origin=core.queue size=33979 enqueued=505265 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:02:58 2017: main Q: origin=core.queue size=12 enqueued=207 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:07:58 2017: global: origin=dynstats
> Wed Jan 18 11:07:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:07:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=574676 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:07:58 2017: json: origin=core.action processed=574709 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:07:58 2017: norm: origin=core.action processed=574709 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:07:58 2017: elastic: origin=core.action processed=574709 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:07:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:07:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:07:58 2017: imrelp[20514]: origin=imrelp submitted=608687
> Wed Jan 18 11:07:58 2017: resource-usage: origin=impstats utime=130764000 
> stime=84716000 maxrss=227096 minflt=56388 majflt=0 inblock=10 oublock=63 
> nvcsw=1741614 nivcsw=50338
> Wed Jan 18 11:07:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:07:58 2017: relp: origin=core.queue size=35002 enqueued=608688 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:07:58 2017: main Q: origin=core.queue size=12 enqueued=220 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:12:58 2017: global: origin=dynstats
> Wed Jan 18 11:12:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:12:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=676472 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:12:58 2017: json: origin=core.action processed=677109 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:12:58 2017: norm: origin=core.action processed=677109 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:12:58 2017: elastic: origin=core.action processed=677109 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:12:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:12:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:12:58 2017: imrelp[20514]: origin=imrelp submitted=711085
> Wed Jan 18 11:12:58 2017: resource-usage: origin=impstats utime=154780000 
> stime=99132000 maxrss=257392 minflt=63988 majflt=0 inblock=10 oublock=66 
> nvcsw=2049218 nivcsw=58542
> Wed Jan 18 11:12:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:12:58 2017: relp: origin=core.queue size=35000 enqueued=711086 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:12:58 2017: main Q: origin=core.queue size=12 enqueued=233 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:17:58 2017: global: origin=dynstats
> Wed Jan 18 11:17:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:17:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=770669 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:17:58 2017: json: origin=core.action processed=771317 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:17:58 2017: norm: origin=core.action processed=771317 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:17:58 2017: elastic: origin=core.action processed=771317 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:17:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:17:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:17:58 2017: imrelp[20514]: origin=imrelp submitted=805293
> Wed Jan 18 11:17:58 2017: resource-usage: origin=impstats utime=177712000 
> stime=113216000 maxrss=285424 minflt=71000 majflt=0 inblock=10 oublock=69 
> nvcsw=2334000 nivcsw=65950
> Wed Jan 18 11:17:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:17:58 2017: relp: origin=core.queue size=35000 enqueued=805294 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:17:58 2017: main Q: origin=core.queue size=12 enqueued=246 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:22:58 2017: global: origin=dynstats
> Wed Jan 18 11:22:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:22:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=873317 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:22:58 2017: json: origin=core.action processed=873717 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:22:58 2017: norm: origin=core.action processed=873717 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:22:58 2017: elastic: origin=core.action processed=873717 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:22:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:22:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:22:58 2017: imrelp[20514]: origin=imrelp submitted=907693
> Wed Jan 18 11:22:58 2017: resource-usage: origin=impstats utime=201500000 
> stime=128332000 maxrss=315576 minflt=78597 majflt=0 inblock=10 oublock=72 
> nvcsw=2643581 nivcsw=75008
> Wed Jan 18 11:22:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:22:58 2017: relp: origin=core.queue size=35000 enqueued=907694 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:22:58 2017: main Q: origin=core.queue size=12 enqueued=259 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:27:58 2017: global: origin=dynstats
> Wed Jan 18 11:27:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:27:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=972862 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:27:58 2017: json: origin=core.action processed=973045 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:27:58 2017: norm: origin=core.action processed=973045 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:27:58 2017: elastic: origin=core.action processed=973045 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:27:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:27:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:27:58 2017: imrelp[20514]: origin=imrelp submitted=1007022
> Wed Jan 18 11:27:58 2017: resource-usage: origin=impstats utime=224592000 
> stime=142848000 maxrss=344840 minflt=85924 majflt=0 inblock=10 oublock=75 
> nvcsw=2943338 nivcsw=84266
> Wed Jan 18 11:27:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:27:58 2017: relp: origin=core.queue size=35001 enqueued=1007023 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:27:58 2017: main Q: origin=core.queue size=12 enqueued=272 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
> Wed Jan 18 11:32:58 2017: global: origin=dynstats
> Wed Jan 18 11:32:58 2017: action 0: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:32:58 2017: omelasticsearch: origin=omelasticsearch 
> submitted=1053233 failed.http=0 failed.httprequests=0 failed.checkConn=0 
> failed.es=0
> Wed Jan 18 11:32:58 2017: json: origin=core.action processed=1053233 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:32:58 2017: norm: origin=core.action processed=1053233 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:32:58 2017: elastic: origin=core.action processed=1053233 
> failed=0 suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:32:58 2017: error: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:32:58 2017: unk: origin=core.action processed=0 failed=0 
> suspended=0 suspended.duration=0 resumed=0
> Wed Jan 18 11:32:58 2017: imrelp[20514]: origin=imrelp submitted=1053233
> Wed Jan 18 11:32:58 2017: resource-usage: origin=impstats utime=243280000 
> stime=154784000 maxrss=368376 minflt=91831 majflt=0 inblock=10 oublock=78 
> nvcsw=3186315 nivcsw=90199
> Wed Jan 18 11:32:58 2017: relp[DA]: origin=core.queue size=0 enqueued=0 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=0
> Wed Jan 18 11:32:58 2017: relp: origin=core.queue size=0 enqueued=1053233 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=35011
> Wed Jan 18 11:32:58 2017: main Q: origin=core.queue size=12 enqueued=285 
> full=0 discarded.full=0 discarded.nf=0 maxqsize=13
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of 
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T 
> LIKE THAT.
>


More information about the rsyslog mailing list