[rsyslog-notify] Forum Thread: Re: Time lag issue of forwarding vs available I/O at 6600 mp - (Mode 'reply')

[noreply at adiscon.com]

User: rajatg 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=26783#p26783

Message: 
----------
Hi David

Thanks a lot for your quick reply.
rsyslog on central server is also 7.4.4. Now I went to square 1.
Yes, the default in-memory queue fills quickly and the enqueued elements
increase at a speed of 2000 mps. I think you are right radosgw might be
waiting on /dev/log because of which there is lag in logs showing up in
/var/log/syslog. Since the lag is in dumping at the client side to
/var/log/syslog, there should not be any problem with the server (confirm
this by the below pstats output). But how to resolve this waiting on
/dev/log problem? Should I bind more unix sockets to syslog or is there any
other alternative?


Following is the output of pstats-:

[code:2ecaxpgt]
Fri Jul 15 11:19:14 2016: imuxsock: submitted=186230
ratelimit.discarded=0 ratelimit.numratelimiters=57
Fri Jul 15 11:19:14 2016: action 1: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 2: processed=132 failed=0
Fri Jul 15 11:19:14 2016: action 3: processed=180413
failed=0
Fri Jul 15 11:19:14 2016: action 4: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 5: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 6: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 7: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 8: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 9: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 10: processed=180403
failed=0
Fri Jul 15 11:19:14 2016: action 11: processed=111
failed=111
Fri Jul 15 11:19:14 2016: action 12: processed=90511
failed=0
Fri Jul 15 11:19:14 2016: action 13: processed=90501
failed=0
Fri Jul 15 11:19:14 2016: action 14: processed=90615
failed=0
Fri Jul 15 11:19:14 2016: action 15: processed=0 failed=0
Fri Jul 15 11:19:14 2016: action 13 queue[DA]:
size=0 enqueued=27520 full=0 discarded.full=0 discarded.nf=0
maxqsize=1224
Fri Jul 15 11:19:14 2016: action 13 queue: size=0
enqueued=90501 full=5 discarded.full=0 discarded.nf=0 maxqsize=1000
Fri Jul 15 11:19:14 2016: main Q: size=100000
enqueued=186237 full=10 discarded.full=3 discarded.nf=0
maxqsize=100000
Fri Jul 15 11:19:24 2016: imuxsock: submitted=202535
ratelimit.discarded=0 ratelimit.numratelimiters=65
Fri Jul 15 11:19:24 2016: action 1: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 2: processed=156 failed=0
Fri Jul 15 11:19:24 2016: action 3: processed=201627
failed=0
Fri Jul 15 11:19:24 2016: action 4: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 5: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 6: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 7: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 8: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 9: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 10: processed=201617
failed=0
Fri Jul 15 11:19:24 2016: action 11: processed=117
failed=117
Fri Jul 15 11:19:24 2016: action 12: processed=113268
failed=0
Fri Jul 15 11:19:24 2016: action 13: processed=102422
failed=0
Fri Jul 15 11:19:24 2016: action 14: processed=102536
failed=0
Fri Jul 15 11:19:24 2016: action 15: processed=0 failed=0
Fri Jul 15 11:19:24 2016: action 13 queue[DA]:
size=0 enqueued=31904 full=0 discarded.full=0 discarded.nf=0
maxqsize=1229
Fri Jul 15 11:19:24 2016: action 13 queue: size=0
enqueued=102422 full=5 discarded.full=0 discarded.nf=0
maxqsize=1000
Fri Jul 15 11:19:24 2016: main Q: size=100000
enqueued=202542 full=17 discarded.full=5 discarded.nf=0
maxqsize=100000
[/code:2ecaxpgt]

Result of top with H-:
It is only the main queue threads (rs: main Q:reg) which are consuming the
CPUs.

Thanks,
Rajat