[rsyslog-notify] Forum Thread: messages are missing from the log - (Mode 'post')

[noreply at adiscon.com]

User: awolfson 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=24469#p24469

Message: 
----------
Hi all,

I am new to that forum and to rsyslog and was not able to find if the
questions was asked before nether on the forum or by googling for almost a
day. 
My apologies if this is a well known issue.
We have a device that from the Linux point of view is a desktop with some
sensors connected.
The device runs Fedora9( rsyslog 3.20.2) and I am trying to upgrade to
Fedora17(rsyslog  5.8.10 and then 7.2.4)
Our app is logging to the special log file different from the system log.
On Fedora 9 none of the messages is missing on F17 some are. I wrote the
custom program to be able to test stand alone.
[code:2n06amy9]
#include <stdlib.h>
#include <syslog.h>
#include <unistd.h>
#include <iostream>
int
main(int argc, char** argv)
{
   int msg_nb;
   int udelay;
   //Skipping any sanity checks for the test programm
   msg_nb = atoi(argv[1]);
   udelay = atoi(argv[2]);
   std::cout << "Messages=" << msg_nb << " delay(usec)=" << udelay
<< std::endl;
   unsigned int extra_options = 0;
   unsigned int logmask       = LOG_UPTO(LOG_NOTICE);
   openlog("CARTD", LOG_CONS | extra_options, LOG_LOCAL0);
   setlogmask(logmask);
   int i=0;
   for (i=0;i<msg_nb; i++){
      syslog(LOG_NOTICE, "%d, RsysLog Log testing", i);
   usleep(udelay);
   }
   std::cout << "Sent " << i << " messages" << std::endl;
   exit(0);
}
[/code:2n06amy9]
This is what I use to run the test:
[code:2n06amy9]
#!/bin/bash
#$1 - number of messages
#$2 - delay(usec) between the messages
sudo rm -f /var/log/test-rsyslog.log
sudo touch /var/log/test-rsyslog.log

#ubuntu < 14.04
#sudo /etc/init.d/rsyslog restart

#fedora 17
sudo systemctl restart rsyslog.service
./test-rsyslog $1 $2
wc /var/log/test-rsyslog.log
[/code:2n06amy9]
I put test-rsyslog.conf file into /etc/rsyslog.d/
The commented out code is for different options that I tried unseccesfully
[code:2n06amy9]
# GEM customizations: 
#   3 File producing logging channels for g4k analytical module, 
#   postgresql database, and g4k network manager info proxy respectively
#$SystemLogRateLimitInterval 0
#$MainMsgQueueSize 10000
$MainMsgQueueType LinkedList
$MainMsgQueueDequeueSlowdown 0
#$NainMsgQueueWorkerThreadMinimumMessages 100
#$MainMsgQueueDequeueBatchSize 5000
#$MainMsgQueueWorkerThreads 24
#$ActionQueueType LinkedList
#$ActionQueueSize 100000
#$ActionQueueWorkerThreads 24
#$ActionQueueDequeueBatchSize 5000
##$OMFilelOBufferSize 32k
##$OMFileAsyncWriting on
#$MaxMessageSize 64k
local0.*                                              
/var/log/test-rsyslog.log
local0.*                                               ~
local2.*                                              
/var/log/pgsql.log
[/code:2n06amy9]

Originally F17 had rsyslog 5.8.10 and a lot of messages were missing. Then
I upgraded it with 7.2.4 taken from F18 (I took only rsyslog rpm not all
the libraries) it is much better now but some messages (5% approximately)
are still missing if I for example run 10000 messages with 10 usec delay.
On our device it creates approximately 1000 messages/sec
I hope somebody can help. Basically I have 2 questions:

[b:2n06amy9]Is version 7.2.4 is OK to use
Can I achieve the no messages lost by modifying configuration, etc.
[/b:2n06amy9]
Thanks in advance,

Alex
Again it worked fine on Fedora9. One of the differences that might have
impact is that F17 uses systemd, so it might be somehow related to
journaling etc.