[rsyslog] Queue System

[Jonathan Bond-Caron]

Hi everyone, I'm been experimenting with rsyslogd to see if I could avoid
losing messages/events when the central server is rebooted.

I'll start with what works, then what doesn't L

 

Both the server & client are running rsyslogd 3.22.1

 

The client has the following configuration:

$ModLoad imklog.so      # kernel logging

$ModLoad imuxsock.so    # provides support for local system logging

 

$WorkDirectory /var/spool/rsyslog

 

$ActionQueueType Disk 

$ActionQueueFileName srvrfwd

$ActionResumeRetryCount -1

$ActionQueueSaveOnShutdown on

 

*.err;*.crit;*.emerg;*.warn;auth.*;user.notice @@server.domain.com

 

# Stop the central server

[root at server] $ service rsyslogd stop

 

# Start client logger

[root at client] $ service rsyslogd start

 

** (client) syslog(test a, NOTICE)

** (client) syslog(test b, NOTICE)

 

Since the central server is down I expect that the client will keep messages
in the disk queue and send them when the server is back online.

 

# Start the central server

[root at server] $ service rsyslogd start

.

2009-10-08T00:44:21-04:00 client php: test a

2009-10-08T00:45:39-04:00 client php: test b

. 

Great it works!

 

****** Now the problem, when client and server are both connected and
exchanging messages

 

# Stop the central server

[root at server] $ service rsyslogd stop

 

** (client) syslog(test c - lost message, NOTICE)

 

With some debugging:

4241.782860452:action 10 queue:Reg/w0: action 10 queue: entry deleted, state
0, size now 0 entries

4241.782860452:action 10 queue:Reg/w0:  server

4241.782860452:action 10 queue:Reg/w0:  server:10514/tcp

4241.782860452:action 10 queue:Reg/w0: TCP sent 36 bytes, requested 36

 

The client 'sends the message over tcp' and actually empties the queue!

Is this expected behavior? rsyslogd was shutdown 'cleanly' on the server.
Shouldn't it force the client to close the TCP connection?

 

Using netstat, the two servers are :

 

Proto Recv-Q Send-Q  Local Address          Foreign Address        (state)

tcp4       0      0  client.10514   server..60726 FIN_WAIT_2

 

Proto Recv-Q Send-Q Local Address               Foreign Address
State      

tcp        1     36  server:60726       client:10514          CLOSE_WAIT  

 

Have I misunderstood something? I've read Rainer's blog and issues with TCP
"unreliability" / case of a power failure but don't think this related