[rsyslog] directing logs to a broadcast address fails

[david at lang.hm]

On Tue, 5 May 2009, Tom Metro wrote:

> Date: Tue, 05 May 2009 02:36:12 -0400
> From: Tom Metro <tmetro+rsyslog at gmail.com>
> 
> david at lang.hm wrote:
>> Tom Metro wrote:
>>> I need to get the syslog broadcast problem resolved...
>>
>> what I'm doing for the syslog broadcast is defining a multicast MAC
>> address for a specific IP, and then setting that IP address up on all the
>> systems that need to see the message.  (see
>> http://www.linux-ha.org/ClusterIP for info on this and examples of how to
>> set it up for testing) this lets me spread the load between multiple
>> machines in one set while still having multiple sets of boxes recieve the
>> same message.
>
> So to distribute the load you send some messages to multicast group A,
> and others to group B?

no, much simpler. I'll describe more below

> Multicast in general makes sense if you are going to be sending a volume
> of messages to N > 1 log servers. (I noticed there was a multicast patch
> for sysklogd sitting in the Debian bug queue.)

that's to send the message to a multicast IP address, somthing different 
from what I am talking about.

it turns out that the ethernet spec says that a mac address with a low bit 
of 1 in the first octect (such as 01:00:00:00:00:00) is defined as 
multicast, and acts like a broadcast packet through a switch, no matter 
what IP address it has (say 192.168.1.100)

then you can use the iptables command (see the clusterIP link I sent you) 
to configure your linux box to say that it is node 1 of 1 and it will see 
all the packets.

you can also say that you are node 1 of 2 (with another box being node 2 
of 2) and both boxes will do a hash of the packet header info (dest IP, 
dest port, source IP, source port) and decide which node should handle 
that packet. that node will process the packet and other nodes will drop 
the packet.

with UDP packets, you can have multiple sets of machines doing this with 
the same MAC address, at which point box 1 can send a packet to 
192.168.1.100 and

box 2 (node 1 of 1) will process it
box 3 (node 1 of 2) will process it
box 4 (node 2 of 2) will ignore it
boxes 5-8  (nodes 1-4 of 10) will ignore it
box 9 (node 5 of 10) will process it
boxes 10-14 (nodes 6-10 of 10) will ignore it

etc.

so you are broadcasting to multiple clusters, but within each cluster it 
load balances and only one box in each cluster will fully process the 
packet.

yes, UDP can be lost (this is _not_ the ultra-reliable option being 
discussed in other threads), but I have measured multiple test runs of a 
billion packets being sent with this mechanism over a switch at Gig-E wire 
speed to several seperate clusters receiving the logs, with no packet loss 
(on an early 4.1 version), so it's pretty reliable

> In my case I'm looking to distribute critical warning messages only,
> which will be rare, and it wouldn't benefit the network to use
> multicast, so I'd rather avoid the configuration overhead.

that is simpler for your purpose.

> See earlier messages in this thread for the details of the problem. The
> summary is that syslog messages directed at a broadcast address
> (x.x.x.255) fail to go anywhere on a Debian Etch box running sysklogd,
> or an Ubuntu 8.04 box running sysklogd or rsyslog, but rsyslog on Ubuntu
> 8.10 seems to work. I've backported that version of rsyslog to the 8.04
> box, but it didn't resolve the problem.
>
> Inspecting the source code for the working version of rsyslog shows a
> lack of code to enable the broadcast flag, so I'm not sure why it works
> on 8.10. Patching the code to enable the broadcast flag didn't seem to
> help on 8.04.
>
> I know broadcast UDP packets work on the 8.04 box, as it uses DHCP
> successfully.
>
> I've started looking at the source to a DHCP client to see how it
> configures its socket to permit broadcasting. One thing I haven't tried
> recently is using the interface defined broadcast address
> (255.255.255.255), which is what the DHCP client uses. I tried that
> early on, but once I confirmed that the subnet broadcast address worked
> correctly on other machines, I ceased trying the global address.

that is a bit different from the broadcast address for your network.

David Lang

> My guess is that there is some socket flag that still needs to be
> enabled to get it to work on 8.04.