[rsyslog] [PERFORM] performance for high-volume loginsertion(fwd)
david at lang.hm
david at lang.hm
Fri Apr 24 09:30:15 CEST 2009
On Fri, 24 Apr 2009, Rainer Gerhards wrote:
>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>> On Fri, 24 Apr 2009, Rainer Gerhards wrote:
>>>> -----Original Message-----
>>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>>> that logic can work for every module that needs it, so this
>>>> an issue.. I was mixing up the API with the need for a config
>>>> David Lang
>>> Lol, our messages crossed. Still the question is if the stringbuilder
>>> support this mode. If so, we need to make deep changes to the way the
>>> property replacer works, thus I am hesitant to do this without real
>>> (not just because of the work to be done, but also because of the
>>> complexity [read: bugs, performance] it introduces).
>> if the string builder is not in the core, the output module can do the
>> work for 'mid' as/if needed.
>> table it for now.
>> food for thought (which may affect the decision when it happens), there
>> benifit in doing the database equivalent of dynafiles (inserting into
> Just to avoid misunderstanding: today, this is easy to acomplish - you just
> need to use a property replacer expression inside the template string like
> this "insert into syslog%hostname% ...".
correct (as long as you didn't need to create the tables)
> That, of course, will not work any longer if we go for prepared statements
> (indeed another subtlety). I'd still expect that it works with the begin...
> insert* ... end exec calls.
it will work with the begin;insert;end approach
for prepared statements you would need to prepare one for each destination
(similar to creating/opening files), and if you ended up doing copy or
multi-value inserts you would need to do seperate ones for different
but in any case, issues for another day.
>> different tables depending on the contents of the message) for pretty
>> the same reasons it's useful to do for flat files.
>> the part that does the string building needs to know about
>> dynafiles/'dynatables' and craft the strings to be written accordingly.
>> since syslog does not guarentee the order of events, selecting 'like'
>> items from the set that have been provided to it and grouping
>> is well within the right of the output side of things.
>> David Lang
>> rsyslog mailing list
> rsyslog mailing list
More information about the rsyslog