[rsyslog] advices about SQL insertions

[Rainer Gerhards]

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Alexandre Chapellon
> Sent: Thursday, November 24, 2011 3:57 PM
> To: rsyslog at lists.adiscon.com
> Subject: Re: [rsyslog] advices about SQL insertions
> 
> 
> 
> Le 24/11/2011 15:48, Rainer Gerhards a écrit :
> > Tough question... it depends. Field-based extraction is *Very* fast
> and hard
> > to beat. So if you can, go with that. If you can re-use Rexpes inside
> the SQL
> > engine, it is probably better to do it there, as rsyslog templates
> can not
> > carry submatches across different regexes.
> More specificly, if I use the same regex several times in a template
> (to
> extract several submatch), is the regex evaluated each time or only
> once?

That's far easier to answer: multiple times.

rainer
> 
> As an example:
> 
> /$template PF_SPF_PG, "INSERT INTO mailsecurity \
>          (datelog, ipmx, msgsrv, hostip, exemptonspf, reason, mailfrom,
> rcptto, hostnameclient) \
>          VALUES \
>          ('%timegenerated:::date-rfc3339%', \
>          '%hostname%', \
>          'SPF', \
>          btrim('%msg:F,32:13%', '[]')::inet, \
>          trim(trailing ';' from '%msg:F,59:3%'), \
>          '%msg:F,59:3%', \
>          '%msg:R,ERE,1,BLANK:.*; from=<(.*)> to=<(.*)>.*
> helo=<(.*)>--end%', \
>          '%msg:R,ERE,2,BLANK:.*; from=<(.*)> to=<(.*)>.*
> helo=<(.*)>--end%', \
>          '%msg:R,ERE,3,BLANK:.*; from=<(.*)> to=<(.*)>.*
> helo=<(.*)>--end%')",stdsql/
> 
> >   Actually, I'd benchmark the whole
> > thing if I had the task...
> Rainer
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of Alexandre Chapellon
> >> Sent: Thursday, November 24, 2011 3:37 PM
> >> To: rsyslog-users
> >> Subject: [rsyslog] advices about SQL insertions
> >>
> >> Hello,
> >>
> >> I have syslog messages I want to put in a database. I don't want to
> >> store raw content of the message but only valuable data found in the
> >> message.
> >> To extract the data I wish to insert, I can eihter use field and
> regex
> >> property replacer features of rsyslog or use SQL funstions like (eg)
> >> trim, substring etc... Both works.
> >> What would you recommend to ensure the lowest impact on the
> ressources
> >> of the server?
> >>
> >> Regards.
> >> --
> >> <http://www.horoa.net>
> >>
> >> Alexandre Chapellon
> >>
> >> Ingénierie des systèmes open sources et réseaux.
> >> Follow me on twitter: @alxgomz<http://www.twitter.com/alxgomz>
> >>
> >> _______________________________________________
> >> rsyslog mailing list
> >> http://lists.adiscon.net/mailman/listinfo/rsyslog
> >> http://www.rsyslog.com/professional-services/
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> 
> --
> <http://www.horoa.net>
> 
> Alexandre Chapellon
> 
> Ingénierie des systèmes open sources et réseaux.
> Follow me on twitter: @alxgomz <http://www.twitter.com/alxgomz>
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/