[rsyslog] tips for managing data

[Andre Lorbach]

Hi, 

the like query can indeed have quiet an impact on performance when doing
queries on large databases. 
But I think we can expand the syntax, so you can either search by part
of a string (LIKE '%search%') or the whole string (= 'search'). This
should be rather easy to implement. I will put this on my todolist, if
it is as easy as I think, the next minor update of the devel branch will
contain this new feature. 

Best regards,
Andre Lorbach

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rory Toma
> Sent: Thursday, July 31, 2008 4:10 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] tips for managing data
> 
> OK, so it seems that doing a query from the query line does a LIKE,
> which can take significantly longer (sample query 8 seconds vs. 50
msecs...)
> 
> So, replacing the LIKE % in logstreamdb.class.db with an = speeds
things
> up quite a but, but I lose some flexibility. Is there some kind of
> search syntax where I can differentiate between LIKE and =?
> 
> If not, I'm thinking something like:
> 
> source:foo.bar.com   # would be using =
> 
> ~source:foo   # would be using LIKE
> 
> 
> 
> Rory Toma wrote:
> > So, my current mysql rsyslog drops about 20 million rows of data per
day.
> >
> > Over time, this gets slow as tables grow.
> >
> > I'm not a dba, so I was wondering if anyone had some suggestions for
> > keeping performance still on the order of seconds, and not minutes
or hours.
> >
> > thx
> >
> > I did add a key for EventSource, as that is commonly searched.
However,
> > using PhpLogCon, it seems that if I search using the web interface
(i.e.
> > I click on a host entry and hit the available searches) it is
relatively
> > quick. However, changing the text field that is generated and
hitting
> > the "search" button is slow. Do these two methods use the same
query, or
> > is something else going on?
> >
> > thx
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> >
> 
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog