[Lognorm] Identifying message types

[Rainer Gerhards]

It's finally done:

http://blog.gerhards.net/2011/04/log-classification-with-liblognorm.html

This is based on some older CEE ideas and not necessarily inline with what
comes up. Also, I can think of a couple of more things that would be good to
add. But at least we now have the core functionality.

Feedback, as usual, appreciated. Official release will follow shortly, either
today or tomorrow. Everything already available via git.

Rainer

> -----Original Message-----
> From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> bounces at lists.adiscon.com] On Behalf Of Wladimir van der Laan
> Sent: Tuesday, March 22, 2011 12:33 PM
> To: lognorm
> Subject: Re: [Lognorm] Identifying message types
> 
> Hello Rainer,
> 
> Thanks for the explanation. Looks like I was right in my feeling that this
was
> missing.
> 
> I understand your rationale to wait for CEE on this, though. I read their
spec,
> and they propose that the identification of a message includes object,
action
> and status. But they haven't defined exactly what these should be, neither
> do they give any examples.
> 
> They still have quite a lot of definition work to to. Hopefully, it won't
take too
> long, a standard for logging is very badly needed, and the longer it takes,
the
> more developers will yet again come up with their own solutions.
> 
> I'm currently classifying all kinds of events in Zenoss Core, and realized
that
> when I was defining regexp patterns I could just as well tell it how to
extract
> out the interesting information for analysis and more useful presentation.
> Which is how I got to this project.
> 
> Wladimir
> 
> BTW: great work on rsyslog.
> 
> 
> On Tue, Mar 22, 2011 at 10:37 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
> 
> 
> 	Hi Wladimir,
> 
> 	This is a good question and you are abosultely right -- this is
currently
> 	missing. In fact, the speace in front of the colon inside the
rulebase is
> 	reserved for tags, which is the classification you are looking for.
> 	Liblognorm is in its infancy, though already quite useful in its
current
> 	state. I have paused development a bit for two reasons:
> 
> 	a) CEE needs to sort out some things -- I'd prefer to have some
issues
> solved
> 	before continuing (and re-doing some work).
> 	b) devel prio -- right now I am working hard on getting a new stable
> v5
> 	rsyslog out, and this is taking quite some toll
> 
> 	The feature you are asking for is definitely on the today list, and I
> hope to
> 	be able to work more on liblognorm within the next couple of weeks
> (this year
> 	has been very busy - and will be - at least until mid-april).
> 
> 	Rainer
> 
> 
> 	> -----Original Message-----
> 	> From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> 	> bounces at lists.adiscon.com] On Behalf Of Wladimir van der Laan
> 	> Sent: Monday, March 21, 2011 7:00 PM
> 	> To: lognorm at lists.adiscon.com
> 	> Subject: [Lognorm] Identifying message types
> 	>
> 	> Hello,
> 	>
> 	> I have a question about the usage of lognorm. As I understand, the
> 	> program extracts data fields from log messages in text format, by
> means
> 	> of examples from a ruleset file. The output is represented as
> metadata
> 	> key/value pairs.
> 	>
> 	> But as far as I can see, it outputs no identifier as to what kind
of
> 	> message the log line represents. For automated log processing, one
> 	> would also need to identify the message, for example, as failed
> 	> authentication, or dhcp request, etc.
> 	>
> 	> Am I overlooking something? Is it possible to add a message type
> field
> 	> in a ruleset?
> 	>
> 	> Greetings,
> 	> Wladimir
> 	>
> 
> 
> 	_______________________________________________
> 	Lognorm mailing list
> 	Lognorm at lists.adiscon.com
> 	http://lists.adiscon.net/mailman/listinfo/lognorm
> 
>