[Lognorm] getting started document

[Rainer Gerhards]

The web site is currently being built (any help on content, design, etc is
deeply appreciated). It is available at

http://www.liblognorm.com

In the menu there is a link to the current *very sparse* documentation.

On data types: there is not yet really such a thing as a "data type" -- all
are strings. Liblognorm uses a different concept right now, that is
"parsers". A parser actually describes a syntax that a string must fullfil in
order to be treated as "correct". There are a number of parsers, which should
be in the doc (but I see Florian did still not include the new ones, hope he
will do soon...). It is these types that the rules are build around.

I have recently asked which additional "types" (parsers) are considered
useful and I am ready to add new ones (I am sure there must be more than we
currently have).

We also need standard field names. I have started an effort on what to use
and Florian started a sample directory. We are looking for feedback here.

Finally, but very importantly, I'd like to add a section with rule bases to
the web site, where device-specifc rules can be found ... and contributed.
Suggestion on how to do this best would be very appreciated.

I'll also see if I can write a small getting started document. I guess it
should focus on the technical issues. Right now, I am doing a lot of writing,
so this should be possible as a side-activity (though I would prefer to code
a little me, but... ;)).

Hope that helps at least a little bit.

Rainer

> -----Original Message-----
> From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, January 18, 2011 12:26 AM
> To: lognorm
> Subject: Re: [Lognorm] getting started document
> 
> From: "Champ Clark III [Softwink]" <champ at softwink.com>
> 
> >On Mon, Jan 17, 2011 at 01:33:00PM -0800, david at lang.hm wrote:
> >> the format of the rulebase files seems pretty straightforward, but
> >> where can I find the list of what datatypes are supported?
> >
> >        Keep in mind that liblognorm is pretty young at this point.
> >So more documentation probably needs to be done.
> 
> that's what I figured, and why I posted what i found needed to be done
> rather than just griping about it :-)
> 
> >Also, there's not much in the way of rulebase files yet, but that will
> >likely change in the future.  Right now, you're probably going to have
> to
> >write your own rulebase files.
> 
> I expected to have to write my own rules, but it's hard to write rules
> without knowing what datatypes are available.
> 
> >When you do that, keep a copy so that you
> >can send them in later.  :)
> 
> I expect to send a bundch in as I work on things, although I also
> expect
> to be using the parsing library in ways that don't directly tie in to
> the 'big normalization' project.
> 
> For example, Rainer wondered out loud about the possibility of using
> this
> library for a rsyslog log parser instead of writing the parsers
> directly
> in C. in that case, most of what you want in terms of normalization
> won't
> matter, as that parser is just focused on figuring out what part of the
> blob that arrives is the message, and what metadata in included, not in
> parsing the message itself into different pieces.
> 
> >> also, where can I find an example of how to compile the rulebase,
> and
> >> an example program that uses the rulebase?
> >
> >        Check out "normalizer.c" with the liblognorm git.  It's a
> basic
> > program that does normalization.
> 
> will do.
> 
> can you point me at what file (or files) the datatypes are defined in?
> 
> David Lang