<div dir="ltr"><div><div><div>Cool, I'll implement $!foo!bar[0].<br><br></div>Let us process this patch-set, because is kinda hard to keep track of old patches and re-send in one shot.<br><br>i'll send the new patch once done(i'll now only get to work on it on monday).<br></div><br></div>Do existing patches look ok except for the indexed-addressing feature?<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 31, 2014 at 4:15 PM, David Lang <span dir="ltr"><<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Fri, 31 Oct 2014, singh.janmejay wrote:<br>
<br>
</span><span class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Yes, I didn't have a need to address tokens individually, but you have a<br>
point.<br>
<br>
Any suggestions on what we want to do for addressing array elements?<br>
<br>
I wonder if its possible to do in $!... notation without breaking backward<br>
compatibility. How about a function?<br>
<br>
I'll be happy to implement support for addressing it in $!... notation if<br>
don't mind breaking a corner case in backward compatibility. Eg.<br>
$!foo!bar![0] ? Its kinda ugly though, or so I think.<br>
</blockquote>
<br></span>
I was thinking just $!foo!bar[0] it's a bit ugly, but not too bad. It does mean that you can't have '[' in a variable name, but I don't think that's likely to be a real problem. I don't think there's ever a really clean way to do something like $!foo[2]!bar[2]!baz no matter what your syntax, it gets messy.<br>
<br>
for templates, we probably need some sort of foreach(array, pattern) function that takes the pattern and repeats it for each item in the array.<span class="HOEnZb"><font color="#888888"><br>
<br>
David Lang</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Fri, Oct 31, 2014 at 3:44 PM, David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On Fri, 31 Oct 2014, singh.janmejay wrote:<br>
<br>
It writes it as a json array, here is a fragment from my manual tests:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
[ "15", "26", "15" ]<br>
<br>
</blockquote>
<br>
right, but how do you access it in rsyslog?<br>
<br>
if you have { 'foo': { 'bar': '10' } } you access this as $!foo!bar and<br>
get the result '10'<br>
<br>
what would you use to access the value '26' in your example?<br>
<br>
we also don't have anything like foreach() in our template language, which<br>
makes it hard to make use of these values as anything other than a JSON<br>
string.<br>
<br>
I'm not saying that it's not useful, but I am pointing out the problems<br>
that we will have using it.<br>
<br>
David Lang<br>
<br>
<br>
It was using time in hh:mm:ss format and tokening by colon(:). I'll add<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
tests for it soon, but until then pasting output here is the best I can<br>
do.<br>
<br>
The idea behind this is to generate structured content from<br>
semi-structured<br>
or unstructured log messages. So array is a good representation for<br>
tokenized-value (it is multi-valued by nature, and array is a good way to<br>
represent that).<br>
<br>
But eventually we should allow user to register value-transformers so that<br>
it can be pre-processed before its emitted. May be have a canned set of<br>
transformers, and allow user to plug in new ones.<br>
<br>
My first instinct was to utilize variable support for this, infact this<br>
was<br>
the motivator for variable support. But it still leads to a fairly complex<br>
config for an access log with 15 - 20 fields, especially given those<br>
fields<br>
can have colon separated entries inside comma separated entries etc.<br>
<br>
So I felt the need for a simpler way of doing it, hence this and other<br>
(recurse) field-type.<br>
<br>
On Fri, Oct 31, 2014 at 3:23 PM, David Lang <<a href="mailto:david@lang.hm" target="_blank">david@lang.hm</a>> wrote:<br>
<br>
On Fri, 31 Oct 2014, singh.janmejay wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Tokenizer followed by tokenizer is something that I have in mind too.<br>
But<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I<br>
promised myself that i'd write a test for that instead of testing it<br>
manually :-). Will add that patch on this thread once I get a chance to<br>
work on it.<br>
<br>
<br>
</blockquote>
At least in the short term, you can use the ability to call mmnormalize<br>
on<br>
a variable to parse subvariables.<br>
<br>
How are the resulting fields addressed? Rsyslog hasn't had array<br>
addressing yet.<br>
<br>
David Lang<br>
<br>
<br>
However, since you are asking about those kind of forms, let met discuss<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
something else that I was thinking about.<br>
<br>
The idea is to have another field type called recurse.<br>
<br>
Similar to how tokenized uses a ctx to parse matching text, recurse will<br>
parse it using the current context. AFAIK, the context is stateless, so<br>
I<br>
don't see any problems with that. I also plan to support tag based<br>
picking<br>
of which rules the text may match, and if it matches something else, it<br>
should be considered no-match.<br>
<br>
Instead of typing it out here, i'll attach a picture I took after<br>
thinking<br>
through it briefly(i'll attach it to the next mail).<br>
<br>
<br>
______________________________<u></u>_________________<br>
</blockquote>
Lognorm mailing list<br>
<a href="mailto:Lognorm@lists.adiscon.com" target="_blank">Lognorm@lists.adiscon.com</a><br>
<a href="http://lists.adiscon.net/mailman/listinfo/lognorm" target="_blank">http://lists.adiscon.net/<u></u>mailman/listinfo/lognorm</a><br>
<br>
______________________________<u></u>_________________<br>
Lognorm mailing list<br>
<a href="mailto:Lognorm@lists.adiscon.com" target="_blank">Lognorm@lists.adiscon.com</a><br>
<a href="http://lists.adiscon.net/mailman/listinfo/lognorm" target="_blank">http://lists.adiscon.net/<u></u>mailman/listinfo/lognorm</a><br>
<br>
<br>
<br>
</blockquote>
<br>
<br>
</blockquote>
______________________________<u></u>_________________<br>
Lognorm mailing list<br>
<a href="mailto:Lognorm@lists.adiscon.com" target="_blank">Lognorm@lists.adiscon.com</a><br>
<a href="http://lists.adiscon.net/mailman/listinfo/lognorm" target="_blank">http://lists.adiscon.net/<u></u>mailman/listinfo/lognorm</a><br>
<br>
______________________________<u></u>_________________<br>
Lognorm mailing list<br>
<a href="mailto:Lognorm@lists.adiscon.com" target="_blank">Lognorm@lists.adiscon.com</a><br>
<a href="http://lists.adiscon.net/mailman/listinfo/lognorm" target="_blank">http://lists.adiscon.net/<u></u>mailman/listinfo/lognorm</a><br>
<br>
<br>
</blockquote>
<br>
<br>
</blockquote>
</div></div><br>_______________________________________________<br>
Lognorm mailing list<br>
<a href="mailto:Lognorm@lists.adiscon.com">Lognorm@lists.adiscon.com</a><br>
<a href="http://lists.adiscon.net/mailman/listinfo/lognorm" target="_blank">http://lists.adiscon.net/mailman/listinfo/lognorm</a><br>
<br>_______________________________________________<br>
Lognorm mailing list<br>
<a href="mailto:Lognorm@lists.adiscon.com">Lognorm@lists.adiscon.com</a><br>
<a href="http://lists.adiscon.net/mailman/listinfo/lognorm" target="_blank">http://lists.adiscon.net/mailman/listinfo/lognorm</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Regards,<br>Janmejay<br><a href="http://codehunk.wordpress.com">http://codehunk.wordpress.com</a><br>
</div>