[rsyslog] making a custom input module

[david at lang.hm]

On Mon, 15 Sep 2008, Rainer Gerhards wrote:

> On Sun, 2008-09-14 at 22:30 -0700, david at lang.hm wrote:
>> my second project is to try and modify rsyslog to recieve logs from an
>> application. the application sends the logs over TCP and expects an
>> application-level handshake (very similar to relp). at the moment I want
>> to try and avoid having to change the application (many different products
>> with different release schedules), and instead teach rsyslog to deal with
>> the existing log format.
>>
>> at the moment I am trying to understand the imtcp module, but I am getting
>> lost in the callbacks. it looks like rsyslog is calling a routine in
>> imtcp, which calls a routine in tcpsrv, which calls a routine somewhere
>> else to actually recieve the log.
>
> The imtcp module has a lot of history and too complex code. This all
> stems back to various stages of GSSAPI integration. Things have been
> simplified with the introduction of the transport stream layer, but the
> imtcp module does not yet reflect this simplicity. So far, I am hesitant
> to split these things, because we still do not have a clean gssapi
> netstream driver.
>
> I can tell you where you need to provide callbacks ... or you could
> start from imrelp, which in regard means mostly from scratch, but
> actually receiving tcp isn't rocket science, so it may be easier to
> start with a proper template without tcp functionality and integrate
> your tcp reception code into it. What do you (think ;)) you prefer?

the simpler the better. I'm pretty rusty with my C and I never did much 
network or threaded programming in the first place, so while I can debug 
programs in any language, I'm currently not much beyond the cut-n-past 
stage here.

it sounds like I should not start from imtcp.

however when I looked at imrelp it looked like everything in in the relp 
libraries, and I was hoping to avoid diving into them for the moment.

are any of the other tcp-based transports in better shape? or should I 
start from scratch with the imtemplate plugin?

David Lang

>> ideally what I would like to use is to take imtcp and replace the message
>> recieving/parsing logic with my own, then have it submit the parsed
>> message into the queue (which looks like it would be via the SubmitMsg()
>> call).
>>
>> but at the moment I am lost in the twisty maze of function calls between
>> source files, all of which look different.
>>
>> I'm also not clear on what fields inside of the Msg structure need to be
>> populated. looking at Msg.c/h I see a lot of fields there, but it looks
>> like many/most of them are optional.
>
> look at imfile. This is what you need:
>
> 	CHKiRet(msgConstruct(&pMsg));
> 	MsgSetFlowControlType(pMsg, eFLOWCTL_FULL_DELAY);
> 	MsgSetInputName(pMsg, "imfile");
> 	MsgSetUxTradMsg(pMsg, (char*)rsCStrGetSzStr(cstrLine));
> 	MsgSetRawMsg(pMsg, (char*)rsCStrGetSzStr(cstrLine));
> 	MsgSetMSG(pMsg, (char*)rsCStrGetSzStr(cstrLine));
> 	MsgSetHOSTNAME(pMsg, (char*)glbl.GetLocalHostName());
> 	MsgSetTAG(pMsg, (char*)pInfo->pszTag);
> 	pMsg->iFacility = LOG_FAC(pInfo->iFacility);
> 	pMsg->iSeverity = LOG_PRI(pInfo->iSeverity);
> 	pMsg->bParseHOSTNAME = 0;
> 	datetime.getCurrTime(&(pMsg->tTIMESTAMP)); /* use the current time! */
> 	CHKiRet(submitMsg(pMsg))
>
> Also look at imtemplate, which is a template input module. Sometimes it
> is a bit outdated, if you have a problem during compile, tell me ;)
>
> Rainer
>>
>> any pointers?
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>