[rsyslog] replacing message parts with regex leads to message corruption

Radoslav Bodó bodik at cesnet.cz
Sun Jan 8 17:21:50 CET 2017


the same problem is replicable on my side with 8.23.0-2

bodik





Dne 01/08/2017 v 05:17 PM Radoslav Bodó via rsyslog napsal(a):
>> the system in question is debian stretch with rsyslog 8.16.0-1+b3
> 
> I'll rerun the tests with more recent rsyslog and report back
> 
> bodik
> 
> Dne 01/08/2017 v 05:04 PM David Lang napsal(a):
>> what version of rsyslog are you using? versions prior to ~8.20 had a
>> known problem due to json-c not being thread-safe.
>>
>> David Lang
>>
>> On Sun, 8 Jan 2017, Radoslav Bodó via rsyslog wrote:
>>
>>> Date: Sun, 8 Jan 2017 16:57:37 +0100
>>> From: Radoslav Bodó via rsyslog <rsyslog at lists.adiscon.com>
>>> To: rsyslog at lists.adiscon.com
>>> Cc: Radoslav Bodó <bodik at cesnet.cz>
>>> Subject: [rsyslog] replacing message parts with regex leads to message
>>>     corruption
>>>
>>> Hi,
>>>
>>> recently I was trying to create a masking template for software which
>>> logs messages including potentialy sensitive information (remctld
>>> logging whole command line executed).
>>>
>>>
>>> So I created a rule for masking that part out before storing messages:
>>>
>>> -------------- rsyslog.d/neweb2.conf
>>> $template Neweb2Format,"%timegenerated% %HOSTNAME%
>>> %syslogtag%%!msg:::drop-last-lf%\n"
>>> if ( ($programname == 'remctld') and ($msg contains 'neweb2') and ($msg
>>> contains 'pwd') ) then {
>>>        set $!ext = re_extract($msg,'(pwd [^ ]+)',0,1,"");
>>>        set $!msg = replace($msg, $!ext, "pwd MASKEDOUT");
>>>        action(type="omfile" template="Neweb2Format"
>>> File="/var/log/syslog")
>>>        stop
>>> }
>>> -------------------------
>>>
>>>
>>>
>>> acording to test a good behavior I've created a test case simulating
>>> remctld logging and check desired output
>>>
>>> ------------------- neweb2/tests/remctl_syslog_masks.sh
>>> #!/bin/sh
>>>
>>> . /puppet/metalib/bin/lib.sh
>>>
>>> RANDOM=$(/bin/dd if=/dev/urandom bs=100 count=1 2>/dev/null |
>>> /usr/bin/sha256sum | /usr/bin/awk '{print $1}' | sed
>>> 's/^\(......\).*/\1/')
>>>
>>> logger -t remctld "neweb2 db ${RANDOM}a --set --pwd 1234567890 --noop"
>>> logger -t remctld "neweb2 db ${RANDOM}b --set --noop --pwd 1234567890"
>>> logger -t remctld "neweb2 db --pwd 1234567890 --noop --set ${RANDOM}c"
>>>
>>> grep "neweb2 db ${RANDOM}a --set --pwd MASKEDOUT --noop" /var/log/syslog
>>> if [ $? -ne 0 ]; then
>>>        rreturn 1 "$0 remctl neweb2 sensitive data not masked A"
>>> fi
>>> grep "neweb2 db ${RANDOM}b --set --noop --pwd MASKEDOUT" /var/log/syslog
>>> if [ $? -ne 0 ]; then
>>>        rreturn 1 "$0 remctl neweb2 sensitive data not masked B"
>>> fi
>>> grep "neweb2 db --pwd MASKEDOUT --noop --set ${RANDOM}c" /var/log/syslog
>>> if [ $? -ne 0 ]; then
>>>        rreturn 1 "$0 remctl neweb2 sensitive data not masked C"
>>> fi
>>>
>>> rreturn 0 "$0"
>>> -------------------------
>>>
>>>
>>>
>>> but according to the test some of the messages gets garbled
>>>
>>> ---------- tail /var/log/syslog -n10
>>> Jan  8 16:28:21 tester remctld: neweb2 db ba539ba --set --pwd MASKEDOUT
>>> --nooc
>>> Jan  8 16:28:21 tester remctld: neweb2 db ba539bb --set --noop --pwd
>>> MASKEDOUT
>>> Jan  8 16:28:21 tester remctld: neweb2 db --pwd MASKEDOUT --noop --set
>>> ba539bc
>>> -------------------
>>>
>>>
>>> see the "--nooc" instead of "--noop" in the first case
>>>
>>>
>>>
>>>
>>>
>>>
>>> I'd suspect:
>>>
>>> a) my usage of replace() is wrong
>>> b) some memory management inside "property replacer" is not correct
>>>
>>> the system in question is debian stretch with rsyslog 8.16.0-1+b3
>>>
>>>
>>>
>>> I'd be glad for any suggestions or cross-tests of this case. I could dig
>>> into code, make some additional testing, or propose a patch, but I'm not
>>> really sure where to start ...
>>>
>>>
>>> Thank you for any help
>>> Best regards
>>> bodik
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>> http://www.rsyslog.com/professional-services/
>>> What's up with rsyslog? Follow https://twitter.com/rgerhards
>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
>>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
>>> if you DON'T LIKE THAT.
>>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
> 


More information about the rsyslog mailing list