[rsyslog] rsyslog error messages
Ryo Fujita
rfujita at redhat.com
Fri Jul 25 16:56:04 CEST 2008
Hi HKS-san,
+1
> 2 - This will avoid a lot of silly questions without any supporting
> information. At least when you get a complaint like "Rsyslogd won't
> start!" it will generally include a "It just says 'configuration file
> invalid: broken syntax at line 135.'
In addition,
'... at line 135. See man 5 rsyslog.conf or $doc/rsyslog_conf.html.'
Any ideas?
Best Rio.
On 2008/07/25, at 23:33, (private) HKS wrote:
> Hi Rainer,
>
> Interesting post. The tendency to ignore errors/diagnostics is not
> limited to rsyslog, it's something that plagues pretty much every
> piece of software ever created anywhere. Some users are ignorant of
> basic diagnostic tools like syslog, --help flags, tcpdump, and man
> pages. Others just need a bit of prodding before they slap themselves
> and realize that they forgot some basic troubleshooting. Some are lazy
> slobs.
>
> However, rsyslog lends itself to error reports that lack
> troubleshooting for one huge reason: errors are not reliably reported
> in the default configuration. It's happened a few times that I start
> rsyslogd at the command line, and yet there's nothing in the process
> table, and nothing dumped to my logs. I've been able to track down the
> problem with -dn switches, but that involves weeding through a lot of
> irrelevant information.
>
> There are three modifications I'd love to see that would help
> resolve this:
>
> 1 - Configuration errors should be fatal
> 2 - Configuration and other runtime errors should be printed to STDERR
> 3 - By default, rsyslogd should log all of its own errors (fatal or
> not) to /var/log/rsyslog.log. This should also be user-configurable
>
> My reasoning:
>
> 1 - If rsyslog doesn't understand your config file, obviously it won't
> be doing what you intend it to. There's little benefit in running with
> a horked configuration, but if you don't test thoroughly, the cost
> could be devastating. Some users won't know they're not logging things
> until they need them - then, if they keep their jobs, they probably
> won't keep rsyslog ("This software doesn't even log ssh attempts!").
>
> 2 - This will avoid a lot of silly questions without any supporting
> information. At least when you get a complaint like "Rsyslogd won't
> start!" it will generally include a "It just says 'configuration file
> invalid: broken syntax at line 135.'" For most users, this kind of
> error reporting will be enough to lead them to a solution. If nothing
> else, it removes the need to explain how to find the errors.
>
> 3 - For those more subtle problems, and things you might miss upon
> bootup. This also gives you a simple, easily accessible debugging
> source with minimal security implications and additional code and no
> required user configuration. I'm not sure how the code is written, but
> I'd prefer to see this happen even if the configuration file can't be
> read. Perhaps an $RsyslogErrors directive that, by default, references
> a different module that just dumps directly to a file? Users can
> configure it to put rsyslog errors through the actual logging engine
> and then manage it with typical syslog selector/action lines.
>
> The HTTP server is an interesting idea, but not something I would make
> use of. The potential security problems are enormous, and it would
> introduce too many additional factors to the debugging process: is the
> firewall right? Was there already a process listening on that port? Is
> the user's configuration file correct? These are the kinds of things
> you already have to deal with on rsyslog itself - why force the
> debugging process to go through them again?
>
> Hope that helps, and sorry for the long email.
> -HKS
>
>
>
>
> On Fri, Jul 25, 2008 at 6:31 AM, Rainer Gerhards
> <rgerhards at hq.adiscon.com> wrote:
>> Thanks, HKS, for the good answers. There is one thing I would like to
>> bring to your attention: I often see that people do not check for
>> rsyslog error messages themselves. That complicates setup. I do not
>> blame anyone, but would like to make you aware of the problem.
>>
>> I have just blogged about a potential (partial) solution and will
>> try to
>> implement it:
>>
>> http://blog.gerhards.net/2008/07/rsyslog-error-reporting-how-to-do-it.ht
>> ml
>>
>> Rainer
>>
>>> -----Original Message-----
>>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>>> bounces at lists.adiscon.com] On Behalf Of (private) HKS
>>> Sent: Thursday, July 24, 2008 11:36 PM
>>> To: rsyslog-users
>>> Subject: Re: [rsyslog] Help with Ommail Configuration
>>>
>>> Oh - one other possibility. rsyslogd has to have mail enabled at
>>> compile time to work with it at runtime. check your catchall
>>> logfile -
>>> if it has messages like this, you didn't compile it in:
>>>
>>> rsyslogd: could not load module '/usr/local/lib/rsyslog/ommail.so',
>>> dlopen: /usr/local/lib/rsyslog/ommail.so: cannot open shared object
>>> file: No such file or directory
>>>
>>> To fix this, you'll need to reinstall. This time, when you run
>>> ./configure be sure to include --enable-mail. make install clean and
>>> you should be good to go...
>>>
>>> -HKS
>>>
>>> On Thu, Jul 24, 2008 at 5:01 PM, (private) HKS <hks.private at gmail.com
>>> >
>>> wrote:
>>>> A few things to try:
>>>>
>>>> - You load ommail.so twice, once at the top and once right above
>>> your
>>>> $ActionMail... lines. I don't think this will break it, but it's
>>>> unnecessary - delete the second one.
>>>>
>>>> - $ActionExecOnlyOnceEveryInterval 21600 means that it's only going
>>>> to attempt sending a message once every 6 hours. For testing
>>> purposes,
>>>> this will be obnoxious. Until you're ready for production, change
>>>> it
>>>> to:
>>>> $ActionExecOnlyOnceEveryInterval 30
>>>>
>>>> - Send everything to make sure you're not missing it based on
>>>> something in the property-replacer/templates/whatever. Replace "if
>>>> $msg contains 'hard disk fatal failure' then :ommail:;mailBody"
>> with:
>>>> *.* :ommail:;mailBody
>>>>
>>>> Try again. Try logging a few messages from the localhost first
>>>> (with
>>>> RHEL you can just run "logger test" to log a user.notice message
>> with
>>>> content "test") and see if you get them.
>>>>
>>>> If you don't, check the mail logs on your mail server to see if it
>>>> ever received the message. If not, it's time to break out tcpdump
>> and
>>>> see if the packets are ever being generated.
>>>>
>>>> Hope that helps.
>>>>
>>>> -HKS
>>>>
>>>>
>>>>
>>>> On Thu, Jul 24, 2008 at 3:50 PM, Goutos, Kevin (DOB)
>>>> <Kevin.Goutos at budget.state.ny.us> wrote:
>>>>> Hello all,
>>>>>
>>>>> First off, I am not very Linux savvy. I don't have a lot of
>>> experience
>>>>> other then a basic course. This is probably way past my knowledge,
>>> but I
>>>>> really need to get it done. Appreciate any help you guys have to
>>> offer.
>>>>>
>>>>> I am working on a Red Hat Enterprise 4 box and I am running the
>>> latest
>>>>> edition of rsyslog. I currently have rsyslog configured to receive
>>>>> messages remotely via UDP. I am trying to set it up so that it
>>>>> will
>>> send
>>>>> out E-mail messages to the system Admin's based on the severity
>>> level of
>>>>> the log files I am receiving. I would like it so that any device
>>> that
>>>>> sends a log with a critical, alert, or emergency level facility
>> will
>>>>> send out an e-mail to a specific address.
>>>>>
>>>>> Here is my rsyslog.conf file. I used the sample code from Rainer
>>>>> Gerhards configuration page. I tried sending a test syslog with
>>> 'hard
>>>>> disk fatal failure' in it, but it is not sending out mail. Also
>>> tried
>>>>> without the templates below thinking it would just send me an
>>>>> email
>>> for
>>>>> every syslog that I received, but it doesn't appear to be sending
>>> mail.
>>>>> Any thoughts on what I am doing wrong. I'm sure there is a lot I
>>> need to
>>>>> do, so please let me know. Thanks!
>>>>>
>>>>> $template mailSubject,"disk problem on %hostname%"
>>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"'
>>>>>
>>>>> I edited out the 3 lines below in the code for security reasons..
>>>>> $ActionMailSMTPServer <ip of smtp server>
>>>>> $ActionMailFrom <from address>
>>>>> $ActionMailTo <my email>
>>>>>
>>>>>
>>>>>
>>>>> Here is my code from rsyslog.conf below.
>>>>>
>>>>>
>>>>>
>>>>> # if you experience problems, check
>>>>> # http://www.rsyslog.com/troubleshoot for assistance
>>>>>
>>>>> # rsyslog v3: load input modules
>>>>> # If you do not load inputs, nothing happens!
>>>>> # You may need to set the module load path if modules are not
>> found.
>>>>>
>>>>> $ModLoad immark.so # provides --MARK-- message capability
>>>>> $ModLoad imuxsock.so # provides support for local system logging
>>> (e.g.
>>>>> via logger command)
>>>>> $ModLoad imklog.so # kernel logging (formerly provided by rklogd)
>>>>> $ModLoad ommail
>>>>>
>>>>> $template TraditionalFormatWithPRI,"%PRI-text%: %timegenerated%
>>>>> %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"
>>>>>
>>>>> # Log all kernel messages to the console.
>>>>> # Logging much else clutters up the screen.
>>>>> #kern.*
>> /dev/console
>>>>>
>>>>> # Log anything (except mail) of level info or higher.
>>>>> # Don't log private authentication messages!
>>>>> *.info;mail.none;authpriv.none;cron.none
>>>>> -/var/log/messages
>>>>>
>>>>> # The authpriv file has restricted access.
>>>>> authpriv.*
>>> /var/log/secure
>>>>>
>>>>> # Log all the mail messages in one place.
>>>>> mail.*
>>>>> -/var/log/maillog
>>>>>
>>>>> # Log cron stuff
>>>>> cron.* -
>>> /var/log/cron
>>>>>
>>>>> # Everybody gets emergency messages
>>>>> *.emerg *
>>>>>
>>>>> # Save news errors of level crit and higher in a special file.
>>>>> uucp,news.crit
>>>>> -/var/log/spooler
>>>>>
>>>>> # Save boot messages also to boot.log
>>>>> local7.*
>>>>> /var/log/boot.log
>>>>>
>>>>> #Catch all incoming syslog messages
>>>>> *.*
>>>>> /var/log/catchall;TraditionalFormatWithPRI
>>>>>
>>>>> # Remote Logging (we use TCP for reliable delivery)
>>>>> # An on-disk queue is created for this action. If the remote host
>> is
>>>>> # down, messages are spooled to disk and sent when it is up again.
>>>>> $WorkDirectory /rsyslog/spool # where to place spool files
>>>>> $ActionQueueFileName uniqName # unique name prefix for spool files
>>>>> $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as
>>>>> possible)
>>>>> $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
>>>>> $ActionQueueType LinkedList # run asynchronously
>>>>> $ActionResumeRetryCount -1 # infinite retries if host is down
>>>>> # remote host is: name/ip:port, e.g. 192.168.0.1:514, port
>>>>> optional
>>>>> *.* @10.57.106.140:514
>>>>>
>>>>> $ModLoad ommail
>>>>> $ActionMailSMTPServer <ip of smtp server>
>>>>> $ActionMailFrom <from address>
>>>>> $ActionMailTo <my email>
>>>>> $template mailSubject,"disk problem on %hostname%"
>>>>> $template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'"
>>>>> $ActionMailSubject mailSubject
>>>>> # make sure we receive a mail only once in six
>>>>> # hours (21,600 seconds ;))
>>>>> $ActionExecOnlyOnceEveryInterval 21600
>>>>> # the if ... then ... mailBody mus be on one line!
>>>>> if $msg contains 'hard disk fatal failure' then :ommail:;mailBody
>>>>>
>>>>>
>>>>> # ######### Receiving Messages from Remote Hosts ##########
>>>>> # TCP Syslog Server:
>>>>> # provides TCP syslog reception and GSS-API (if compiled to
>>>>> support
>>> it)
>>>>> $ModLoad imtcp.so # load module
>>>>> $InputTCPServerRun 514 # start up TCP listener at port 514
>>>>>
>>>>> # UDP Syslog Server:
>>>>> $ModLoad imudp.so # provides UDP syslog reception
>>>>> $UDPServerRun 514 # start a UDP syslog server at standard port
>>>>> --------------------------------------------------------
>>>>> This e-mail, including any attachments, may be confidential,
>>> privileged or otherwise legally protected. If you have received this
>> e-
>>> mail in error, or from someone who was not authorized to send it to
>>> you, do not disseminate, copy or otherwise use this e-mail or its
>>> attachments. Please notify the sender immediately if you have
>>> received
>>> this e-mail by mistake, and delete it from your system.
>>>>> _______________________________________________
>>>>> rsyslog mailing list
>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>>>>
>>>>
>>> _______________________________________________
>>> rsyslog mailing list
>>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
########################################################################
Ryo Fujita <rfujita at redhat.com>
Senior Solution Architect, RHCE
Red Hat K.K.
TEL +81-3-5798-8500
FAX +81-3-5798-8599
Ebisu Neonato 8F
4-1-18 Ebisu, Shibuya-ku,
Tokyo Japan 1500013
########################################################################
More information about the rsyslog
mailing list