Demon in Unix call the program that works "on background" without requiring terminal control, and giving you the ability to do your other work "in the foreground". The daemon can be started either by another process, for example, by one of the system startup scripts without accessing any control terminal at all, or by the user from some terminal, but in this case daemon "does not take over" terminal for the duration of its work. Naturally, the question arises, which daemons are needed on your system, and which can be disabled.
Origin of the term
In the real (ie, non-computer) world, the term (or word) "demon" denotes a spirit (most often - an evil one) or an "inner voice". It is worth noting that both of these values apply to Unix daemon programs as well. Like mythological demons, Unix daemons hide behind the scenes, trying to remain invisible. And, like the inner voice or our subconsciousness, they are always “on the alert”, always available, can always “manifest”. The word "demon" itself represents one of those computer acronyms, the origin of which is just as difficult to find out as it is to decide whether there was a chicken or an egg in the beginning. Presumably this term ("DAEMON") comes from the words "Disk And Execution MONitor" program.
Having set out to study the issue, I went to a priest who uses a computer with the Ubuntu operating system. Just in case, I took a disk with Windows. You never know ... I'm a straightforward person, I didn't beat around the bush, I immediately gave out all the information. Explained for a long time and in detail. To my great happiness, he took it all calmly, in some places, while I talked, he even smiled, especially when he told about the reasons for my sister's husband's refusal from ubuntu. Father, a normal man, he is already 50 years old. We talked a little more, or rather I talked, in order to understand that he had correctly assimilated what I said. After which he told me, I don't remember literally, but I didn’t think to record it on a dictaphone, so I’ll describe it in my own words. In general, there is nothing wrong with the fact that this term is used in Linux, the main thing is faith in God.
Introduction to services
Daemons referenced by /etc/init.d are designed to run on Linux as a service or service. Services are programs that are started and stopped through init scripts located in the /etc/init.d directory. Many of these services are launched during the boot phase of the system. The / sbin / service utility provides a user interface (interaction) with initialization scripts. And these scripts themselves provide an interface for managing services, providing the user with options to start, stop, restart, query the status of the service, and perform other actions on the service. For example, the httpd service init script has the following options:
/ sbin / service httpd Usage: httpd (start | stop | restart | condrestart | reload | status | fullstatus | graceful | help | configtest)
You can view the current state of all system services using the following option of the service utility:
/ sbin / service --status-all acpid (pid 2481) is running ... anacron (pid 2647) is running ... atd (pid 2657) is running ... auditd (pid 2189) is running ...
Information about the runlevel of these services, that is, setting at which of the system runlevels a particular service is started at boot time, can be obtained or changed using the chkconfig utility. Let's see, for example, the current settings for the syslog system logging service:
/ sbin / chkconfig --list syslog syslog 0: off 1: off 2: on 3: on 4: on 5: on 6: off
We see that the syslog service automatically starts when we go to levels 2, 3, 4, and 5. To disable it from starting at levels 3 and 4 (not a good idea, by the way), you can use the following option to the chkconfig command:
/ sbin / chkconfig --levels 34 syslog off
The / usr / bin / system-config-services utility provides a graphical interface to system services that allows you to view and modify their current state, as well as set them to run at different runlevels
Let's see how these services and daemons appear in the ps command output. Here's a quick excerpt:
UID PID PPID C STIME TTY TIME CMD root 1 0 0 23:36? 00:00:00 init root 2161 1 0 23:37? 00:00:00 auditd root 2177 1 0 23:37? 00:00:00 syslogd -m 0 root 2180 1 0 23:37? 00:00:00 klogd -x root 2207 1 0 23:37? 00:00:00 mcstransd root 2254 1 0 23:37? 00:00:00 rpc.statd root 2287 1 0 23:37? 00:00:00 rpc.idmapd root 2577 1 0 23:37? 00:00:00 crond root 2631 1 0 23:37? 00:00:00 / usr / sbin / atd root 2654 1 0 23:37? 00:00:00 rhnsd --interval 240
What is interesting to note here (besides, of course, that I sat too late at the computer today)? For each of the daemons, the parent process identifier (PPID) is 1. This indicates that the daemons were started by init at boot time.
This result can be seen more clearly in the output of such a useful command as "pstree", which displays the "tree" of processes with the indication of "parents". Here's a small snippet from pstree's output:
Init- + | -NetworkManager --- 2 * [(NetworkManager)] | -NetworkManagerD | -acpid | -atd | -auditd - + - python | `- (auditd) | -avahi-daemon --- avahi-daemon | -bonobo-activati --- (bonobo-activati) | -crond | -cupsd --- cups-polld | -2 * | -dbus-launch | -dhcdbd --- dhclient
More about daemons on your system
We are done with the introductory information. Now let's talk about system daemons in isolation and see which ones you can safely experiment with. Again, this post is about a system running Red Hat Enterprise Linux Beta 2 in a workstation configuration. Depending on the specifics of your system, you may see more or less daemons, including some that are not covered here.
The descriptions for each daemon include links to sites where you can get more information, but it is still best to start learning about the daemon by looking at the respective man pages. O "Reilly has an excellent alphabetical listing of Linux commands, and wikipedia.org also has sections on most of these daemons. Be sure to check the README files too.
It is the Advanced Configuration and Power Interface (ACPI) service. ACPI is an open industry standard that defines system management actions, primarily plug-and-play device definition and power management, including actions when the system starts, stops, and when it goes into a low-power mode.
You should probably not disable this service unless you are doing it for debugging purposes or to resolve hardware problems.
More information: http://www.acpi.info
If you are using a laptop, as many do these days, then one of the problems is that when you ask the cron daemon to do something, you are not always sure that your computer will be turned on at the time at which execution is scheduled. tasks. Anacron (this name comes from "anachronistic cron", which means something like "obsolete cron") solves this problem by checking to see if the jobs were executed within a certain period of time. For example, anacron will start a task if it has not been started for a specified number of days.
When can you stop using anacron? If your system is running constantly.
Can you just refuse to run cron if you have anacron running? No; for anacron, the task restart period can only be specified in days, not minutes and seconds.
Additional Information:
It is the Advanced Power Management (APM) daemon via a driver in the BIOS. The APM standard and the apmd daemon have now been superseded by ACPI and acpid. If your hardware supports ACPI, you don't need to run apmd.
This is a daemon for starting jobs at a specified time. If you are not using it, it is quite possible to disable it.
This daemon automatically mounts the disks and file systems that are defined in the configuration file. Using this daemon makes it easier to work with removable disks.
More information: http://freshmeat.net/projects/autofs
The Linux auditing system consists of kernel-level system call loggers and user-space log collection and viewing tools. The auditd daemon writes logged messages to disk. Auditd is configurable, allowing you to control and manage what information is stored on disk.
Should auditd be kept running? Information from the logs can be very helpful in configuring everything related to system security. For example, auditd is used in SELinux event logging. There are also utilities such as aureport that allow you to view audit logs. Here is an example of a report generated by the aureport utility:
Summary Report
======================
Range of time in logs: 11/28/2006 06: 07: 04.800 - 02/06/2007 21: 10: 09.957 Selected time for report: 12/31/1969 19:00:00 - 02/06/2007 21: 10: 09.957 Number of changes in configuration: 285 Number of changes to accounts, groups, or roles: 32 Number of logins: 145 Number of failed logins: 11 Number of users: 2 Number of terminals: 22 Number of host names: 11 Number of executables: 27 Number of files: 91 Number of AVC denials: 688 Number of MAC events: 12 Number of failed syscalls: 404 Number of anomaly events: 0 Number of responses to anomaly events: 0 Number of crypto events: 0 Number of process IDs: 14022 Number of events: 70694 Avahi-daemon and avahi-dnsconfd
The Avahi website defines it this way: "Avahi is a system that provides the ability to discover services on a local network. This means that once your computer is connected to a local network, you can instantly discover available printers, see what shares are available on the network, find out with whom from other network users you can talk through chat and so on. " Avahi is an implementation of the Zeroconf protocol. And Zeroconf is an approach that allows users to create IP networks without special configuration services like DNS servers.
Most often, avahi-daemon is used in conjunction with Rhythmbox so that you can see music files that are shared with others. If you are not sharing music or files from your computer, you can disable this daemon.
More information: http://avahi.org, http://zeroconf.org
Bluetooth and the hidd and pand daemons
The names themselves explain everything. Start these services if you want to use bluetooth devices. The name of the actually running daemon is hcid (Host Controller Interface Daemon).
The hidd daemon's name comes from "the Bluetooth Human Interface Device Daemon". It provides Bluetooth keyboard, mouse and trackball support. And the pand daemon keeps your computer connected to the ethernet network using Bluetooth.
More information: http://www.bluetooth.com,
This daemon supports the Common ISDN Application Programming Interface. It should be run if you are connecting to ISDN hardware components. This service starts capiinit.
More information: http://www.capi.org/pages
No, it has nothing to do with the overnight real estate investment broadcasts. The conman service (and the conmand daemon) support console management. Provides support for multiple console devices and simultaneous user work, support for local serial devices and remote terminal servers (via telnet protocol). If you are managing multiple servers, you might need conman.
More information: http://home.gna.org/conman/
cpuspeed
This demon changes the speed of work central processing unit in order to reduce energy consumption. If the CPU is idle, you can reduce the speed, thereby lowering power consumption, and to increase performance you have to increase power consumption. If you work for laptop, it makes sense to run cpuspeed.
More information: http://carlthompson.net/Software/CPUSpeed
crond
This daemon is used to automatically start tasks. This is required on all Linux and Unix systems. Do not stop or disable this service.
More information: http://www.unixgeeks.org/security/newbie/unix/cron-1.html, http://www.linuxhelp.net/guides/cron/
CUPS and cups-config-daemon
It is the "Common UNIX Printing Solution" daemon. As the name suggests, it is a printing system that handles a variety of file formats and different types printers. If you want to print, have this daemon run on your system.
More information: http://www.cups.org, http://www.easysw.com/cups/index.php
The name of this daemon is an acronym for "DHcp Client D-Bus Daemon". The Free DeskTop wiki defines it as follows:
D-Bus is a message bus system, a simple way for applications to communicate (or interact) with each other. In addition to interprocess communication, D-Bus helps coordinate the process life cycle; it provides a simple and reliable implementation in code of the ability to run a "separate instance" of an application or daemon, which allows you to run applications and daemons on demand, when the need arises for the appropriate services.
Do you need to run this daemon? If your system works on a network (how could it be otherwise?), Especially if you are moving between networks (for example, switching from a wired network to a wireless one), then you should start NetworkManager (we will look at NetworkManager below).
The dhcdbd daemon provides a D-Bus interface for dhclient, a DHCP client from ISC. This allows NetworkManager to access and control dhclient.
More information: http://www.freedesktop.org/wiki/Software/dbus
This daemon gives you the ability to use the mouse in text-based applications such as file manager Midnight Commander. This can be useful if you frequently use the console; otherwise, that is, if you are using the X Window System all the time, you may never need gpmd.
No, this is not the evil computer from a 2001 Space Odyssey movie. In this context, HAL stands for Hardware Abstraction Layer. The HAL daemon collects (using the kernel and the devices themselves) information about hardware devices and presents it to applications in a consistent manner.
Do not disable this service. Many applications use it.
More information: "Desktop and hardware configuration," by David Zeuthen
This daemon provides support for the HP Linux Imaging and Printing (HPLIP) system used for printing, scanning, and faxing on inkjet and laser devices from HP. HPLIP interacts with CUPS, providing the latter with access to devices from HP.
Additional Information:
It is a daemon for relational databases in Java. It inherited its name from the Hypersonic SQL project, which is no longer supported. hsqldb is widely used in open source projects such as OpenOffice.org, as well as in demos, as it can run entirely in RAM. Due to what it works quickly. Do you need to run this service? Only if you run any programs that use it. But actually it is very useful tool and, if you are not familiar with it, it is worth looking at it.
More information: http://hsqldb.org, http://dba.openoffice.org
Apache web server. Used by nearly 60% of all websites. If you want to maintain a website, you run Apache. Do I need to say anything else?
More information: http://httpd.apache.org
ip6tables and iptables
These are firewalls or firewalls. According to Wikipedia, "A firewall or firewall (jargon. Firewall or firewall from the English. Firewall) is a set of hardware and / or software tools that control and filter network packets passing through it at various levels of the OSI model in accordance with specified rules. firewall is protection computer networks or individual nodes from unauthorized access. "
Iptables works by setting IPv4 packet filtering rules in the form of a kernel table. Incoming and outgoing packets are checked against these rules, and those that do not match the rules are blocked. Ip6tables does the same for IPv6 packets.
Which of the two services do you want to run? Both. Is always. The network is dangerous!
More information: http://www.netfilter.org, http://www.ipv6.org
IrDA (Infrared Data Association) is an industry standard for infrared wireless communication between devices. Most laptops are equipped with an IrDA-compliant infrared transmitter. You need to start this service only if you are going to organize communication with other devices via infrared.
Additional Information:
irqbalance
This daemon is responsible for distributing hardware interrupts between CPUs in symmetric processor (SMP) multiprocessor architectures to improve performance.
There is no need to run this daemon on uniprocessor systems; it only affects multiprocessor systems.
More information: http://www.irqbalance.org
It is a very useful daemon. It runs at boot time and checks which hardware devices have been added to or removed from the system. It makes sense to run kudzu at boot time, even if you don't plan on adding or removing devices frequently. Because sooner or later, you will equally find yourself in a situation where you add a device and expect the system to detect it. In addition, since kudzu runs at boot time, it does not negatively impact system performance.
More information: http://fedora.redhat.com/projects/additional-projects/kudzu
This daemon gets its name from the Lan Information Server. Lisa provides functionality similar to that provided by the Network Neighborhood service in MS-Windows, that is, it provides access to servers on your local network, including CIFS (Common Internet File Systems) servers. lisa runs over TCP / IP, sending ICMP requests to IP addresses in the range you specified in the config file, and waiting for computers to respond.
More information: http://docs.kde.org/stable/en/kdenetwork/lisa, http://docs.kde.org/userguide/networking-with-windows.html,
lm_sensors
This daemon monitors temperatures and voltages on the motherboard. For the monitoring system to work, it is necessary to have appropriate sensors in the equipment. It only makes sense to start this daemon if hardware support is provided. You probably shouldn't run it on regular workstations. Rather, it is needed on hi-end servers that perform critical functions.
More information: http://www.lm-sensors.org, http://freshmeat.net/projects/lm_sensors
mcstrans
SELinux Context Translation System Daemon. This daemon converts security context informartion into human readable form. You can stop this daemon, but if so, you will see the SELinux information displayed by the ls -Z command change. For example, with the daemon running, you will see:
Ls -Z -rw-r - r-- jsmith jsmith user_u: object_r: user_home_t bookmarks.html drwxr-xr-x jsmith jsmith user_u: object_r: user_home_t Desktop -r-xr-xr-x jsmith jsmith user_home: object_rt hello -r - r - r-- jsmith jsmith user_u: object_r: user_home_t hello.c
And if the daemon is stopped, you will see the following:
Ls -Z -rw-r - r-- jsmith jsmith user_u: object_r: user_home_t: s0 bookmarks.html drwxr-xr-x jsmith jsmith user_u: object_r: user_home_t: s0 Desktop -r-xr-xr-x jsmith jsmith user : object_r: user_home_t: s0 hello -r - r - r-- jsmith jsmith user_u: object_r: user_home_t: s0 hello.c
Note that if the daemon is stopped, the security context value "s0" is displayed. Mctrans has zeroed out the context in this case. Other security context values are converted from alphanumeric values to their names.
More information: http://fedoraproject.org/wiki/SELinux/Understanding, http://danwalsh.livejournal.com
mdmonitor and mdmpd
These two daemons are used in redundant array of inexpensive / independent disks (RAID) storage systems. Mdmonitor starts, stops and restarts mdadm (multipath device monitoring and management), a software RAID monitoring and management service. You only need to start this service if your system has RAID devices.
More information: http://www.linuxdevcenter.com/pub/a/linux/...12/05/RAID.html
messagebus
It is the D-BUS system message bus daemon. It broadcasts system events and events such as changes to the print queue, or the addition or removal of devices (note that this is not the same as kudzu, as such events can occur at any time during system operation, not only at the boot stage).
More information: http://www.freedesktop.org/software/dbus
netplugd and ifplugd
These daemons configure Ethernet devices when a network cable is connected, and disable those devices when the cable is disconnected. When do you need it? For example, on laptops, so that your network connections restored only while the network cable is connected.
Note that support for netplugd has been discontinued and is now replaced by ifplugd.
More information: http://0pointer.de/lennart/projects/ifplugd
NetworkManager and NetworkManagerDispatcher
The NetworkManager daemon automates switching between network connections. This daemon is useful for laptop users who switch between WiFi wireless connections and Ethernet connections. The NetworkManagerDispatcher daemon automatically launches scripts to perform the necessary operations (for example, scripts to set specific routing directions for packets) when the NetworkManager changes the state of the network.
More information: http://www.gnome.org/projects/NetworkManager
It is a daemon that acts as a Domain Name Server. You should only run it if your computer is the DNS server for your network.
More information: http://www.dns.net/dnsrd
The nfsd daemon provides support for the nfs network communication protocol, which is used to provide access to network resources in TCP / IP networks. You need to run it if you give other users access to your file systems using the nfs protocol.
Additional Information:
This is the caching daemon for the name service. It maintains a table of groups and passwords for running programs, and then produces a remembered result on the next request for services that are otherwise too slow, such as NIS or LDAP. If you have these services running, you should start nscd as well.
It is a daemon that supports the Network Time Protocol. It sets the system time and synchronizes it with the time set by Internet servers that maintain the reference time. If your system is connected to the Internet (isn't it?), Then this daemon will monitor the correct setting of the system time on your computer.
More information: http://www.ntp.org
The oddjobd daemon runs the com.redhat.oddjob service on the system bus. Each opportunity provided by oddjobd is provided as separate method D-Bus. Oddjobd provides support for performing privileged operations for non-privileged applications.
You should only start this daemon if you are using applications that require it, such as Conga.
More information: http://people.redhat.com/nalin/oddjob/oddjob.html,
This daemon supports virtual private networks (VPNs). This daemon's startup script says the following:
OpenVPN is a robust and highly flexible tunneling application that uses the encryption, authentication, and certification capabilities of the OpenSSL library to securely tunnel an IP network over a single UDP port.
If your system is a VPN host, then you should probably run OpenVPN.
More information: http://openvpn.net
pcscd (PC / SC Smart Card Daemon) is a daemon for pcsc-lite (smart card access software) and (java based) MuscleCard development environment. It provides communication with smart card readers and smart cards themselves.
(A smart card is a small circuit board that contains either a memory module or a microprocessor with a memory module. Muscle stands for Movement for the Use of Smart Cards in a Linux Environment.
More information: http://www.smartcardalliance.org, http://pcsclite.alioth.debian.org, http://www.linuxnet.com/musclecard/index.html
The portmapper daemon manages RPC (remote procedure call) connections. It converts RPC program numbers to TCP / IP (or UDP / IP) port numbers. The portmapper is most commonly used by NFS and NIS.
So if your system depends on NIS or NFS, do not disable the portmap daemon.
More information: http://www.linux-nis.org/nis-howto/HOWTO/portmapper.html
It is a mail transfer (transport) agent. If your system is not a relay to the email system, you do not need to start this service.
More information: http://www.postfix.org
This daemon (the router discovery daemon) finds routes on the local subnet. It is run at boot time to add the default routes to the routing tables.
More information: http://www.informit.com/articles/article.asp?p=23951&rl=1
restorecond
This is a daemon from SELinux. Restorecond monitors file creation (for files listed in /etc/selinux/restorecond.conf) and ensures that the files have the correct file context for the specified policy, and also defines the default SELinux file context.
Do not disable this service. It is required for SELinux.
More information: http://fedoraproject.org/wiki/SELinux/Understanding, http://danwalsh.livejournal.com/
This daemon periodically checks what operations should be performed through network interface Red Hat (Red Hat Network web interface), and runs them. These operations include installing, removing or updating software, rebooting the system, installing configuration files, and so on.
More information: https://www.redhat.com/rhn/
rpcgssd, rpcidmapd and rpcsvcgssd
The rpcgssd and rpcsvcgssd daemons are for security over RPC. Rpcidmapd converts usernames to UID and GID numbers.
If you are using NFS or NIS, you should have these daemons running.
Additional Information:
readahead_early and readahead_later
The readahead daemon ensures that startup programs are loaded into memory before they are used, which reduces boot time.
saslauthd
This is the SASL authentication server daemon. SASL (Simple Authentication and Security Layer) adds authentication capabilities to protocols based on remote connections.
More information: http://asg.web.cmu.edu/sasl
sendmail
It is an SMTP (Simple Mail Transfer Protocol) server. Sendmail forwards mail from one system to another, that is, it is a Mail Transport Agent. If you are using mailers such as Thunderbird or Evolution, you do not need to run sendmail.
More information: http://www.sendmail.org
setroubleshoot
This is the SELinux problem resolver daemon. Setroubleshooter is one of the great recent innovations in SELinux. Setroubleshooter provides real-time feedback to users on SELInux AVC (Access Vector Cache) failures. Moreover, this feedback is provided in a convenient format.
More information: https://hosted.fedoraproject.org/projects/setroubleshoot
This daemon monitors the readings of SMART (Self-Monitoring, Analysis and Reporting Technology) sensors installed in many types of drives, for example, in SCSI-3 hard drives. The daemon monitors the normal operation of such devices and performs a self-test. If your hardware supports SMART technology, you need to start this service.
Additional Information:
spamassassin
This daemon uses the Apache SpamAssassin program to check mail for spam. It usually runs in conjunction with a mail deleivery agent (MDA) server. If you are using client programs like Thunderbird or Evolution to access your mail, you do not need to run spamassassin.
More information: http://spamassassin.apache.org
This is a daemon for the open ssh protocol. Ssh replaces the insecure rsh and rlogin programs and provides encrypted connections between hosts on insecure networks. If you are using connections to other systems over the open Internet, you must use ssh and run this daemon.
More information: http://www.ssh.com, http://www.openssh.com
syslog is the standard Linux logging system. Do not turn it off.
More information: http://www.syslog.org
This daemon is part of the Samba package. It enables Windows domain users to connect as Unix users to Unix servers. This daemon should be started when you are dealing with a mixed network of Windows and Linux / Unix computers.
More information: http://www.samba.org/samba/docs/man/Samba-...on/winbind.html, http://www.samba.org
This is a font server. This daemon loads fonts into memory so that graphics applications run faster than having to load fonts from the hard drive. This service should be started to improve the performance of applications on your system.
More information: http://linuxreviews.org/howtos/xfree/xfs
This service binds the NIS client to the NIS domain. The letters "yp" in its name come from the "yellow pages," as NIS directories are like the yellow pages telephone directories. You only need to start this service if your system uses NIS (Network Information Service) to provide access to user budgets and system names.
More information: http://www.linux-nis.org
yum-updatesd
yum-updatesd monitors the availability of software updates and sends notifications of these updates by email, d-bus or system messages, and can also produce automatic update ON. D-bus messages are received by the "puplet" utility (package updater), which informs the user that an update is available and also allows the user to install the update.
More information: http://linux.duke.edu/projects/yum, http://www.redhat.com/magazine/024oct06/features/fc62
Thanks to ZOOL for the information provided on the website http://www.centrlan.ru/node/17929
Source not specified
Each of the novice Linux users sooner or later encounters some problems in setting up and organizing the functioning of their system. And each of the newbies almost certainly heard from more experienced users the advice: "Look at the logs." This advice is good, but a beginner still needs to know: what are logs and where to find them! So I will try in this article to tell you what and where to look.In programming slang, "logs" are the protocols of work that are maintained both by the operating system itself and independently by many programs. The word "log" is often used as a synonym for the word "protocol" in this sense. There are two main situations in which it becomes necessary to analyze the protocol: when something in the system does not work as we expected (problem resolution), and when there is a suspicion that the system has been hacked by some intruder and it is necessary to find out what exactly happened. how it was done, and what needs to be done to deal with the aftermath of the invasion.
One of the most famous cases of using log files to detect intrusion of an attacker is the story of the capture by computer security specialist Tsuomo Shimomura of the famous hacker Kevin Mitnick. Here is one paragraph from an article describing how this happened.
“On Christmas day, when Shimomura went skiing in Nevada on vacation, someone (we already know who) broke into his super-secure home computer in Solana Beach, California, and began copying his files - hundreds of classified files. from the San Diego Supercomputer Center, where Shimomura worked, noticed changes in the system "log" (log) files and quickly realized what was happening. to a backup computer in San Diego. The student called Shimomura and he rushed home to take an inventory of the stolen goods. "
I will not tell the whole story here, it is important for us to note only that the analysis of the system operation protocols served as the basis for the success of the investigation of illegal actions. A more detailed description of how such investigations are carried out can be found in the article. But in order to be able to use the results of logging, you need to understand how the protocols are created, where they are stored and what can be extracted from them. This article is devoted to the consideration of all these questions.
How messages are generated for the protocol
We need to start with the fact that when creating programs in the C language, programmers have the opportunity, if necessary, to insert a call of special functions openlog, setlogmask, syslog and closelog included in the standard C library. These functions are used to send messages about certain events during program execution to a special system daemon syslogd leading the system protocol. Function openlog establishes a connection with a demon syslogd, function syslog generates a specific message to be written to the protocol, and the function closelog closes an open link.
Messages generated by the function syslog, consist of multiple fields separated by spaces. Every message starts with a field PRI, which encoded information about the facility's category and the severity level or priority of the message.
The category (facility) is information about which class the given message belongs to. The category is encoded with a number from 0 to 23. There are the following categories (they are defined in the file /usr/include/sys/syslog.h):
Table 1.
| Numerical value | Symbol | Decryption |
| 0 | kern | Kernel messages |
| 1 | user | Intended for miscellaneous messages from user programs. (Messages from user programs) |
| 2 | Messages from the postal system. | |
| 3 | daemon | Messages from those system daemons, which, unlike FTP or LPR, do not have specific categories. |
| 4 | auth | Everything related to user authorization, like login and su (security / access rights) |
| 5 | syslog | The logging system can log messages from itself. |
| 6 | lpr | Messages from the printing system. |
| 7 | news | Messages from the news server. (Netnews, USENET) |
| 8 | uucp | Messages from UNIX-to-UNIX Copy Protocol. This is part of the history of UNIX and most likely you will never need it (although still some mail is delivered via UUCP). |
| 9 | cron | Messages from the system scheduler. |
| 10 | authpriv | The same as auth, but messages in this category are written to a file that only a few users can read (perhaps this category is highlighted because messages belonging to it may contain open user passwords that should not be seen by unauthorized people, and hence the log files must have the appropriate permissions). |
| 11 | ftp | Through this category you can configure your FTP server to record its actions. |
| from 12 to 15 | - | Reserved for system use. |
| from 16 to 23 | local0 - local7 | Reserved categories for use by the system administrator. The local7 category is usually used for messages generated during the system boot phase. |
Category auth has an obsolete synonym security which is not recommended for use. In addition, there is a special category mark(has no digital equivalent), which is assigned to individual messages generated by the daemon itself syslogd... This category is used to place special marks in the protocol at a specified time interval (by default, every 20 minutes), which allows, for example, to know with 20-minute precision when your computer is frozen.
Note that the category generally has nothing to do with the name of the program that sends messages to the daemon. syslogd... As they say, any coincidence is pure coincidence. Moreover, some programs can generate messages of different categories. For example, demon telnetd in case of unsuccessful logging attempts, generates category messages authpriv, and in other cases categorizes its posts daemon.
The second parameter, on the basis of which the field value is formed PRI, is the level or priority of the message (priority), that is, information about the degree of importance of the message. 8 severity levels are set as standard (they are also defined in the file /usr/include/sys/syslog.h), which are encoded with numbers from 0 to 7:
Table 2.
| Numerical value | Symbol | Decryption |
| 0 | emerg(old name PANIC) | Emergency. The system is inoperative. |
| 1 | alert | Anxiety! Immediate intervention required. |
| 2 | crit | Fatal error (critical condition). |
| 3 | err(old name ERROR) | Error message. |
| 4 | warning(old name WARN) | A warning. |
| 5 | notice | Information about some normal but important event. |
| 6 | info | Announcement. |
| 7 | debug | Messages generated during debugging. |
Field PRI the message is formed as follows: the numerical value of the category is multiplied by 8 and added to the numerical value of the priority, the resulting number is enclosed in angle brackets and written in the field.
Following the field PRI The following fields are included in the message:
- TIMESTAMP- message formation time,
- HOSTNAME- host name or IP-address in decimal notation,
- MSG- arbitrary text of the message - some text (informational) string in the US-ASCII code (0x20 - 0x7e).
Time (local!) Is written in the format: Feb 13 21:12:06. If the day number is a single-digit number, then an additional space is placed in front of it (not 0!). Pay attention to the absence of the year and zone in the date, which must be taken into account when organizing long-term storage log files. In order for the time in the message to be correct, it must be synchronized (for example, using the NTP protocol).
The hostname is included in the message in order not to confuse messages from different hosts, since, as will be shown below, logging can be carried out on one of the dedicated computers on the network. The hostname is the simple network name of the computer, without specifying the domain. If a computer has several interfaces with different IP addresses, then any of them can be used as the host name or address.
Message text ( MSG) usually contains a label ( TAG), pointing to the program or process that issued this message, and the message body ( CONTENT). The label can contain Latin letters and numbers. Typically, a simple program name is used as a label, sometimes followed by a process identifier, enclosed in square brackets. The message body is separated from the label by special characters - in Linux, these are usually colon and space.
Syslogd message handling
All messages generated individual programs using the function syslog sent via socket / dev / log system daemon syslogd who is responsible for processing these messages. I must say that, in fact, two logging daemons are launched on the system - syslogd and klogd... Both daemons are included in the package sysklogd, the latest version of which you can find on the website.
Daemon klogd responsible for logging events occurring in the system kernel. The need for a separate demon klogd is explained by the fact that the kernel cannot use the standard function syslog... The fact is that the standard libraries (including the library in which the function is located syslog) are intended for use by common applications only. Since the kernel also needs similar functions, it includes its own libraries that are not available to applications. Therefore, the kernel uses its own message generation mechanism. Daemon klogd is designed to organize the processing of these messages. In principle, he can carry out such processing completely independently and independently of syslogd, for example, by writing these messages to a file, but in most cases the default setting is used klogd where all messages from the kernel are forwarded to the same daemon syslogd.
To make sure the demons syslogd and klogd running on your system, run the command ps -ax | grep log... This command gave the following result for me:
$ ps -ax | grep log 569? S 0:00 syslogd -m 0 574? S 0:00 klogd -x 1013? S 0:00 login - kos 1191? S 0:00 kalarmd --login The first two lines indicate that the logging daemons are running on the system.Daemon Handling Messages syslogd consists in the fact that it constantly monitors the appearance of messages and compares each received entry with the rules that are in the file /etc/syslog.conf... Each rule is written as a line of the file /etc/syslog.conf consisting of two fields. The left field ("selector") specifies one or several templates by which messages are selected. Templates are separated by semicolons (see below example file /etc/syslog.conf). The right margin ("action") determines the order in which the selected messages are processed. The fields are separated by one or more spaces or tabs.
Each template in the "selector" field has the form "category.level" (that is, "facility.priority"). The values of the "category" field can be:
- one of the conventional names of categories listed in Table 1,
- several such names (in this case they are separated by commas)
- or the * character (which means "all categories").
The values of the "level" field can be:
- one of the conditional names of the level listed in Table 2,
- character * (record all messages of this category, regardless of the level),
- or word none(do not record messages in this category).
Specifying a specific value in the "level" field is interpreted as "all values of this level and above". If you need to record messages of only one level, then you need to put an equal sign ("=") in front of the specified value. If you want to record messages of all levels, except for the indicated one, then an exclamation mark ("!") Is placed before the name of the level. Placing these two signs at the same time is interpreted as "do not record messages of the specified level or higher".
Uppercase and lowercase letters in the "selector" field do not differ. You can also use numbers (see /usr/include/syslog.h). In addition to the categories listed in Table 1, mark(regular timestamps) and security(deprecated synonym for auth). In addition to the priority values listed in Table 2, you can use warn(synonym for warning), error(synonym for err), panic(synonym for emerg).
When the category and level of the received message matches one of the templates of the "selector" field of some string, syslogd processes the record in accordance with the action specified in the "action" field of this line.
The "action" field can contain
- the name of a regular file (log file), and the full path to the file must be specified, starting with the root "/", and if the specified file does not exist, syslogd creates it,
- named pipe name - FIFO; in this case a vertical bar ("|") is placed in front of the name, and the channel itself must be created before starting syslogd team mkfifo,
- pointing to a terminal or console (like device: / dev / tty1),
- a reference to the remote host (preceded by the @ symbol),
- or a list of users (separated by commas), to whose terminals a message will be sent according to this rule. Instead of the list of users, you can put an asterisk (*), which will mean that messages are sent to all users working in this moment in system.
Most often, the "action" field still contains the name of the log file. Moreover, you can put a minus sign ("-") in front of the file name, which will mean that the system can store the file in the cache buffer, and not flush it after each message is written to disk. This, of course, speeds up the work, especially if many large messages are written to the log, but it can lead to the loss of some messages in the event of an unexpected system crash, that is, exactly when such messages are especially needed. You can also specify a printer - / dev / lp0 as a device in the "action" field. This option of "action" is advisable to apply in cases when it comes to particularly important systems. Protocols that are printed cannot be erased or altered by hackers, so this is a good use for an old dot matrix printer.
In addition to the lines with the rules in the file /etc/syslog.conf may contain blank lines and comment lines beginning with #. More about file structure /etc/syslog.conf you can read the syslog.conf man page for quite a few examples of rule entries in this file. Note that when specifying pairs "category.level" in the file syslog.conf you cannot use the numerical values given in tables 1 and 2, only their conventional names are allowed.
If a message matches the patterns of two or more lines, it will be processed according to each of these rules (that is, message processing does not stop on the first success). This means that an arbitrary number of actions can be performed for one message. Therefore, you can both write a message to a log file and send it to the user (s) or to a remote host.
In addition, if several "category.level" pairs are listed in the "selector" field (separated by semicolons), then subsequent pairs can cancel the action of the previous ones. You can see an example of such cancellation in the file listing below /etc/syslog.conf: All messages with a level equal to or higher than info are written to the file / var / log / messages, but messages of the categories mail, authpriv and cron are skipped (not written).
| Listing 1. File /etc/syslog.conf from a system built on the basis of the Red Hat Linux 7.1 distribution kit (I just translated into Russian the comments contained in this file and selected in bold rule lines). |
| # Print all messages from the kernel to the console. # kern. * / dev / console# Log all messages of info level or higher, # except for mail messages containing confidential # authentication messages and cron messages. * .info; mail.none; authpriv.none; cron.none / var / log / messages# Write messages containing confidential # authentication information to a separate file, regardless of their level. authpriv. * / var / log / secure# All messages of the mail system should also be recorded separately. mail. * / var / log / maillog# Log the actions of the cron daemon. cron. * / var / log / cron# Emergency messages should be received immediately # by all users of the system * .emerg *# Messages from news services of crit level and higher should be written to a separate file. uucp, news.crit / var / log / spooler# The messages displayed during the boot phase should be copied to the boot.log file local7. * /var/log/boot.log |
Log file / var / log / messages
Of course, there is no way to tell here about the contents of each line of this file. In order for the reader to get an idea of what information can be found in the protocol, we present separate message lines with very short comments.
Each line in the log file contains a single message record, consisting of the following message fields, separated by spaces:
- date in standard text format (field TIMESTAMP from post syslog),
- hostname (field HOSTNAME from post syslog)
- message text (fields TAG and CONTENT from post syslog)
Firstly, it is worth noting that if your computer does not work around the clock, but turns off at night, then in this file you can find records of several "work cycles", starting with the computer startup and ending with its shutdown. Such a cycle begins with a message about the start of the logging daemons (this is understandable, before they were started, messages were not recorded):
Sep 17 08:32:56 kos3 syslogd 1.4-0: restart. Sep 17 08:32:56 kos3 syslog: syslogd succeeded Sep 17 08:32:56 kos3 kernel: klogd 1.4-0, log source = / proc / kmsg started. Sep 17 08:32:56 kos3 kernel: Inspecting /boot/System.map-2.4.2-2 Sep 17 08:32:56 kos3 syslog: start klogd succeeded
Entries in other protocol files mentioned in the file have approximately the same structure. /etc/syslog.conf.
Logger and tailf commands
From the previous description, we can conclude that the issuance of all messages for system logs should be set by the programmer at the stage of program creation. This is not entirely true. The user also has the ability to send a message to the daemon syslogd... To do this, Linux has a command logger, which provides sending a message from the command line (sh, bash, etc.). It is included in the util-linux package. Naturally, this command is primarily intended to provide logging capabilities when the user creates various kinds of shell scripts. But it can also be launched directly from the command line, for example, to get acquainted with the capabilities of the logging system. Command launch format: logger [-isd] [-f file] [-p PRI] [-t TAG] [-u socket] Command line parameters have the following meaning:
- -i- include the process number in the message
- -s- duplicate message to stderr
- -d- use datagram mode when sending messages (instead of the usual streaming)
- -f filename- save the message in the specified file (by default it is used / var / log / messages)
- -p facility.level- set the category and priority of the message (by default: user.notice)
- -t TAG - set the TAG field
- -u socket- send a message to the specified socket instead of calling syslogd
- MSG- Message text
Send multiple messages using the program logger and admire the result in the file / var / log / messages(unless, of course, you have changed the default).
By the way, there is a very interesting way to view messages written to a file / var / log / messages team logger... This method is based on the use special program tailf... Open a terminal window, get superuser rights (with the command su) and run the command in this window
tailf / var / log / messages
After that, switch to another terminal and run the command there logger arbitrary_text... Your message will immediately appear in the window where the program is running tailf... That is, using this program, the system administrator can monitor the recording of new messages in the protocol in real time. True, the system administrator is unlikely to have time to monitor the behavior of the system in this way (unless in emergency situations). Therefore, special programs have been developed to analyze the protocols. But about them a little lower, but for now let's move on to the question of how to organize supervision of a remote computer (remember how you caught the attacker Shimomura?).
Network logging
As mentioned, log messages can be sent by a daemon syslogd to the remote host. But someone has to accept it there. It turns out that the same demon does it syslogd running on this remote host. More precisely, syslogd on any computer, it can listen not only on the / dev / log socket (thereby receiving messages from local sources), but also on port 514 / UDP, which ensures that messages from other computers on the local network are received (and then written to a local file). This provides the ability to create a "logging server", which can be very convenient for the system administrator (all events in the network are monitored in one place), and also increases the security of the network, since messages about the penetration of a hacker to one of the network hosts cannot be immediately removed from the protocol by this hacker.
However, some extra effort is needed to set up this "network logging".
First, since port 514 / UDP is used to send and receive messages over the network, it must be available on both computers (client and server). To do this, in the file / etc / services the line must be present on both computers
syslog 514 / udp
If such a line in / etc / services absent, syslogd can neither receive messages nor send them to the network because it cannot open a UDP port. If this situation arises, syslogd immediately stops writing any messages, even to the local log. Moreover, as the command shows ps, it remains in memory and even stores messages in some buffers, because if the string " syslog 514 / udp " restore in file / etc / services on the client, then at least some of the "missing" messages still appear in the log (after restarting syslogd).
Second, when starting the daemon syslogd the server must have the option -r which provides remote logging capability (by default, the daemon syslogd only waits for messages from the local socket). How and where to set this option will be described below, in the section on starting the daemon. syslogd.
Well, and thirdly, the settings in the files must be corrected accordingly. /etc/syslog.conf on both computers. For example, if you want to redirect all messages to the logging server, you must write in the file on the client computer /etc/syslog.conf a line like this:
*. * @hostname
If during the start of the demon syslogd the server will be unavailable (for example, it is disconnected from the network at this moment) or it will not be possible to find it by name (the DNS service did not work properly) syslogd makes another 10 attempts to find the server and only if after that the server cannot be found, stops attempts and sends a corresponding message.
If your network has several domains that are served by one logging server, then do not be surprised that the log on the server will contain the full names of the clients (including the domain). True, at startup syslogd you can use options -s domain_list or -l hostlist, which ensure that the protocol replaces fully qualified host names with their short names (without specifying a domain).
Do not forget after adjusting startup options and file /etc/syslog.conf restart the daemon, because unlike cron, sysklogd does not reread configuration files automatically.
Syslogd daemon startup options
Since we have touched in the previous subsection the question of setting parameters for starting the daemon syslogd, let's take a closer look at this issue. As already mentioned, both logging daemons are started at the stage of system initialization, and more specifically, through a script /etc/rc.d/init.d/syslog(for which, as well as for scripts to start other services, symbolic links are created in the /etc/rc.d/rc?.d/ directories). However, in order to set the startup parameters, there is no need to adjust this script, because starting from version 7.2 in the Red Hat distribution, startup options for both daemons are read from a separate configuration file / etc / sysconfig / syslog... Here is a short list of possible parameters for the daemon syslogd.Launch parameters syslogd:
- -a socket- Specifies an additional socket that the daemon will listen on syslogd... You can specify up to 19 sockets (more can be specified, but you need to recompile the package). This option is used when some other daemon (such as ftp or http) is running in a restricted environment (doing chroot).
- -d- Debug mode. In this case, the daemon does not go into the background and issues all messages to the current terminal.
- -f config_file_name Specifies the name of an alternative configuration file that will be used instead of the default /etc/syslog.conf.
- -h By default in sysklogd transmission of messages received over the network to some other computer is prohibited. This is done in order to avoid endless message transfers around the ring. The -h option allows you to change the usual behavior and ensure that messages received from remote hosts are passed along the network (and take care of the possible looping yourself).
- -l host-list- Specifies a list of hosts, the names of which should not be recorded with the indication of the full domain name (FQDN - Full Qwalified Domain Name); the names in the list are separated by colons.
- -m minutes Started without this option sysklogd regularly (every 20 minutes) logs messages of the category mark, that is, just time stamps. With the option -m you can either change the interval between marks, or completely cancel the issuance of such messages, for which you need to set a zero interval: -m 0.
- -n Do not go to the background; this option is needed when syslogd is started and controlled by a process init.
- -p socket Specifies an alternate UNIX socket (instead of the default listening / dev / log). Please note: option -a specifies additional sockets, and -p- alternative!
- -r Allow to receive messages from remote hosts. We talked about this in the previous section, so I omit the details.
- -s domain-list Specifies a list of domains whose names do not need to be logged along with the host name (that is, for these domains, only host names will be logged instead of the fully qualified domain name (FQDN). The names in the list are separated by colons. The name of the domain where the syslogd server is located) , it is not required to be specified in this list (its name is removed by default).
- -v Show version and exit.
- -x Forbid to determine the host name by its address, prevents deadlock when working on the same host with the DNS server.
After starting the demon syslogd a status file is created / var / lock / subsys / syslog zero length, and file with process ID /var/run/syslogd.pid.
Using the command
kill -SIGNAL `cat / var / run / syslogd.pid`
you can send to demon syslogd one of the following signals:
- SIGHUP - restart the daemon (reinitialization); all open files are closed, the daemon starts over, re-reading its configuration file.
- SIGTERM - shutdown.
- SIGINT, SIGQUIT - if debug mode is enabled (option -d), signals are ignored, otherwise - exit.
- SIGUSR1 - enable / disable debug mode (works only if the daemon was started with the -d switch).
Daemon klogd has no less startup options than syslogd, but we will not list them here, referring the reader to the corresponding man-page (the user should not be involved in setting klogd, it is better to leave it in the same state as it was produced by the developers of the distribution kit).
Dmesg file and dmesg command
As stated, the log files referenced in the file /etc/syslog.conf usually located in the directory / var / log and its subdirectories. But, if we look into this directory, then we will find several files there, which in /etc/syslog.conf were not mentioned. Let's take a look at their purpose. And let's start with the file dmesg.First, it should be mentioned that Linux has a command with the same name. If you compare the output of this command (when it is run without parameters) with the contents of the file / var / log / dmesg, you will find that they are very similar, although not identical (pipe the output of the command to the file dmesg2 and compare files dmesg and dmesg2). More precisely, the file / var / log / dmesg one to one coincides with the beginning of the output that we will receive by the command dmesg... As follows from the kernel, there is a ring buffer in which messages from the kernel logging daemon are written. Those messages that are written to this buffer during the loading process, and make up the contents of the file / var / log / dmesg... Apparently, this file is generated at the end of the system boot.
If you look at the above file listing again /etc/syslog.conf then you will see that all posts in the category kern are also issued to the console. But there they quickly run across the screen and you hardly have time to read and comprehend them. But they are saved in the file / var / log / dmesg and thus are available for leisurely comprehension (if the download process has completed successfully). After the end of the boot process, writing messages from the kernel to the circular buffer continues. When the command is executed dmesg, the current state of the buffer is displayed. Therefore, the output of this command contains more messages than the file / var / log / dmesg: in the output of this command, you also see the messages that the kernel issues after the end of the boot process.
All messages from / var / log / dmesg you will find in the file / var / log / messages, only there they alternate with messages from other programs. There is only one significant difference: in the file dmesg the time and source of the message (hostname and message category) are not specified. The host is always "local" here, and the start of the countdown is determined by the last restart of the computer.
Lastlog, wtmp and utmp files
Besides the file dmesg in the catalog / var / log / there are two more files not mentioned in /etc/syslog.conf, but directly related to logging - these are files lastlog and wtmp... But looking at them is the same as we were looking at the file / var / log / messages does not make sense - you will not understand anything. The fact is that information in these files is recorded in a special format, and it must be viewed using special software tools. But first, a few words must be said about the purpose of these files.
File lastlog stores information about the last user logged on to the system. I don’t know if you noticed that when you enter the username and password, the following message appears on the screen:
Localhost login: kos Password: Last login: Wed Oct 9 19:25:53 on tty1 These three lines are generated by the utility login, which, after determining that the user has the right to log in, accesses the file / var / log / lastlog, retrieves information about the previous successful user login from there, prints it to the screen, and then updates the entry in the file lastlog... You can suppress this message by creating an empty .hushlogin file in your home directory. However, it is not recommended to do this, rather, on the contrary, you should pay special attention to the content of this message so as not to miss cases when someone else entered the system under your name.
Unlike file / var / log / lastlog which contains records of the time the last each user's login, in the file / var / log / wtmp are remembered all user logins and logoffs since the creation of this file. As in the file lastlog, entries in / var / log / wtmp are made in a special format, so they can only be viewed using special commands. But, before talking about these commands, let's say that there is another file containing entries about user logging - this is the file / var / run / utmp... This file contains information about which user is currently logged into the system.
Now you can talk about how to view information about the users currently working or previously working in the system. The main command for this is the command last... It outputs all records from the file / var / log / wtmp, moreover, the user name, an indication of the terminal from which the user worked, the time the user entered the system and the time he logged out, as well as the duration of the user's session in the system are indicated. If the user's work was interrupted only due to the system shutdown, the word "down" is used instead of the user's exit time (these lines can easily determine the system shutdown time). The restart time is displayed on separate lines beginning with the word "reboot".
Command lastb like a team last but displays information about unsuccessful user logon attempts. However, it should be noted that this command will only work if the file exists. / var / log / btmp... However, none of the programs discussed here create log files, so if any of them is deleted, then the recording ends.
Command lastlog formats and outputs the content of the file / var / log / lastlog... The fields will be the username, the name of the terminal from which the user logged in, and the time of the last login to the system. By default (when the command is entered without parameters) file elements / var / log / lastlog will be displayed in order of user ID numbers. If you specify the -u login-name parameter, only information about the time of the last login of the specified user will be displayed. By specifying the -t days parameter, you will only get records from the last days. If the user has not logged into the system at all, then the string "** Never logged in **" will be indicated instead of the terminal name and the time of the last login.
When executing the command lastlog on a slow computer, in some cases it may appear that the command is stuck. This happens due to the fact that even if only two users are registered in the system (root and user), in the file / var / log / lastlog there is still room for as many users as possible who can work on the system. Therefore, in the file / var / log / lastlog there may be large gaps between the ID numbers of the users logged into the system. Since when viewing such intervals, the program does not display information on the screen, and the impression of "freezing" appears.
To display information about who is currently working in the system, use the commands w, who and users... Command users it is used when you only want to know which of the users is currently working in the system, but are not interested in which terminal he connected from and what he is doing. If you want to know who logged in from which terminal, use the command who... This command outputs information from a file / var / run / utmp... You can make it output data from a file / var / log / wtmp(or any other file for which it makes sense) if you specify the name of that file on the command line. But in the output, you will still see only the username, an indication of the terminal from which the user logged in, the login time, and, if logged in from a remote computer, the name of that computer.
Significantly more information is displayed by the command w... In its output, you will see the current time, how long the system has been running, how many users are currently working in the system and the average load of the system for the last minute, 5 and 15 minutes. Then, for each user, it prints:
As already mentioned, the command w outputs information stored in a file utmp... By the way, the guide man states that ordinary users should be denied write access to the file utmp since many system programs(for some inexplicable reason) depend on its integrity. You run the risk of confusing system statistics files and making changes to system files if you allow any user to write to a utmp file.
Log files of other programs
In addition to the files that we have already described, there are other protocol files that are created by separate programs. The most typical examples are daemons' protocols. samba, ftpd or httpd which are kept in separate files. Some of these programs create their protocols in subdirectories of the directory / var / log / others keep the protocols elsewhere. And the structure of these files may differ significantly from the structure of files created by the system. syslog... For example, I will give a few lines from the server log Apache running on my computer: 192.168.36.21 - - "GET / ve / papers / new / log / HTTP / 1.1" 200 1774 "-" "Mozilla / 5.0 (X11; U; Linux i686; ru-RU; rv: 1.0 .0) Gecko / 20020530 "192.168.36.21 - -" GET /icons/back.gif HTTP / 1.1 "304 -" - "" Mozilla / 5.0 (X11; U; Linux i686; ru-RU; rv: 1.0.0 ) Gecko / 20020530 "192.168.36.21 - -" GET /icons/folder.gif HTTP / 1.1 "304 -" - "" Mozilla / 5.0 (X11; U; Linux i686; ru-RU; rv: 1.0.0) Gecko / 20020530 "192.168.36.21 - -" GET /icons/text.gif HTTP / 1.1 "304 -" - "" Mozilla / 5.0 (X11; U; Linux i686; ru-RU; rv: 1.0.0) Gecko / 20020530 "192.168.36.21 - -" GET /ve/papers/new/log/protok_lovim.htm HTTP / 1.1 "200 46597" http: // linux / ve / papers / new / log / "" Mozilla / 5.0 (X11; U ; Linux i686; ru-RU; rv: 1.0.0) Gecko / 20020530 "192.168.36.21 - -" GET /bugtraq.css HTTP / 1.1 "404 279" http: // linux / ve / papers / new / log / protok_lovim.htm "" Mozilla / 5.0 (X11; U; Linux i686; ru-RU; rv: 1.0.0) Gecko / 20020530 "192.168.36.21 - -" GET /img/bug1.gif HTTP / 1.1 "404 280" http: // linux / ve / papers / new / lo g / protok_lovim.htm "" Mozilla / 5.0 (X11; U; Linux i686; ru-RU; rv: 1.0.0) Gecko / 20020530 "192.168.36.21 - -" GET /img/title.gif HTTP / 1.1 "404 281" http: //linux/ve/papers/new/log/protok_lovim.htm "" Mozilla /5.0 (X11; U; Linux i686; ru-RU; rv: 1.0.0) Gecko / 20020530 "As you can see, the structure of this log file is significantly different from what we saw in the system logs.The Samba server, in addition to the main server operation protocol, creates in the subdirectory / var / log / samba a number of log files for different cases, in particular, separate files for each of the users who are allowed to use the resources provided by this server. The following two records are taken from one such file:
Smbd / service.c: make_connection (550) linux (192.168.36.10) connect to service public as user kos (uid = 500, gid = 500) (pid 1366) smbd / service.c: close_cnum (550) linux (192.168. 36.10) closed connection to service public These examples show that if you can read a little in English and have some understanding of the structure of the records, you can extract a lot of useful information from the log files. Only it has to be extracted from entire deposits of "waste rock" - huge sequential log files, which is a non-trivial task. Therefore, special software tools for protocol analysis were developed.
Tools for processing protocols
A lot of different programs and scripts for protocol analysis have been developed. I will limit myself brief description two such means: logwatch and swatch.
logwatch is a Perl script included with the standard Red Hat Linux distribution. Just during the preparation of this article, version 4.1 of this program was released on the developer's website (and he is Kirk Bauer).
The main idea behind this program is that the log file is passed through a filter that extracts from the log all lines (that is, messages) that meet a given criterion. The results are sent by email to the specified user (default is root). Filters can be written in any programming language, but the author of the package prefers Perl. Filters should be written so that they read data from stdin and output the result to stdout.
The main way to use logwatch consists in including a link to the main script ( /etc/log.d/scripts/logwatch.pl) to the /etc/cron.daily directory, which causes the daily execution logwatch with default parameters. The link is given a name that starts with "00" (eg 00-logwatch) so that the script runs before logrotate.
But you can run the script from the command line as well. Of course, if you did not change the information output parameter, then the result of its work should be looked for in the superuser's mailbox. If you want to see these results on the screen, then you must set the command line parameter --print- issue a report to stdout.
General script launch format:
/etc/log.d/scripts/logwatch.pl [--detail level ] [--logfile group-logs ] [--service service-name ] [--print] [--mailto address ] [--save File name ] [--archives] [--range date-interval ]
The default parameters are stored in the /etc/log.d/logwatch.conf file, comments in which allow you to understand the meaning of the command line parameters:
- LogDir - the directory relative to which the file names are considered;
- MailTo - to whom to send the report;
- Print - instead of sending a report by mail, issue it to stdout;
- Save - instead of sending the report by mail, save it in the specified file;
- Archives - process not only current versions of logs, but also old copies created by logrotate;
- Range - process the specified time interval: All, Today, Yesterday (yesterday's calendar day);
- Detail - level of detail of the report: from 0 to 10 or Low, Med, High;
- Service - All or filter name from /etc/log.d/scripts/services/ (multiple filters can be specified);
- LogFile - All or the name of the log group (multiple groups can be specified).
More information about the script logwatch you will find in.
This article describes another script for processing log files that is included with the Mandrake Linux distribution. This script is called swatch("Simple WATCHer") and is also written in Perl.
Work management swatch carried out using a single configuration file, by default $ HOME / .swatchrc... This file contains sample text (in regular expression form) that swatch will search the log files. Each such pattern is followed by an action that swatch should take if it finds text that matches the pattern.
For example, you might want to receive a warning every time you attempt a buffer-overflow attack on your web server for a very long file name. And you know that in such cases in the log file /var/apache/error.log a message appears with the words "File name too long". In this case, to your file .swatchrc the following note must be inserted:
Watchfor / File name too long / mail [email protected], subject = BufferOverflow_attempt
I will not give a more detailed description of the program here. swatch... An enthusiastic article is dedicated to her, and the program itself can be found on the website. I would only like to note, and swatch, and logwatch implement a rather primitive algorithm for processing protocol files, which boils down to searching the protocol for a given character string (signature). Meanwhile, firstly, the presence of such a line often does not yet indicate an intruder's intrusion, and, secondly, a competent attacker can take care to erase the traces of his activity. Another obvious drawback of the reviewed products is that they work in "delayed mode", since they are launched only on schedule. And the fight against intruders must be carried out "in real time", immediately reacting to attempts to penetrate the system. Therefore, commercial products offer monitoring systems that work constantly and implement "intelligent" algorithms for analyzing protocols. These algorithms are based on a statistical analysis of the message flow and the identification of statistically significant deviations of the system from its normal behavior.
In conclusion of this section, I would like to note that the SecurityLab.ru website (http://www.securitylab.ru/tools/?ID=22111) contains a list of links to the sites of various software tools for processing protocols with a brief description of these tools. You can experiment with different programs and choose the one that suits you.
Rotating log files
You will of course understand that if the system is heavily used, then the log files grow rapidly. Which entails a waste of disk space. And the problem of "taming" the protocols arises. Red Hat Linux fixes this problem with scripts. logrotate which is located in the directory /etc/cron.daily, and, therefore, is started by the daemon cron daily. This script allows you to process more than just the system logs syslog, but also any other programs.Script logrotate monitors the growth of log files and provides the so-called rotation of these files if they have exceeded the specified size (or after the specified time interval has expired). Rotation is nothing more than sequential copying of previous versions of archive files, something like this:
and creating a new messages file to record subsequent messages.
List of files to be processed by the script logrotate and the parameters of this processing are determined by configuration files, of which there can be several. The names of the configuration files are set in the command line of the script launch (see below for the launch parameters). On Red Hat Linux, the default configuration files for logrotate file used /etc/logrotate.conf, which may look something like this:
Weekly rotate 4 create compress include /etc/logrotate.d / var / log / wtmp / var / log / lastlog (monthly create 0664 root utmp rotate 1)
This file is like any configuration file for the script. logrotate consists of several sections. The first section defines global parameters (one per line) that set the default parameters for all logs. The following sections define local parameters for a series of log files. The names of these files are listed in the first line of the section, and then local parameters are set in curly braces that are effective only when processing the specified files (also one parameter per line). Log file names can be quoted by shell rules if they contain spaces or other special characters. You can specify multiple file names or file name patterns separated by spaces (patterns also follow shell rules). Processing each section is treated as a single action. Lines starting with the "#" character are comments. Local parameters take precedence over global parameters.
In the given example of the configuration file, the global parameters are described first, and then, in a separate section, the parameters for processing the / var / log / wtmp and / var / log / lastlog files. In addition, among the global parameters, a link to the directory is given /etc/logrotate.d, into which each package writes local parameters for its logs.
In the section of global parameters, first of all, one of the following parameters is specified, which determine the criterion for file rotation:
and parameter include followed by the name of another (additional) configuration file or even the name of a directory. In the latter case, all files in the specified directory, except for subdirectories, special files and files with suffixes from the list of exclusions, are considered configuration files for the script. logrotate(directive include cannot be used inside a section that specifies processing parameters for a group of files).
Parameter rotate number can be found both among global and among local parameters and determines how many old versions should be kept; if the number is 0, then archived versions of the protocol are not created.
If the parameter is set compress then old versions are compressed using gzip, and if specified nocompress- they do not shrink. Parameter compresscmd allows you to specify which compression program will be used (gzip by default), and uncompresscmd sets the decompression program (ungzip by default). compressoptions sets the parameters of the compression program; the default is "-9", i.e. maximum compression for gzip. Using the parameter compressext you can change the suffix for compressed files, and the parameter extension suffix specifies the suffix to add to filenames during rotation (before the compression suffix).
Among the keywords found in configuration files, the words postrotate and prerotate which serve as open parentheses for including shell scripts in configuration files. All lines of the config file from line postrotate to the line endscript are executed as shell commands after the process of changing the version of the log file. Accordingly, all lines from the line prerotate to the line endscript are executed before the rotation of the log files. With the help of these scripts, you can organize various procedures for processing the log files during the rotation.
Using other parameters of the configuration file, you can override the access rights to the log files (if this parameter is not specified, the attributes of the old log file are used); indicate to whom to send messages about errors in the operation of the logging system; send an archive copy of the log to the specified user; set the directory to which the protocols will be moved during the version change (the directory must be located on the same physical device as / var / log) or specify a list of exclusion suffixes for the directory include... A detailed description of these features and parameters (as well as those that were not mentioned) can be found under the command man logrotate.
As already mentioned, additional configuration files logrotate can be specified on the command line of the script launch. You can specify an arbitrary number of names for configuration files or directories. The names of files and directories in this list are simply separated by spaces. The order of the names in the list is important because the parameters specified in the following configuration file override the values of the parameters specified in previous file... The order of the files in the configuration directory is undefined.
In addition, the following parameters can be specified on the startup command line:
- -d- debug mode, no real changes are made,
- -f- make changes even if logrotate does not see the need - it is used when there are changes in the list of processed logs,
- man logwatch
- Mick Bauer, "Paranoid Penguin: swatch: Automated Log Monitoring for the Vigilant but Lazy"
- RFC 3164. C. Lonvick, The BSD Syslog Protocol, August 2001.
- RFC 3195. D. New, M. Rose, Reliable Delivery for syslog, November 2001.
- Per. S.Lapshansky, "The demon monitors the system"
- Denis Kolisnichenko,
The syslog protocol and support software ensure that event information is written to the system log (logs, system console), as well as transmitted to the logging server over the network, sorted and processed depending on the source and severity of the messages. This article describes the syslog protocol, its implementation in Solaris and Linux (syslogd), Cisco IOS, and auxiliary tools: logrotate, logwatch.
The system components are a message generator (device or process), an exchange protocol, a message collector (collector, syslog server), relays (relay, receives messages from one or more generators and transmits to one or more collectors or next relays). The generator (or relay in transmission) does not know whether the receiver is a relay or a collector, it can transmit one message to several receivers, it can process the message itself (for example, by writing to a file).
The syslog protocol does not provide any protection against message spoofing. Even worse, the use of UDP protocol allows attackers to send messages on behalf of any host. The local network must be protected by a screen (IOS ACL, ipchains) from receiving packets with fake local addresses (although this does not prevent sending fake messages from inside the local network) and from receiving packets from the outside on port 514 / udp. There are known cases of disk overflow with false messages.
The syslog protocol and UDP do not provide guaranteed delivery (messages can be lost during network congestion or intercepted, corrupted messages are deleted without warning), correct delivery sequence (the message about the end of the process may come before the message about its start), priority delivery.
Messages are not kept confidential as they are transmitted in clear text.
If, when configuring the message generator, you specify the wrong address of the collector or relay, then there will be no error messages - the messages will be deleted (or written to someone else's log;).
Means of protection against looping the transmission of messages by incorrectly configured relays are not provided.
RFC 3195 proposes a new protocol over TCP (syslog-conn, 601) to ensure delivery in the correct sequence. The implementation is not known to me. A monstrous mixture of XML, SMTP-style commands and return codes, and MIME headers (BEEP) wrapped in standard syslog messages. Two modes are used - RAW (analogue of the current UDP protocol) and COOKED (messages are structured by fields). BEEP allows for authentication, integrity and confidentiality (SASL, TLS).
The syslog-sign RFC proposes to provide authentication, ordering, message integrity and missing message detection by generating special messages containing a digital signature of a block of previous messages while maintaining the standard syslog protocol and format and using UDP. Used by SHA1, OpenPGP DSA.
Port 514 / UDP is used to receive messages. It is also recommended to use source port 514 for messaging. The message is a text string and cannot be more than 1024 bytes in length, otherwise it is allowed to be truncated or even discarded. Even a malformed message sent to port 514 should be considered a syslog message. However, if the relay passes the message further, then it must add standard headers to it (at the same time, possibly cutting it to 1024 bytes) - USER, NOTICE, its local time and the simple name of the message source.
The message begins with a PRI field that encoded the facility and Severity level of the message. It is followed by time (TIMESTAMP), space, host name or IP address in decimal notation (HOSTNAME), space, arbitrary message text (MSG) in US-ASCII (0x20 - 0x7e).
The hostname (simple, not FQDN!) Is written as known to the message generator. If the device has several interfaces with different IP addresses, then any of them can be used as the host name or address.
The message text (MSG) usually contains a tag (TAG) that identifies the program or process that issued the message and the message body (CONTENT). The label can contain Latin letters and numbers. The beginning of the message body is identified by the first special character, usually a colon or opening square bracket. For example, Cisco IOS uses a sequential message number followed by a colon as a label, and Unix uses a simple program name (the message body begins with the process number in square brackets and a colon).
syslogd receives messages from port 514 / UDP and from local sources (socket / dev / log), routes them depending on the source of messages and the severity level. Allows you to output messages to the log, output to the console, to the terminal, send to another server. An additional source type MARK is introduced (regular marks, info)
Each line of the log contains a single message record, consisting of fields separated by spaces:
- date in standard text format (TIMESTAMP field from syslog message)
- hostname (fqdn or shorthand, HOSTNAME field from syslog message)
- message text (TAG and CONTENT fields from syslog message)
Launch parameters:
- -a additional-listening-socket (useful for daemons doing chroot; there may be several)
- -d(debug mode)
- -f config-file-name (default, /etc/syslog.conf)
- -h(change the usual behavior whereby messages received from remote hosts are not forwarded to be written to the remote host to avoid looping)
- -l host-list (list of hosts whose names should not be written as FQDNs; separated by colons)
- -m minutes (interval for regular temporary recordings; by default - 20; if 0, then do not do at all)
- -n
- -p listening-socket (default: / dev / log)
- -r(allow receiving messages from remote hosts; firewall should be slightly open; syslog for 514 / udp should be defined in / etc / services)
- -s domain-list (strip the names of the specified domains from hostnames; separated by colons; by default, the domain that matches the domain of the syslog server is truncated)
- -v
- -x(prohibit determining the hostname by its address, prevents deadlock when working on the same host with the DNS server)
Files used:
- /etc/syslog.conf- configuration file (changed at startup by parameter -f)
- / dev / log- socket from which local messages are read (changed at startup by parameter -p)
- /var/run/syslogd.pid- process id
Reaction to signals:
- SIGHUP - reinitialization (closes all files, reads the configuration file again)
- SIGTERM - shutdown
- SIGINT, SIGQUIT - exit if debugging is disabled
- SIGUSR1 - enable / disable debugging (only when using the -d switch)
Runs as root. Doesn't change file permissions. If he is forced to create a file, he does it with 644 permissions. If it is necessary to restrict access to the log, the corresponding file must be created manually (or changed permissions). Creates special problems logrotate.
It is a set of message routing rules. Each rule consists of a selector and action, separated by tabs (Solaris 5 on older systems) or spaces (Linux). Upon receiving a message to be written to the log (from klogd, from a local or remote program), syslogd checks for each rule if the message matches the pattern specified by the selector. If applicable, the action specified in the rule is performed. For one message m. an arbitrary number of actions have been performed (i.e., message processing does not stop at the first success).
The selector has two parts, separated by a dot: the source of the message and the severity. Uppercase and lowercase letters are the same. You can also use numbers (see /usr/include/syslog.h). In addition to the sources defined in syslog.3, you can specify mark(regular timestamps) security(deprecated synonym for auth). In addition to the severity levels defined in syslog.h, you can use warn(synonym for warning), error(synonym for err), panic(synonym for emerg). Messages with a level equal to or higher than that specified in the selector and a source equal to that specified in the selector are considered valid. An asterisk before a point corresponds to any source, after a point - to any level. Word none after the dot - no level for the given source. You can specify multiple sources in one selector (separated by commas). Multiple selectors can be specified on one line. The semantics are not clear: if you use positive selectors, then the logical OR if negative ( none and an exclamation mark), then the logical AND.
In the new syslogd (linux), the level can be preceded by an equal sign - only messages with the specified level (but not the highest) will match the selector; exclamation mark - messages with a level equal to or greater will not match; exclamation mark and equal will not match messages with a level equal to the specified one.
As an action, you can specify:
- regular file name (full path from the root), minus in front of the name disables record synchronization
- named channels - fifo (a vertical line is placed in front of the name), the channel itself should be created before starting syslogd with mkfifo command
- terminal or console
- @hostname (pass messages for remote logging)
- list of users (separated by commas), to whose terminals the message will be sent
- an asterisk to send a message to all terminals (wall)
When parsing a config file syslogd compares the address loghost(defined in / etc / hosts, not via DNS) with the address of your computer and, if it matches, defines the variable LOGHOST... The syslog.conf is then passed through the m4 (1) macro processor. Basically, this is used so that the same config file can be used on client and server (from a syslog point of view) hosts.
Starting and stopping procedure: /etc/init.d/syslog(links to it from the /etc/rc?.d directories). The process number is stored in /etc/syslog.pid.
klogd reads kernel messages (either through / proc / kmsg or system calls), determines the level, translates command addresses to program names, and sends a message to syslogd.
Launch parameters:
- -c level(messages of this level and less serious will be sent to syslog, and more serious ones will be output to the console; by default - 7; 0 cannot be specified)
- -d(debug mode)
- -f File name (log to the specified file instead of syslog)
- -i(reload module symbols in already running klogd, must be used every time a module is loaded or unloaded; hopefully current versions of insmod, rmmod and modprobe do it themselves)
- -I(reload kernel and module symbols in already running klogd)
- -k File name (use the specified file as the kernel symbol table instead of /boot/System.map)
- -n(do not go to the background; required to run from init)
- -o(one-time mode - log all messages accumulated in the kernel buffer and exit)
- -p(just in case, reload the module symbol table at the time of address translation - the kernel may not be able to do this)
- -s(use only system calls and don't go to / proc / kmsg to get the original messages)
- -v(show version and finish)
- -x(do not convert addresses to names)
- -2 (kernel crash messages - Oops! - are issued twice: before converting addresses to names - for ksymoops - and after)
Kernel message levels (determined by the number in angle brackets and converted to syslog severity, unchanged on file output):
- KERN_EMERG - 0 (system is unusable)
- KERN_ALERT - 1 (action must be taken immediately)
- KERN_CRIT - 2 (critical conditions)
- KERN_ERR - 3 (error conditions)
- KERN_WARNING - 4 (warning conditions)
- KERN_NOTICE - 5 (normal but significant condition)
- KERN_INFO - 6 (informational)
- KERN_DEBUG - 7 (debug-level messages)
Reaction to signals:
- SIGINT, SIGKILL, SIGTERM and SIGHUP - shutdown
- SIGTSTP - stop logging
- SIGCONT - resume, possibly choosing another source of messages (/ proc / kmsg or system calls)
- SIGUSR1 - Reload Symbols of Modules
- SIGUSR2 - Reload kernel and module symbols
The process number is stored in /var/run/klogd.pid.
Logging initialization: openlog- the standard prefix added to all subsequent messages is indicated (usually the program name, process number in square brackets and a colon); source and options. closelog- end logging. syslog- logging (specifies the source, severity and format of the string as in printf).
logrotate(version 3.2-1 / 3.3.2-1 / 3.5.9) - Fight against growing logs: rotation (versioning), compression, deletion and mailing. Runs daily by cron ( /etc/cron.daily/logrotate) and allows you to process logs if they exceed the specified size or at the specified time interval. Allows you to process not only syslog logs, but also any other programs. Options:
- [-d](debug mode, no real changes are made)
- [-f](make changes even if logrotate does not see the need - it is used when changes in the list of processed logs)
- [-s statefilename ] (the current state of the logs is stored in this file between runs, by default - /var/lib/logrotate.status)
- configfilenames (names separated by spaces; order matters; if a directory name is specified, then each file in it is considered a configuration file; RH uses a file /etc/logrotate.conf and directory /etc/logrotate.d)
The configuration file defines global parameters (one per line) that set the default parameters for all logs. For each batch of logs processed, local parameters are specified: the base filename is specified, followed by the local parameters in curly braces, one per line. The filename can be quoted by shell rules if it contains spaces or other special characters. You can specify multiple file names or file name patterns separated by spaces (patterns also follow shell rules). Processing each section is treated as a single action. Lines starting with the "#" character are comments. The parameters specified in the following configuration file override the values of the parameters specified in the previous file. Local parameters take precedence over global parameters. The order of the files in the configuration directory is undefined.
Options:
- compress | nocompress(older versions are compressed or not compressed with gzip)
- compresscmd(sets the compression program, gzip by default)
- uncompresscmd(sets the decompression program, by default - ungzip)
- compressext(sets the suffix for compressed files)
- compressoptions(sets the parameters of the compression program; the default is "-9", i.e. the maximum compression for gzip)
- copytruncate | nocopytruncate(usually the old version is renamed and a new version of the log is created; when this parameter is set logrotate copies the log to a new file and then truncates the old one; used if the program creating the log cannot close it; records made between copying and truncating are lost ; But does it help if the program that creates the log, instead of append mode, simply writes to a file using an internal pointer?)
- create [ access-rights owner group] | nocreate(immediately after renaming the old version of the journal and before calling postrotate a new log is created with the specified attributes - permissions are set in octal, as in chmod.2; if attributes are not specified, they are taken from the old log)
- daily(version change in the series occurs daily)
- delaycompress | nodelaycompress(some programs do not immediately close the log, in which case the compression should be postponed until the next cycle)
- errors email (to whom to send bug reports)
- extension suffix (sets the suffix added to file names when rotating before the compression suffix)
- ifempty | notifempty(change versions even if the file is empty; this is the default)
- include File name | directory-name (textually substitute a file or all files from the specified directory; subdirectories, special files and files with suffixes from the list of exclusions are not included; cannot be used inside a section)
- mail address | nomail(when a version change makes it necessary to delete the old log, then send it to the specified address)
- mailfirst(send not the deleted version of the journal, but the first one)
- maillast(send the deleted version of the log; this is the default)
- missingok | nomissingok(do not send error messages if the log is missing)
- monthly(version change occurs monthly)
- olddir directory | noolddir(during a version change, the log is moved to the specified directory; should be on the same physical device)
- postrotate endscript are executed as shell commands after the upgrade process)
- prerotate(all further lines up to the line endscript are executed before the upgrade process)
- rotate number (how many old versions to keep; if 0, then none)
- size byte (version change occurs if the size of the log has exceeded the specified number; you can use the suffixes "k" - kilobytes - and "M" - megabytes)
- sharedscripts | nosharedscripts(execute commands prerotate and postrotate only once for all files described in the section)
- tabooext [+ ] suffix-list (specifying a list of suffix exceptions for include; if the plus sign is specified, then addition, otherwise replacement; defaults: .rpmorig, .rpmsave, .rpmnew, ", v", .swp and "~")
- weekly(version change occurs weekly)
In delivery RH /etc/logrotate.conf describes the global parameters and parameters for / var / log / wtmp and / var / log / lastlog and refers to the directory /etc/logrotate.d, into which each package writes local parameters for its logs.
logwatch is a framework for writing programs (called filters) to extract useful information from numerous, large and varied logs (not just syslog). The "package" contains several filters designed for Red Hat Linux (some kind of ancient version, since inetd is mentioned instead of xinetd), but you will have to adapt them to the specific situation yourself. The last change was made by the author in September 2000, so there is no need to wait for further development.
Filters can be written in any programming language, but the author of the package prefers perl. Filters should be written so that they read data from stdin and output the result to stdout. Before calling the filter, the environment variables are set: LOGWATCH_DATE_RANGE, LOGWATCH_DETAIL_LEVEL, LOGWATCH_TEMP_DIR, LOGWATCH_DEBUG. The main program is also written in perl: /etc/log.d/scripts/logwatch.pl(/etc/log.d/logwatch, / usr / sbin / logwatch and /etc/cron.daily/00-logwatch are symbolic links to it).
Directory /etc/log.d/conf/logfiles/ contains configuration files of log groups, in which records of maintained services are stored. Each group is described in a separate file group-name.conf, in which are set:
- LogFile = name of the file containing the log, or name pattern; multiple names or patterns can be specified; names m. relative to LogDir
- Archive = the name of the file created by the logrotate of the archived version of the log, or a naming pattern; multiple names or patterns can be specified; names m. relative to LogDir
- filter names ( only once, although the other is shown!) from /etc/log.d/scripts/shared/ as
*filter-name = options , for example, to filter the log by date, if it is written in the standard syslog format, use the line:
* ApplyStdDate =
Directory /etc/log.d/conf/services/ contains the configuration files for the services whose log entries logwatch will process. Each service is described in a separate file service-name.conf, in which are set:
- LogFile = log group name
- filter names from /etc/log.d/scripts/shared/ as
*filter-name = options before the service filter - $environment-variable-name = meaning
Directory /etc/log.d/scripts/logfiles/ contains filters for processing log groups: when processing a group of logs, all files in the directory /etc/log.d/scripts/logfiles/ group-name used as filters.
Directory /etc/log.d/scripts/services/ contains filters for processing records of specific services.
Directory /etc/log.d/scripts/shared/ contains general filters used in log group config files:
- applystddate - filters the log by the required date, if it is written in syslog format (here and in private filters by date, insert LANG = before calling date, otherwise Mar does not coincide with Mar in any way;)
- expandrepeat - turns the "last message repeated" lines into the corresponding number of message lines from the previous line
- onlycontains - leaves only those log lines that contain the specified string (I put quotes around "$ *")
- onlyservice - extracts from the log in syslog format the lines related to the specified service (the service name is passed as a parameter)
- remove - leaves only those log lines in syslog format that do not contain the specified line ( I put quotes around "$ *" and did remove1, remove2, etc. since I did not understand how to specify several subpatterns for egrep in one line; by the way, the parameters are substituted into the shell, so special characters cannot be used either)
- removeheaders - remove standard fields (date, time, hostname, service label and process number)
- removeservice - extracts lines from the log in syslog format that are not related to the specified service (the service name is passed as a parameter)
The default parameters are stored in the /etc/log.d/conf/logwatch.conf file (/etc/log.d/logwatch.conf has a symbolic link to it), comments in which allow you to understand the meaning of the parameters:
- LogDir - the directory relative to which the filenames are considered
- MailTo - to whom to send the report
- Print - instead of sending a report by mail, issue it to stdout
- Save - instead of sending the report by mail, it will save it in the specified file
- Archives - use versions of logs generated by logrotate
- Range - considered time interval: All, Today, Yesterday (yesterday's calendar day)
- Detail - the level of detail of the report: from 0 to 10 or Low, Med, High
- Service - All or filter name from /etc/log.d/scripts/services/ (multiple filters can be specified)
- LogFile - All or the name of the log group (multiple groups can be specified)
Launch parameters:
- --detail level (level of detail of the report: high, med or low)
- --logfile group-logs (process only logs of this group; the group is specified by a symbolic name in the configuration file; multiple groups can be specified)
- --service service-name (process only those entries in the logs that relate to this service; the service is specified by a symbolic name in the configuration file; multiple services can be specified; name All calls processing of records for all services)
- --print(report to stdout)
- --mailto address (send a report to the specified address)
- --save File name (write the report to the specified file)
- --archives(process not only current versions of logs, but also old copies created by logrotate)
- --range date-interval (process only those entries in the logs that relate to the given time interval: Yesterday, Today, All)
The main use is to include a 00-logwatch file (starting at "00" to run before logrotate) in the /etc/cron.daily directory, which causes logwatch to run daily with default options.
Unfortunately, all filters are designed so that the logs are written on the same host that the service is running on.
All logs are kept on one computer (if you have paranoid tendencies, you can record logs on two servers at once).
The correspondence between the formal name of the source and the real device or program:
- local0 - Cisco
- local3 - ftp (there is a special source name, but Solaris 2.5 doesn't know it)
- local4 - reserved for accounting
- local5 - POP3 / IMAP
- local6 - tac_plus>
A screen for port 514 / udp should be open on the server (you can restrict the source addresses of packets, but this will only help from accidents). Starting syslogd (parameters in /etc/rc.d/init.d/syslog or / etc / sysconfig / syslog) must be started with the keys "-r -m 0" (and also "-x" if the same computer is running DNS server). Start klogd with "-2 -c 1" switches. Syslog.conf setup:
- * .crit - messages of CRIT severity level and higher to be sent to terminals and written to a separate file (chmod 600), your messages to be sent to the backup server; sendmail considers messages about problems with mail reception as critical
- kern - create a kern file for all levels of messages (chmod 600)
- mail - create a mail file for messages of all levels (no synchronization)
- auth, authpriv - create a secure file for messages of all levels (chmod 600)
- news - in the news directory, create a separate file for each severity level (debug without synchronization)
- cron - create a cron file for messages of all levels (cron in RH 6.2 and Solaris 2.5 cannot use syslog)
- local0 - create a separate file for each severity level in the cisco directory (err and below without synchronization)
- local3 - in the ftp directory, create a separate file for each severity level (info and debug without synchronization)
- local5 - create imap.log file for messages of all levels
- local6 - create a tac_plus.log file for messages of all levels
- local7 - boot.log file (messages when the system boots and syslogd and klogd start or stop)
- all messages of INFO level and higher that did not get into one of the files defined above, write to the messages file (chmod 600)
On client computers, configure syslog so that all messages are sent to the syslog server, error messages are duplicated in / var / log / syslog, critical status messages are duplicated to the console, user terminals. On linux machines also dump boot messages to the local file (local7, boot.log). The backup syslog server should receive critical messages from the network and write them to a file (hole in the screen, "-r" startup switch).
logrotate: keep forever, change versions as rarely as possible (monthly, except squid), dump to separate directories (except squid) and compress (in delayed mode, except ftpd, linuxconf, sendfax), send errors and deleted files to me. Match parameters for syslog.
Placing a pipe symbol (|) in front of the filename will allow you to use fifo (first in - first out, first came - first came out) or named pipe as a receiver for messages. Before starting (or restarting) syslogd, you must create a fifo using the mkfifo command. Sometimes fifos are used for debugging purposes.
Terminal and Console
Terminal such as / dev / console.
Remote machine
To forward messages to another host, precede the host name with an @ symbol. Note that messages are not forwarded from the receiving host. (for this assignment to work on the client and server in the file / etc / services the line must be written syslog 514 / udp, and UTP port 514 is open)
a list of users
A comma-separated list of users receiving messages (if the user is logged in). This often includes the root user.
All registered users
To notify all registered users with the wall command, use the asterisk (*) symbol.
An example of an uncomplicated syslog.conf:
# Output all kernel messages to the console. # kern. * / dev / console # All logs at info level or higher, except for email messages, and # do not log authentication messages and cron daemon messages! * .info; mail.none; authpriv.none; cron.none / var / log / messages # Write messages containing confidential # authentication information to a separate file, regardless of their level. authpriv. * / var / log / secure # All messages from the mail system should also be written to a separate file. mail. * - / var / log / maillog # Log scheduler messages to / var / log / cron cron. * / var / log / cron # Emergency messages should be received immediately # all system users * .emerg * # Save messages news of crit level and higher into a separate file. uucp, news.crit / var / log / spooler # Save boot messages to boot.log local7. * /var/log/boot.log
As with many config files, the syntax is as follows:
- lines starting with # and blank lines are ignored.
- The * symbol can be used to indicate all categories or all priorities.
- The special keyword none indicates that no logging for this category should be performed for this action.
- A hyphen in front of the file name (like / var / log / maillog in this example) indicates that the log should not be synchronized after each entry. In the event of a system crash, you may lose information, but disabling synchronization will improve performance.
In the syntax of the config file, you can put before priority sign! to indicate that the action should not be applied, from this level and above... Similarly, the priority can be given to sign = to indicate that the rule applies only to that level, or != to indicate that the rule applies to all levels except this one. Some examples are shown below (many other examples can be found in man syslog.conf):
# Send all kernel messages to / var / log / kernel. # Send all critical and higher messages to the remote sysloger machine and to the console # Send all info, notice and warning messages to / var / log / kernel-info # kern. * / Var / log / kernel kern.crit @sysloger kern .crit / dev / console kern.info; kern.! err / var / log / kernel-info # Send all mail system messages except info level to / var / log / mail. mail. *; mail.! = info / var / log / mail
I tried to show the work of syslogd as clearly as possible in the diagram:
Starting the syslogd daemon
The logging daemon is started at the stage of system initialization by means of a script /etc/rc.d/init.d/syslog, however, in order to set launch parameters, there is no need to adjust this script - starting from version 7.2, launch options are read from a separate configuration file / etc / sysconfig / syslog (/ etc / default / syslogin debian).
Here are some possible syslogd daemon startup options:
- -a / folder / socket- specifying an additional listening socket (do not forget to create a socket first)
- -d- debug mode. In this case, the daemon does not go into the background and sends all messages to the current terminal;
- -f config-file-name... Specifies the name of an alternative configuration file to be used instead of the default /etc/syslog.conf;
- -l host-list- setting a list of hosts whose names should not be recorded with the indication of the full domain name (FQDN - Full Qwalified Domain Name);
- -m minutes- sysklogd launched without this option logs messages of the mark category (timestamps) every 20 minutes. Using the -m option, you can either change the interval between marks, or completely cancel the issuance of such messages;
- -p socket- setting an alternative UNIX socket (instead of listening by default / dev / log);
- -r- permission to receive messages from remote hosts;
- -x- prohibition of determining the host name by its address to prevent freezing when working on the same host with the DNS server.
- -v- show version and finish work
After starting the syslogd daemon, a status file is generated / var / lock / subsys / syslog zero length, and file with process ID /var/run/syslogd.pid.
Using the command
kill -SIGNAL `cat / var / run / syslogd.pid`
can be sent to the syslogd daemon one of the following signals: SIGHUP- restarting the demon; SIGTERM- completion of work; SIGUSR1- enable / disable debug mode.
There are actually two logging daemons running on the system - syslogd and klogd... Both daemons are included in the package sysklogd.
The klogd daemon responsible for logging events occurring in the core of the system... The need for a separate klogd daemon is because the kernel cannot use the standard syslog function. The point is that the standard C libraries (including the library that contains the syslog function) are intended to be used only conventional applications... Since the kernel also needs logging functionality, it includes its own libraries that are not available to applications. Therefore, the kernel uses its own message generation mechanism.
The klogd daemon is designed to organize the processing of these messages. In principle, it can do this processing completely independently and independently of syslogd, for example, by writing these messages to a file, but in most cases the default klogd setting is used, in which all messages from the kernel are forwarded to the same syslogd daemon.
Automatic rotation (updating full files) and archiving of logs
Over time, the log file tends to grow, especially with intensive work of any service. Accordingly, you need to be able to control the size of the logs. This is done with logrotate commands which is usually done cron daemon... I will talk about the work of cron in the following articles. the main objective logrotate commands is to periodically back up the logs and create new clean logs. Multiple generations of logs are retained, and when the last generation log reaches its expiration date, it can be archived (compressed). The result can be sent by mail, for example, to the person in charge of maintaining the archives.
To determine the order of rotation and archiving of logs, use config file /etc/logrotate.conf ... For different journals, you can set a different frequency, for example, daily, weekly or monthly, in addition, you can adjust the number of accumulated generations, as well as specify whether copies of archives will be sent to the person in charge of maintaining archives and, if so, when. Shown below example /etc/logrotate.conf file:
# the "default" parameters (global options) are set first # update the log files weekly weekly # keep an archive of logs for the last 4 weeks rotate 4 # create a new (empty) file after rotation (update) create # uncomment if you want the saved ones files were compressed #compress # enable rotation settings from the specified directory include /etc/logrotate.d # do not store wtmp, or btmp - log data rotation settings are as follows: / var / log / wtmp (missingok monthly create 0664 root utmp rotate 1) / var / log / btmp (missingok monthly create 0664 root utmp rotate 1) # specific system logs can be configured below
Global options are placed at the beginning logrotate.conf file... They are used by default if nothing more specific is specified elsewhere. In the example, the rotation of the journals occurs weekly and the backups are kept for four weeks. As soon as the journal is rotated, a new one is automatically created in place of the old journal. Logrotate.conf file may contain specifications from other files. So, it includes all files from the directory /etc/logrotate.d.
This example also contains special rules for / var / log / wtmp and / var / log / btmp(storing information about successful and unsuccessful attempts to log into the system), which are rotated monthly. If the files are missing, no error message is displayed. A new file is created and only one backup is saved.
In this example, when the backup reaches the latest generation, it is deleted because it is not defined what to do with it.
Log backups can also be generated when logs reach a certain size, and scripts can be generated from sets of commands to execute before or after a backup operation. Example:
/ var / log / messages (rotate 5 mail [email protected] size 100k postrotate / usr / bin / killall -HUP syslogd endscript)
In this example, the rotation / var / log / messages produced when it reaches 100 KB. Five backups are accumulated, and when the oldest backup expires, it is mailed to [email protected] The postrotate command word enables a script to restart the syslogd daemon after rotation is complete by sending a HUP signal. The endscript command word is required to terminate the script and also if the prerotate script is available. See the logrotate man pages for more information.
Options given in the configuration file logrotate.conf:
- compress| nocompress(older versions are compressed or not compressed with gzip)
- compresscmd(sets the compression program, gzip by default)
- uncompresscmd(sets the decompression program, by default - ungzip)
- compressext(sets the suffix for compressed files)
- compressoptions(sets the parameters of the compression program; the default is "-9", i.e. the maximum compression for gzip)
- copytruncate| nocopytruncate(usually the old version is renamed and a new version of the log is created; when this parameter is set, logrotate copies the log to a new file, and then truncates the old one; used if the program creating the log cannot close it; records made between copying and truncating are lost ; but does it help if the program that creates the log, instead of append mode, simply writes to a file using an internal pointer?)
- create[permissions owner group] | nocreate(immediately after renaming the old version of the log and before calling postrotate, a new log is created with the specified attributes - permissions are set in octal, as in chmod.2; if no attributes are specified, they are taken from the old log)
- daily(version change in the series occurs daily)
- delaycompress| nodelaycompress(some programs do not immediately close the log, in which case the compression should be postponed until the next cycle)
- errorsemail(to whom to send bug reports)
- extensionsuffix(sets the suffix added to file names when rotating before the compression suffix)
- ifempty| notifempty(change versions even if the file is empty; this is the default)
- includeFile name| directory-name (textually substitute a file or all files from the specified directory; subdirectories, special files and files with suffixes from the list of exclusions are not included; cannot be used inside a section)
- mailaddress| nomail(when a version change makes it necessary to delete the old log, then send it to the specified address)
- mailfirst(send not the deleted version of the journal, but the first one)
- maillast(send the deleted version of the log; this is the default)
- missingok| nomissingok(do not send error messages if the log is missing)
- monthly(version change occurs monthly)
- olddirdirectory| noolddir(during a version change, the log is moved to the specified directory; should be on the same physical device)
- postrotate(all further lines up to the endscript line are executed as shell commands after the version change process)
- prerotate(all further lines up to the endscript line are executed before the version change process)
- rotatenumber(how many old versions to keep; if 0, then none)
- sizebyte(version change occurs if the size of the log has exceeded the specified number; you can use the suffixes "k" - kilobytes - and "M" - megabytes)
- sharedscripts| nosharedscripts(execute the prerotate and postrotate commands only once for all files described in the section)
- tabooext[+] suffix-list(specifying a list of suffix-exclusions for include; if the plus sign is specified, then addition, otherwise replacement; by default: .rpmorig, .rpmsave, .rpmnew, ", v", .swp and "~")
- weekly(version change occurs weekly)
Exploring and monitoring logs
Log entries usually contain a timestamp, the hostname on which the described process is running, and the process name. View logs you can use a pagination program, for example, less, you can search for specific entries (for example, kernel messages from a specific daemon) using the command grep:
# less / var / log / messages # grep "ppp" / var / log / messages | tail Dec 17 16:34:25 proxy pppd: Connection terminated. Dec 17 16:34:25 proxy pppd: Exit. Dec 17 16:35:57 proxy pppd: LCP terminated by peer (^ P] kV ^ @ The computer may not work all the time and turn off, say at night. Therefore, records in / var / log / messages are stored cyclically from computer startup to shutdown, this can be seen from the messages: Dec 17 08:32:56 syslog-server syslogd 1.4-0: restart. Dec 17 08:32:56 syslog-server syslog: syslogd succeeded Dec 17 08:32:56 syslog-server kernel: klogd 1.4-0, log source = / proc / kmsg started. Dec 17 08:32:56 syslog-server syslog: start klogd succeeded Dec 17 08:32:56 syslog-server kernel: Kernel command line: auto BOOT_IMAGE = linux ro root = 303 BOOT_FILE = / boot / vmlinuz-2.4.2-2 Dec 17 08:32:56 syslog-server kernel: Memory: 125652k / 130560k available (1365k kernel code, 4200k reserved, 92k data, 236k init, 0k highmem) Dec 17 08:32:56 syslog-server kernel: CPU: Intel (R) Pentium (R) 4 CPU 1.60GHz stepping 02 Also, in this file you can find information about disk memory (including information about disk geometry, partition structure and interrupts used), information about peripheral devices, about starting individual services and services, information about connecting file systems and messages about user logon. as well as error messages. Sometimes it may be necessary monitoring system logs in order to search for current events. For example, you can try to catch a rare event at the moment it happened. In this case, you can use the command tail with option -f to track the contents of the syslog. Example: # tail -f / var / log / messages | grep syslog-server Dec 17 16:46:09 syslog-server pppd: pptpd-logwtmp.so ip-up ppp0 maikop 94.77.0.150 Dec 17 16:46:09 syslog-server pppd: Script / etc / ppp / ip-up finished (pid 12552), status = 0x0 Dec 17 16:46:49 syslog-server pptpd: CTRL: Client 85.175.197.65 control connection started Dec 17 16:46:49 syslog-server pptpd: CTRL: Starting call (launching pppd, opening GRE) Dec 17 16:46:49 syslog-server pppd: Plugin /usr/lib/pptpd/pptpd-logwtmp.so loaded. In addition to the log files specified in /etc/syslog.conf, there are also other files, for example, a file that stores information about the system boot process before starting syslogd, as well as files that have a binary format and store information about the last entry user logon, all successful user logins, and all unsuccessful user logons, respectively. Also, the / var / log / directory can contain the log files of daemons such as a web server or a proxy server. The format of these files is similar to the syslogd logs. Finally, I would like to emphasize that this protocol is not very secure, tk. syslog does not contain any means of protecting against message spoofing. Worse, the use of UDP allows attackers to send messages on behalf of any host. Your local network should be screened from receiving packets with fake local addresses (although this does not prevent you from sending fake messages from inside the local network) and from receiving packets outside on port 514 / udp. There are known cases of disk overflow with false messages. The syslog protocol and UDP do not provide guaranteed delivery (messages can be lost during network congestion or intercepted, corrupted messages are deleted without warning), correct delivery sequence (the message about the end of the process may come before the message about its start), priority delivery. Messages are not kept confidential as they are transmitted in clear text. If, when configuring the message generator, you specify the wrong address of the collector or relay, then there will be no error messages - the messages will be deleted (or written to someone else's log). Several projects have been proposed to improve the syslog protocol. For example, RFC 3195 proposes a TCP-based logging system (syslog-conn) to ensure that messages are delivered in the correct sequence. The syslog-sign project proposes to provide authentication, ordering, message integrity and missing message detection by generating special messages containing a digital signature of a block of previous messages while maintaining the standard syslog protocol and format and using UDP. Let's summarize: Linux has a single daemon responsible for logging events on the local system and remote systems. All events are collected from the / dev / log socket, UDP port - 514, as well as from the "helper" - the klogd daemon, which sends messages from the kernel. All collected messages are filtered by the syslogd daemon through the rules in the /etc/syslog.conf file and distributed according to the rules to the appropriate destinations. Log files are periodically "truncated". The frequency is determined by the logrotate.conf file and the logrotate command, which is run by the system scheduler - cron. That's all for today. I hope I described everything as clearly as possible. Over time, I will supplement the article! Regards, Mc.Sim! SERGEY SUPRUNOV FreeBSD tips: using syslog - Excuse me, comrades, I have all the moves written down!
“The office is writing,” said Ostap.
I. Ilf, E. Petrov "12 chairs".
Logging the work of any system, and above all the server, is one of the most important components of administration. It is in the log files that we look first of all when there are some problems in the system. From there we derive confidence that this or that program works as expected. When developing web applications, the log file becomes the most important source of debugging information. The features of the syslog daemon, which is responsible for logging system information, will be discussed. What is syslog Syslog is a centralized service that collects and records log information from various operating system components and user processes. Third-party programs are usually able to work with their log files on their own, although many of them can be configured to work with the syslogd daemon. The advantages of using syslog include the ability to control the process of collecting the necessary information using a single configuration file, which ensures the consistency of the received log-files and, as a result, simplifies their management. The syslog service is provided by the syslogd daemon, which is automatically started at system startup. It is constantly in memory, waiting for messages from other processes and processing them in accordance with its settings. Each message is characterized by a level and a facility. The level specifies the degree of importance of the message from the point of view of the functioning of the system. The syslog.h file defines the following levels (priorities): Table 1. Message levels Level Description emerg, panic Panic state alert A condition requiring immediate attention crit Critical situation err, error Other work errors warning, warn Warnings notice Messages that are not errors but should be noted info Info messages (normal operation) debug Debug information Table 1 lists the message levels in descending order. This order will be needed later in the discussion of the syntax of the configuration file. Message level designators written on the same line (eg emerg and panic) are synonyms, and any of them can be used. However, panic, error, and warn (listed second in the table) are deprecated and should be avoided when editing the configuration file. Use emerg, err, and warning instead. Possible sources of the message are listed in Table 2: Table 2. Sources of messages A source Description kern Kernel messages auth, authpriv security Security messages (e.g. from a firewall) console Messages coming to the console (/ dev / console) syslog Syslog native messages cron Cron subsystem messages Time service messages Posts FTP servers Print Subsystem Messages mail Messages from postal services news News Service Messages uucp Posts UUCP daemon Messages from other daemons running on the system user User process messages local0 - local7 Can be used for various user needs (sometimes also used by the system) When configuring an application to work with the syslog service, check which facility it is using. Some programs allow you to specify the source yourself. For example, the clamd daemon (the main process of the clamav antivirus) uses local6 by default, but allows you to specify a different source (the LogFacility parameter in the configuration file). In some cases, it can be useful to combine its messages with messages from mail services, specifying LOG_MAIL as the source. Daemon startup options A complete list of startup options can be found in the syslogd man page. Here we will consider only a few of them. Syslog is able to write incoming messages to log files (which are usually located in the / var / log directory), send them to the console or a connected user terminal, send them to a remote host, or pipe them to an external utility for processing. For networking, syslog uses ports 514 (UDP and TCP). Syslogd, started without additional parameters, will listen on these ports for incoming messages. The -s switch disables incoming messages, but sockets on ports 514 are still created for outgoing connections so that syslogd can send messages to remote hosts. Doubling the key (-ss) also denies outgoing connections. By default, syslogd is started with the -s switch (see /etc/defaults/rc.conf, syslogd_flags parameter), so incoming connections are denied. If you want to use your server's syslog service to serve multiple machines, add the following line to /etc/rc.conf: syslogd_flags = ””
This will override the value set in /etc/defaults/rc.conf so that syslogd can handle incoming connections. Naturally, in this case it is necessary to ensure the required level of security, excluding the possibility of foreign hosts writing anything to your logs. The -a switch allows you to set restrictions on incoming connections (see man syslogd). A properly configured firewall also plays an important role. Another useful switch, -l, allows you to specify additional socket files that syslogd listens to in addition to the default / var / run / log. For example, this switch is required so that syslog can handle programs running in a chroot environment. In particular, this is how named works since BIND 9. On FreeBSD 5.3, syslogd is started with the following keys: # ps -wax | grep syslog Configuration file syntax Logging is configured in the /etc/syslog.conf file. It goes without saying that only the root user can edit it. This file contains two kinds of strings - let's call them "filters" and "rules". The filter string specifies the program or host to apply the following rules to messages from. A filter like "! Prog" (or "#! Prog") indicates that the following lines refer to the logs generated by the specified prog program. The synonym for this entry is "! + Prog". Conversely, the line "! -Prog" means that the following rules will apply to all messages except those coming from the prog program. It is allowed to list several programs separated by commas, while the "+" or "-" sign is applied to the entire list. If the filter string is specified as "+ hostname", then the specified host will be used as the source of messages to be processed by subsequent rules. As in the case of programs, the “-” symbol indicates that the rules will apply to all messages except those coming from the specified host. You can specify a list of hosts separated by commas. The "*" character specified as a program or host will override the previously set filter. That is, the rules specified after such a line will apply to all messages. Filters with a hostname or program are independent of each other and can operate at the same time. For example: # Rules located here apply to all messages My.host # Rules apply to all messages from my.host Logger # Rules apply to messages from logger with my.host (host filter continues) # Rules apply to messages from su with my.host # Rules apply to messages from su from any hosts (host filter is canceled, program filter continues) # The rules are applied to all messages (the program filter is also canceled) The rule lines are as follows: facility.CMPlevel destination Here facility is the source of the message ("*" stands for any sources), level is the level of messages that will be processed in accordance with this rule. The service word "none", specified instead of the level, instructs to completely exclude messages from this source. CMP - comparison operation (characters ">", "<», «=», «>=», «<=», а также символ отрицания «!»). Если символ сравнения не указан, подразумевается «больше или равно», то есть обрабатываются все сообщения уровня level и выше.
Destination specifies where this message should be saved. This can be a log file (the full path is indicated, starting with "/"), the address of the remote server (starting with the "@" symbol), a username (messages will be sent to the terminal to which this user is connected). Also, the message can be passed to an external program for processing, for which the pipe symbol "|" is used. A few examples of rules All kernel messages will be sent to the kern.log file. kern. * /var/log/kern.log All error messages as well as emerg, alert and crit level messages will be placed in all-err.log. Notice the hyphen in front of the log file name. By default, messages are buffered in memory and written to disk as the buffer fills. This reduces the load on the file system, but it can cause some information to be lost when the machine crashes. The hyphen in front of the file name causes the syslogd daemon to immediately write messages to disk. * .err - / var / log / all-err.log Debug messages from user processes will be output to the terminal to which the user vasya is currently connected. user.debug vasya All messages from news and e-mail services will be forwarded to port 514 of the syslog.host.ru machine. mail. *, news. * @ syslog.host.ru All print service messages below warning will be placed in the specified file. As stated earlier, the default is greater than or equal to comparison, and the! it inverts, replacing it with "less". If you need to exclude a specific level, you should use the "! =" Construction. lpr.! warning /var/log/printers.log Debug level messages (and only debug messages) with facility level0 and level3 will be output to all connected terminals. level0, level3. = debug * Time Service messages that are higher than critical (i.e. emerg and alert) and less than or equal to notice (notice, info, and debug) will be sent to the ntpuser and root terminals. ntp.> crit,<=notice ntpuser,root
All warning messages except those from mail services will be written to the warn.log file. *. = warning, mail.none /var/log/warn.log In this case, all messages with a level greater than or equal to crit will be emailed to the root user. * .crit | mail -s “critical message” root As you can see, the configuration file format allows multiple levels, sources and destinations to be listed on one line, which makes the configuration more flexible. For the changes to take effect after editing, you need to send a HUP signal to the syslogd process: # kill –HUP `cat / var / run / syslog.pid` It should be noted that if this or that message falls under the action of several lines in the configuration file, then it will be processed in accordance with each of them. Example: mail. * / var / log / maillog *. = err /var/log/error.log In this case, mail.err messages will go to both the maillog and error.log. Configuration File Strategy Concluding my consideration of the configuration file, I will say a few words about the strategies for its compilation. In addition to the very popular "if only it works", two main ones can be distinguished, which we will conditionally call "by source" and "by purpose". Logger utility The system includes a logger utility that allows you to send messages to the syslog service directly from the command line. It is convenient to use it when testing configuration rules, as well as in developing scripts for logging their work. Examples: user $ logger -p user.err “Error in user program!” user $ logger -h syslog.host.ru “Test it” The first example will send an err-level facility user message, which will be processed according to the configuration file. The second command will send a message to the remote host, source and default level will be set to user.notice. Using rotation mechanisms The logging mechanism discussed above does not provide any kind of log file management. Messages are simply placed in them according to the rules described in the configuration file. This means that the log files will constantly grow, and if nothing is done, sooner or later they will fill up all the available space on the corresponding partition. To avoid this, a rotation mechanism is used. For example, as soon as the debug.log file exceeds 100 MB, it is renamed to debug.log.0 and packed into debug.log.0.bz2. A new debug.log is created instead. Then, when the size of the new file reaches 100 MB, debug.log.0.bz2 is renamed to debug.log.1.bz2, and the above procedure is repeated. The system stores only a certain number of archive files, deleting the oldest ones. This is rotation. Newsyslog utility On FreeBSD, rotation is handled by the newsyslog utility, which by default is run at the beginning of every hour by the cron daemon. You can change the rotation period by editing the / etc / crontab file. The rotation parameters specified in the above example (file size, packing), as well as a number of others, are set in the /etc/newsyslog.conf configuration file. For each file that needs rotation, it contains a line of generally nine fields: file name, owner and group, access rights, largest number in the archive file name, maximum file size, rotation period, additional flags, and the path to The PID file of the application to send the signal after rotation, and the number of the signal to be sent. (You might need to send a signal to a process, for example, if the process keeps the log file open all the time; otherwise, the file descriptor used will not match the newly created file.) Some fields can be omitted. Here are two examples, complete and typical: / var / log / pflog root: wheel 600 3 100 * JB /var/run/pflog.pid 1 According to this rule, the pflog file must be overwritten as soon as its size exceeds 100 MB (5th field), regardless of the time ("*" in the 6th field). Archive files will have names like pflog.X.bz2 (pflog.0.bz2, pflog.1.bz2, etc.), and X cannot be more than three (parameter 3 in the 4th field). The J flag indicates that the archive file should be compressed using the bzip2 utility. Thanks to the B flag, the service messages about rotation will not be added to the archived and newly created files, since the file is perceived as binary. The newly created file will be owned by the root user and the wheel group (this field could have been omitted since this is the default owner). The permissions will be set to rw ------- (numeric value 600), which means that only the owner can read and write to this file. Finally, a 1 (HUP) signal will be sent to the process whose PID is stored in /var/run/pflog.pid to reopen its log file. / var / log / maillog 640 7 * @ T00 J This rule specifies the rotation parameters for the maillog file: regardless of size (an asterisk in the 5th field), the file will be rewritten every night at 00:00 (man newsyslog.conf describes the format for specifying the time in some detail). The archive must be packed, the last 8 archives will be saved (with numbers from 0 to 7 inclusive), the newly created file will be owned by the root user (the default value, so the owner field is omitted) and have the rw-r ----- permissions. You can use the zcat and bzcat utilities (for gzip and bzip2, respectively) to view the packed log files. Midnight Commander invokes the appropriate utilities automatically. After editing the newsyslog.conf file, no signals need to be sent anywhere - the configuration will be re-read the next time newsyslog is called. Note that the newsyslog utility is not tied to the syslogd daemon. That is, it can be used to rotate any files that need it. If rotation is enabled for logs that the application maintains on its own (for example, for clamd or squid logs), then pay particular attention to specifying the owner and access rights. Incorrectly specifying them can lead to the fact that after rotation an application launched as an unprivileged user will not be able to write to the newly created file. Summing up The syslog system is a very powerful and efficient tool for logging various events occurring in the system. The default settings are fairly balanced and work well for most systems. However, understanding how syslog works will allow you to significantly improve the quality of logging by customizing the logging to meet your specific needs.
321 ?? Ss 0: 01.67 / usr / sbin / syslogd -l / var / run / log -l / var / named / var / run / log -s
