Stephen Rothwell
IBM OzLabs Linux Technology Centre
http://www.ibm.com/linux/
While auditing the running processes on my laptop, I came to the realisation that knowing when particular files are updated is a common problem for many daemons. In all the cases I noticed, the solution used is to poll the files and/or directories containing them using the (f)stat(2) system call. This results in several processes on an otherwise idle system waking up at intervals varying from a few minutes right down to 1 second and stat(2)ing files.
In the past this has seemed a reasonable solution as using up idle time would not have appeared to be a problem and there was no other way to do the job anyway. Today, however, with laptops proliferating and battery technology not keeping up with the power requirements of modern CPUs (and all the gadgets we need installed), maximising the idle time on a machine can be very important. Also, we now have the tools necessary to signal asynchronous events when files and directories are modified.
This paper will discuss alternate methods, including directory notification and file leases, that can be used to eliminate the polling loops in some common programs and the effects that these can have on the power consumption of computers in general.
Introduction
There are a reasonable number of applications running as daemons on a typical Linux system that monitor the status of files and directories while they would be otherwise idle. Currently this is normally done by sleeping for some amount of time and then checking the files and directories using stat(2).
The 2.4 series of Linux kernels introduced two new features to aid in tracking changes in file and and directories - directory notification and file leases. This paper discusses these two new APIs and how to use them in user mode applications in order to aid in power management.
Directory Notification
The directory notification API was introduced in the Linux 2.3 kernel development cycle in order to better support funtionality required by Samba. However, its design was driven by a desire to implement something that had wider application and in this we succeeded.
Usage is fairly straight forward. The directory to be monitored must be kept open by the program while notifications are required. Notifications are delivered via signals. Enabling notifications is achieved using fcntl(2).
To be more concrete, the following includes are required:
#define _GNU_SOURCE
#include <fcntl.h>
#include <signal.h>
#include <unistd.h>
This is the file descriptor that an event occurs on:
int eventd_fd;
Declare a signal handler:
void handler(int sig, siginfo_t *si, void *data)
{
event_fd = si->si_fd;
}
In order to get the siginfo_t structure delivered with the signal, a real time signal must be used:
struct sigaction act;
act.sa_sigaction = handler;
sigemptyset(&act.sa_mask);
act.sa_flags = SA_SIGINFO;
sigaction(SIGRTMIN, &act, NULL);
Open the directory (“.” is a convenient example), associate the real time signal with it and then enable notification:
fd = open(“.”, O_RDONLY);
fcntl(fd, F_SETSIG, SIGRTMIN);
fcntl(fd, F_NOTIFY, DN_MODIFY | DN_CREATE);
This will cause a signal to be delivered if any file in the directory is modified or if a new file is created. If DN_MULTISHOT is added to the notify bitmask, the signal will continue to be delivered until the F_NOTIFY fcntl is called with a zero mask to turn off notications.
It is currently possible to request notifications for the following events (or combinations of these):
File Leases
File leases were also introduced in order to better support Samba, but again they have uses beyond the initial intent. They are most useful (and this is where Samba uses them) to allow for caching of files for remote file system clients. Holding a file lease ensures that no other process has the same file open with an incompatible mode and also provides notification if such an open is attempted.
A file lease can be taken over any open file in either shared (F_RDLCK) or exclusive (F_WRLCK) mode. A shared lease will allow other processes to open the same file read-only, but will cause a notification if an open for writing is attempted. An exclusive lease will cause a notification if any open is attempted on the file. Neither lease will be granted if the file is already open with an incompatible mode. Lastly, a lease may only be obtained by the owner of the file (or a process running with the CAP_LEASE capability).
When a process has a lease on a file and another process attempts to open the file with and incompatible mode, a signal is sent to the first process. The lease owning process should then do whatever cleanup is necessary and release the lease (or convert it from an exclusive to a shared lease if that is sufficient) and/or close the file. If the lease holding process does not respond within a (system wide and configurable) amount of time (45 seconds by default), the lease is unilaterally broken by the operating system. This is to prevent resource starvation.
Concrete example time ...
Again we need some includes and to set up our signalling to get a real time signal delivered with its siginfo_t structure:
#define _GNU_SOURCE
#include <fcntl.h>
#include <signal.h>
#include <unistd.h>
int eventd_fd;
struct sigaction act;
void handler(int sig, siginfo_t *si, void *data)
{
event_fd = si->si_fd;
}
act.sa_sigaction = handler;
sigemptyset(&act.sa_mask);
act.sa_flags = SA_SIGINFO;
sigaction(SIGRTMIN, &act, NULL);
Lastly, open the file, associate the signal with it and acquire the lease:
fd = open(“file”, O_RDWR);
fcntl(fd, F_SETSIG, SIGRTMIN);
fcntl(fd, F_SETLEASE, F_WRLCK);
Once a signal is delivered, a call to fcntl(fd, F_GETLEASE) will return the level of lease required to satisfy the conflicting open. If the open is read only, F_RDLCK will be returned otherwise F_UNLCK. At other times and by other processes, this fcntl can be used to determine the current lease held (if any).
Power Management
After turning off a laptop's screen and spinning down it's disk, the next best way to extend battery life is to keep the microprocessor at it's lowest power level. The kernel can only so this if the user mode processes can give the kernel a hint about their idleness or they actually remain idle for as long as possible.
As noted in the introduction, these two APIs may be used to monitor the modification of files and thus eliminate the need to loop while polling these files. The obvious first example is cron. In normal operation, it should be able to sleep until the next scheduled operation, however, it must also monitor some of the crontab files in case they are monitored. Currently this is done by waking up every minute and stating the files or directories that contain them.
The solution needed is twofold: directory notification to monitor /var/spool/cron/crontabs and /etc/cron.d (at lease on Debian) and a file lease to monitor /etc/crontab. The only down side is that after the file lease causes a notification, cron would have to loop polling the status of /etc/crontab as some editors will write a new file instead of modifying the file directly. As an alternative, cron could monitor /etc, which would not be too bad as files in /etc are not changed all that often.
References
Documentation/dnotify.txt in the Linux kernel sources
Documentation/leases.txt - which should exist, but doesn't
Legalese
IBM is a registered trademark of IBM Corporation in the United States of America and other countries.
Linux is a trademark of Linus Torvalds in the United States of America and other countries.
Copyright ã 2001 Stephen Rothwell, IBM Corporation