Yesterday, I wrote about our init actually being /bin/sh, since all sysvinit does it start a single shell script.

Today, I’m going a bit into how services are brought up with sysv-style init. I won’t be writing much about runlevels, since most users will not make use of multiple runlevels anyway.

So what does a typical init script in /etc/init.d do?

If you want to find out yourself, consider looking at /etc/init.d/skeleton, which is a template for init scripts.

A modern init script should (as required by the LSB) start with a formatted comment with metadata such as dependencies and description. This is already trying to address some shortcomings of sysv-style init, but no actual code.

The first thing most init scripts do is to check if their application is actually still installed. Init scripts are sometimes created by users, or are not automatically uninstalled since being treated as ‘configuration files’; so it occurs often that the init script is still around even when the service was uninstalled. So they test if they’re installed and executable.

Next they probably read a configuration file, usually named /etc/default/packagename, with user settings (people used to do such changes in the init script itself, but for upgrading it proved to be a lot better to put all settings the user might want to modify there).

The third step is to do the requested user action, which usually is one of ‘start’, ‘stop’, ‘reload’ and ‘restart’. Some init scripts also have extra actions such as ‘status’; not all init scripts support ‘reload’. Doing such an action usually means to print out a status message and launching the service in the background (usually using the ‘start-stop-daemon’ daemon helper, which offers functionality to e.g. record the process PID in a file or forcing applications into the background that don’t do so on their own), or locating the service (usually via a PID file, again this is commonly done using start-stop-daemon) and reloading or stopping it. Upon success, they again print a status message.

An init script could be as simple as:

#!/bin/sh
if [ "$1" = "start" ]; then
  /usr/local/sbin/mycheapdaemon &
fi

There are many good reasons not to do it that way, but it will work.

Some characteristics of sysv-style init scripts:

  • When starting or stopping services manually, init is not involved. Or notified. (In particular this means init will still try to stop the service on shutdown, and will not try to stop services that aren’t on it’s list of services to stop.) - there is no system-wide state tracking.
  • if a service quits, nobody notices
  • init scripts do a best-effort to return a meaningful status if the startup of the service was successful. But often this is not possible (e.g. when a daemon backgrounds unconditionally before checking if it has a working configuration)
  • init scripts are often run by the root user, and can sometimes even be executed successfully by non-root-users. This is particularly annoying with SELinux in ‘strict’ mode, that tries to differentiate between applications started by the system administrator and system services. For full protection, you need to use run_init /etc/init.d/script start, which will transition into the system user role, then start the service.
  • many services aren’t very cooperative and need the help of start-stop-daemon for writing pid files and/or backgrounding
  • Since services are started/stopped by the admin occasionally (e.g. during upgrades), and init scripts are supposed to print a status message (these “Starting service foobar: foobard [done]” messages), they will have file descriptors open to the admins console. They will also inherit environment variables from the admin. When a daemon fails to handle them appropriately, odd things can happen. This ranges from services dying when the admin logs out (and they quite on a SIGPIPE signal triggered by the file descriptor being closed; this is why ‘nohup’ in the coreutils package exists. An example is IBMs tivoli backup software) to e.g. tomcat behaving differently when it has a $DISPLAY variable (i.e. started with a graphical output available).

Keep your eyes open for the next part of this series, where I’ll show how some init systems improve on these issues.

[Update: next part]