Understanding Early debug shell, Emergency and rescue targets in CentOS/RHEL 7
If there are problems during the starting of services, there are a few tools available to system administrators that can help with debugging and/or troubleshooting in a CentOS/RHEL 7 and above system.
How systemd boots the system
Now, we can see how systemd implements the bootstrap. systemd is run by the kernel as a result of /sbin/init being symbolically linked to /lib/systemd/systemd. It runs the default target, default.target, which is always a link to a desired target such as multi-user.target for a text login or graphical.target for a graphical environment. For example, if the default target is multi-user.target, you will find this symbolic link:
/etc/systemd/system/default.target -> /lib/systemd/system/multi-user.target
The default target maybe overridden by passing system.unit= on the kernel command line. You can use systemctl to find out the default target, as shown here:
# systemctl get-default
multi-user.target
Starting a target such as multi-user.target creates a tree of dependencies that bring the system into a working state. In a typical system, multi-user.target depends on basic.target, which depends on sysinit.target, which depends on the services that need to be started early. You can print a graph using systemctl list-dependencies .
To list all the services and their current states:
# systemctl list-units --type service
Similarly, to list all the available target:
# systemctl list-units --type target
Early debug shell
By running systemctl enable debug-shell.service, a root shell will be spawned on TTY9 (Ctrl+Alt+F9) early during the boot sequence. This shell is automatically logged in as root so that an administrator can use some of the other debugging tools while the system is still booting.
Emergency and rescue targets
By appending either systemd.unit=rescue.target or systemd.unit=emergency.target to the kernel command line from the boot loader, the system will spawn into a special rescue or emergency shell instead of starting normally. Both of these shells require the root password. The emergency target keeps the** root** file system mounted read-only, while **rescue.target **waits for sysinit.target to complete first so that more of the system will be initialized, for example, logging, file systems, etc. Exiting from these shells will continue with the regular boot process.
Stuck jobs
During startup, systemd spawns a number of jobs. If some of these jobs cannot complete, they will block other jobs from running. To inspect the current job list, an administrator can use the command systemctl list-jobs. Any jobs listed as running must complete before the jobs listed as waiting can continue.