Linux Magic SysRq Key

What is the "Magic" SysRq key?

According to the Linux kernel documentation:

It is a 'magical' key combo you can hit which the kernel will respond to regardless of whatever else it is doing, unless it is completely locked up.

The sysrq key is one of the best (and sometimes the only) way to determine what a machine is really doing. It is useful when a system appears to be "hung" or for diagnosing elusive, transient, kernel-related problems.

How do I enable and disable the SysRq key?
To use it, you need to have it enabled in your kernel (CONFIG_MAGIC_SYSRQ).
It usually is.
For security reasons, Red Hat Enterprise Linux disables the SysRq key by default. To enable it, run:
# echo 1 > /proc/sys/kernel/sysrq

To disable it:

# echo 0 > /proc/sys/kernel/sysrq

To enable it permanently, set the kernel.sysrq value in /etc/sysctl.conf to 1. That will cause it to be enabled on reboot.
kernel.sysrq = 1
Since enabling sysrq gives someone with console access extra abilities, it is recommended to disable it when not troubleshooting a problem or to ensure that console access is properly secured.

How do I trigger a sysrq event?
There are several ways to trigger a sysrq event. On a normal system, with an AT keyboard, sysrq events can be triggered from the console with the following key combo:

Alt+PrintScreen+ [CommandKey]

For instance, to tell the kernel to dump memory info (command key "m"), you would hold down the Alt and Print Screen keys, and then hit the m key.
Note that this will not work from an XWindows screen. You should first change to a text virtual terminal. Hit Ctrl+Alt+F1 to switch to the first virtual console prior to hitting the sysrq key combo.
On a serial console, you can achieve the same effect by sending a Breaksignal to the console and then hitting the command key within 5 seconds.
If you have a root shell on the machine (and the system is responding enough for you to do so), you can also write the command key character to the /proc/sysrq-trigger file. This is useful for triggering this info when you are not on the system console or for triggering it from scripts.

# echo 'm' > /proc/sysrq-trigger

If you find that holding three keys is difficult, you do have some other choices. You can trigger it manually:
echo t > /proc/sysrq-trigger

Or, you can make the Alt-SysRq "sticky" by:
echo 1 > sysrq-sticky

With that done, you can hold Alt-SysRq, release it, and then leisurely press "t" or whatever key you wanted.

You could also change the SysRq itself in /proc/sys/kernel/sysrq-key (it's normally 84 - Alt-SysRq for Intel).


When I trigger a sysrq event that generates output, where does it go?

When a sysrq command is triggered, the kernel will print out the information to the kernel ring buffer and to the system console. This information is normally logged via syslog to /var/log/messages.
Unfortunately, when dealing with machines that are extremely unresponsive, syslogd is often unable to log these events. In these situations, provisioning a serial console is often recommended for collecting the data.

What sort of sysrq events can be triggered?

There are several sysrq events that can be triggered once the sysrq facility is enabled. These vary somewhat between kernel versions, but there are a few that are commonly used:
· m - dump information about memory allocation
· t - dump thread state information
· p - dump current CPU registers and flags
· c - intentionally crash the system (useful for forcing a disk or netdump)
· s - immediately sync all mounted filesystems
· u - immediately remount all filesystems read-only
· b - immediately reboot the machine
· o - immediately power off the machine (if configured and supported)

Alt+SysRq+r takes keyboard and mouse control from the X server. This can be useful if the X-Server crashed, you can change to a console and kill the X-Server or check the error log.
Alt+SysRQ+k kills all processes on the current terminal. Its a bad idea to do this on a console where X is running. The graphic will stop and you cant see what you type.
Alt+SysRQ+b is like a reset: a reboot without umounting or sync.
Alt+SysRQ+o shuts down via APM.
Alt+SysRQ+s writes all data from the disc cache to the hard discs, its a sync.
Alt+SysRQ+u remounts all mounted file system read-only. After using this key, you can reboot the system with Alt+SysRQ+b without harming the system.
Alt+SysRQ+m prints memory information to the console.
Alt+SysRQ+e sends SIGTERM to all processes except init.
Alt+SysRQ+i sends SIGKILL to all processes except init.
Alt+SysRQ+l sends SIGKILL to all processes, inclusive init. (The system is not working after using this.)...
Linux is very stable, but sometimes a kernel panic could happen, sometimes this stops the X-server and you can’t change to the console. What should be done? Hitting the reset button and risk filesystem integrity?
NO! There is a possibility to shut down the system cleanly or find out the source of the kernel panic.
To shut down the system after a really bad kernel panic, do the following:
Alt+SysRQ+e (sends TERM-signal, processes can shutdown properly (e.g. save data))
Alt+SysRQ+u (a sync will be done when unmounting anyway)
Alt+SysRQ+i (for the processes that didn't listen for the TERM signal, this is a kill -9 process)
Alt+SysRQ+b (reboot).