CVE-2009-2793: Iret #GP on pre-commit handling failure: the NetBSD case

A few months ago, Tavis Ormandy and myself have used the fact that iret can fail with a General Protection (#GP) exception before the processor "commits" to user-mode (switches privileges by setting CS) on multiple occasions (more on this at upcoming PacSec)

It's not necessarily obvious that an inter-privilege iret (typically from kernel mode to user mode) can fail before the privilege switch occurs. It's however the case if the restored EIP is past the code segment limits: a #GP exception will be raised while in kernel mode.

When this occurs, an exception is raised from kernel mode with a handler in kernel mode: since there is no privilege level switch, no stack switch occurs and the trap frame will not contain saved stack information.

If an operating system's kernel does not expect this to happen, it may assume a full trap frame with saved stack registers. This is what happens in NetBSD.

An interesting point in the NetBSD case is that due to the lazy handling of the non executable stack emulation, a legitimate program could trigger the bug:

1. The legitimate program has code on the stack. For instance due to a GCC-genereated trampoline for a nested function.
2. The stack with be marked as executable but the code segment limit will not be raised yet: on stack execution, the kernel will handle the #GP exception and raise the limit (lazy handling).
3. A signal handler gets set to this nested function
4. The kernel delivers a signal to the process and iret to the code on the stack, such raising #GP pre-commit.

You can read our full NetBSD related advisory here (CVE-2009-2793).