The NT Insider

Beware the Guarded Mutex
Blocking Special Kernel APCs at IRQL PASSIVE_LEVEL (By: The NT Insider, Vol 11, Issue 2, Mar-Apr 2004 | Published: 08-Apr-04| Modified: 03-May-04)

In the folklore of file systems, we know that IRQL APC_LEVEL is used to prevent the delivery of all forms of APC - both normal and kernel mode APCs.  This is because paging I/O operations are always sent at APC_LEVEL.  However, beginning in Windows Server 2003, this is no longer the case.  Instead, the Memory Manager now uses guarded mutexes. These are similar to fast mutexes except that instead of raising the IRQL of the processor, they merely block the delivery of all forms of kernel APC object.

Thus, for those developing file systems and file system filter drivers, it is now important to use KeAreApcsDisabled rather than just relying upon the IRQL of the system (it would be wonderful if KeAreApcsDisabled returned TRUE if IRQL > PASSIVE_LEVEL but right now that is not the case).  This is particularly critical because if special kernel APCs are disabled, no calls should be passed that rely upon I/O completion processing - otherwise the operation might hang and never completed.

For a network file system, no calls should be made to TDI if KeAreApcsDisabled returns TRUE.  Similarly, no kernel component should call any of the Zw API operations for I/O operations if KeAreApcsDisabled returns TRUE.

Note that in the current (Windows Server 2003, Build 3790) the documentation for KeAreApcsDisabled is incorrect - it says "The system still delivers special kernel APCs even if KeAreApcsDisabled returns TRUE".  This was certainly true in Windows XP and earlier versions, but is not the case in 2003.  This can be seen from the disassembly (from Windows Server 2003, Build 3790):

0: kd> u nt!KeAreApcsDisabled
nt!KeAreApcsDisabled:
80827bf0 64a124010000     mov     eax,fs:[00000124]
80827bf6 83787000         cmp     dword ptr [eax+0x70],0x0
80827bfa 0f95c0           setne   al
80827bfd c3               ret

Thus, this returns the DWORD value at offset 0x70 in the KTHREAD (fs:[124] is always the address of the current thread). From the command dt nt!_KTHREAD we can find the value at offset 0x70:

   +0x070 KernelApcDisable : Int2B
   +0x072 SpecialApcDisable : Int2B
   +0x070 CombinedApcDisable : Uint4B

This is quite different than on Windows XP:

lkd> u nt!KeAreApcsDisabled
nt!KeAreApcsDisabled:
804f3756 64a124010000  mov eax,fs:[00000124]
804f375c 33c9          xor ecx,ecx
804f375e 3988d4000000  cmp [eax+0xd4],ecx
804f3764 0f95c1        setne cl
804f3767 8ac1          mov al,cl
804f3769 c3            ret

And at offset 0xd4:

+0x0d4 KernelApcDisable : Uint4B

There is no concept of SpecialApcDisable in Windows XP. In Windows Server 2003, however, this function will return TRUE if special kernel APCs are disabled through this explicit mechanism.

This can be a rather substantial change (we found this here at OSR while testing a low level I/O routine in a file system.  The system hung, we could see the special kernel APC had not been delivered and yet the thread was blocked at PASSIVE_LEVEL.)  It is an unfortunate oversight that this was not documented in the IFS Kit for Windows Server 2003, although we can hope that this oversight is addressed in the SP1 kit.

This article was printed from OSR Online http://www.osronline.com

Copyright 2017 OSR Open Systems Resources, Inc.