If you have ever had the pleasure of debugging a system that blue screened with the stopcode KMODE _EXCEPTION_NOT_HANDLED, then you probably have experienced the joy of trying to debug without the luxury of a simple stack trace.  Most of the time, one of the first things we try to find when analyzing a crashed system or its dump is a stack trace for the thread that stopped the system.  We look for the stack trace because we want to know what code caused the system to crash, and how the system got there. Unfortunately, as is often the situation with KMODE_EXCEPTION_NOT_HANDLED-- or any of its not-so-distant cousins, FAT_FILE_SYSTEM, NTFS_ FILE_SYSTEM, and CDFS_ FILE_SYSTEM--the alleged perpetrator's stack is seemingly damaged resulting in a useless stack trace, and a frustrating debugging experience.  This article will walk through debugging the two major varieties of KMODE_ EXCEPTION_NOT_HANDLED crashes, and obtaining useful stack traces from WinDbg.

Researching the Bug Code

The very first thing I do when looking at a crashed system or a crash dump is inspect the numbers from the top of the blue screen, and try to divine their meaning.  Fortunately, when you are given a crash dump, you do not have to ask the customer what was displayed on the blue screen.  When you open a crash dump, WinDbg displays the important information from the blue screen.  That output is shown in Figure 1. This is analogous to the top two lines of the blue screen, as shown in Figure 2.  "Bugcheck 0000001e" means "KMODE_EXCEPTION _NOT_HANDLED."  If you compare the two figures, you will notice WinDbg does not perform this translation for you.  Fortunately, if you are analyzing a crash dump, you can determine the alphanumeric equivalent of the Bugcheck number by looking in "\NTDDK\INC\BUGCODES.H."      If you search BUGCODES.H,  you will find "KMODE_EXCEPTION _NOT_HANDLED" is defined to be "0000001E."  Of course this still does not tell us exactly what this bugcheck number means, or the four numbers displayed next to it.

Kernel Debugger connection established for F:\dumps\Memory.dmp

Kernel Version 1381 Free loaded @ 0x80100000

Bugcheck 0000001e : c0000005 80112bf0 00000000 0000001d

Figure 1 -- Windbg Displays Blue Screen Information

*** STOP: 0x0000001E (0xC0000005,0x80112BF0,0x00000000,0x0000001D)             

KMODE_EXCEPTION_NOT_HANDLED*** Address 80112bf0 has base at 80100000 - ntoskrnl.

exe

Figure 2 -- Top of Blue Screen

The five numbers together are the five parameters to the function KeBug CheckEx(), as documented in the DDK.  The system or a driver calls KeBugCheckEx() to bring the system down in a controlled fashion when the system or driver detects an unrecoverable error.  Ultimately, this results in a blue screen, and a crash dump if you are lucky and you configured the system to create crash dumps.  We can seek the meaning of these five numbers by referring to Microsoft Knowledge Base Article "Q103059 - Descriptions of Bug Codes for Windows NT." By searching the article for "KMODE_ EXCEPTION_NOT_ HANDLED", we expect to find a description of the bugcode, and the meaning of the next four parameters to KeBugCheckEx(). In the case of "KMODE_EXCEPTION_ NOT_HANDLED", instead of a description of the bugcode, we find a tip on debugging the problem, which we will see the value of shortly, and a terse description of the four parameters.

The Knowledge Base article describes the second parameter to KeBugCheckEx() as "The exception code that was not handled”.  The first time I read this, I wondered what an exception code was until I looked at the value of the second parameter.  The value of the second parameter in Figure 1 is "c0000005."  This looks remarkably like an NTSTATUS value.  If you search for "c0000005" in the DDK's NTSTATUS.H, you will find that STATUS_ACCESS _VIOLATION is defined to be "c0000005."  This section of NTSTATUS.H is labeled "Standard Error values."  Therefore, exception codes are NTSTATUS values. As for STATUS_ ACCESS_VIOLATION, it almost always means a thread tried to dereference an uninitialized, NULL, or corrupted pointer.  This is a very common C programming error, and usually the reason NT crashes.  Consequently, STATUS_ACCESS_VIOLATION is almost always the exception code associated with a KMODE_EXCEPTION_ NOT_HANDLED bugcode.

The third parameter is described as "The address at which the exception occurred."  The first time I read this, I wondered if the address was an instruction address or a data address.  An easy experiment with WinDbg provided the answer shown in Figure 3.  According to the Knowledge Base article's tip on debugging the problem, this address indicates the driver or function at fault.  By that reasoning, there is a serious bug in the kernel's IoDeleteDevice() function.  However, it is entirely possible a driver passed a bad argument to the IoDeleteDevice() function, in which case the driver is probably at fault, not the IoDeleteDevice() function.  At this point we could disassemble IoDeleteDevice() to determine if the function was at fault, or try to obtain a stack trace for the thread that called KeBugCheckEx() to see what driver called IoDeleteDevice(), and what arguments the driver specified.  Knowing that drivers I have written have successfully called IoDeleteDevice() many times prior to this crash, I chose to examine the stack.

KDx86> ln 80112bf0

NT!_IoDeleteDevice@4+0x6

NT!_IoDetachDevice@4-0x62

Figure 3 -- Windbg's List Nearest Symbols Command

Dumping the Stack

WinDbg's "k" command displays the stack of the thread that was being executed by WinDbg's concept of the current processor.  You can modify WinDbg's concept of the current processor by using the "~n" command where "n" is the zero-based index of the processor whose context you wish to examine.  If the system that crashed utilizes multiple processors, you can usually find the thread that called KeBugCheckEx() by starting with processor zero and dumping the stack of each processor's current thread.  Figure 4 illustrates this process.

KDx86> ~0

KDx86> kbvs

FramePtr  RetAddr   Param1   Param2   Param3   Function Name

ffdff600  ffdff600  00000000 00000000 00001564 NT!KiIdleLoop+0xe

KDx86> ~1

KDx86> kbvs

FramePtr  RetAddr   Param1   Param2   Param3   Function Name

f744bb6c  80139a26  f744bb94 8013d5ff f744bb9c NT!PspUnhandledExceptionInSystemThread+0x18 (FPO: [1,0,0])

f744bf7c  8014547e  8010bc1e 00000001 00000000 NT!PspSystemThreadStartup+0x5e(UKNOWN FPO TYPE)

00000000  00000000  00000000 00000000 00000000 NT!KiThreadStartup+0x16

KDx86> ~2

Command Error

Figure 4 -- Finding Stack of Thread that Called KeBugCheckEx

Notice that I use the "bvs" options with the "k" command.  I do not remember what these options mean, but years ago I determined that they produce the most useful output.  "Command error" tells me the system being debugged only contains two processors.  Another important observation is that the function KeBugCheckEx() does not appear on the stack of the threads that were being executed.  WinDbg does not display the KeBugCheckEx() function when you are examining crash dumps, although it does when you are performing "live" system debugging.  Regardless, since the second processor executed the function PspUnhandledExceptionInSystemThread(), and the system crashed with the bugcode KMODE_EXCEPTION_ NOT_HANDLED, it is reasonable to deduce that the second processor's thread called KeBugCheckEx() from within the function PspUnhandledExceptionInSystemThread().  The most important observation about this thread's stack is what is conspicuously missing.  If IoDeleteDevice() generated a STATUS_ACCESS_VIOLATION exception that caused the system to crash, why isn't IoDeleteDevice() displayed as part of the thread's stack?  The answer is exception handling.

Exception Handling

When a thread tries to use a bad pointer, the thread usually incurs a page fault exception.  The processor transfers control to a special function called a trap handler.  In the case of a page fault on an x86 system, that trap handler is named KiTrap0E, as we have seen in many crash dumps.  KiTrap0E inherits its name from the x86 processor's exception number for page faults, namely "E."  The trap handler builds a trap frame which contains enough of the thread's context to restart the thread at the faulting instruction.  Then the trap handler calls the virtual memory manager to resolve the page fault.  The virtual memory manager may have to fetch data from pagefile.sys or a memory mapped file.  If the page fault is resolved, then the trap handler restarts the instruction that caused the page fault.  Otherwise, if the page fault cannot be resolved as is the case with a bad pointer, then the memory manager raises a STATUS_ ACCESS _VIOLATION structured exception.

Structured exception handling, also known as SEH, is similar to the C++ language's exception handling.  Unlike C++'s exception handling, SEH is the NT system's native platform independent exception handling.  For information on how to use SEH in your drivers, refer to the article "The Exception to the Rule:  Structured Exception Handling," in the March-April 1999 issue of The NT Insider.  For additional information on SEH, refer to the Platform SDK.  For in depth information on the mechanics of SEH, read Matt Pietrek's article in MSJ, "A Crash Course on the Depths of Win32 Structured Exception Handling."  If you have an MSDN Library subscription, you will find Matt's article on your Oct. '99 MSDN Library disc.

Briefly, each SEH exception handler registered by the thread that incurred the exception is given a chance to handle the structured exception in LIFO order according to when the exception handler was registered. This usually results in an exception filter function being called for each exception handler. The filter function returns a value that tells the system if the filter function handled the exception, and if so whether the system should restart the faulting instruction or resume execution in the function that registered the exception handler. This may entail unwinding the stack if execution must be resumed in a stack frame preceding the one where the exception occurred.  PspUnhandledExceptionInSystem Thread(), the last function displayed in WinDbg's stack trace, is an exception filter used by system worker threads to catch bugs in worker functions.  PspUnhandledExceptionIn SystemThread() calls KeBugCheckEx() with the parameters we described above.  Consequently, that is why IoDeleteDevice() does not appear in our stack trace.  The stack frame for IoDeleteDevice() is not currently linked into the chain of stack frame pointers starting from KeBugCheckEx().

Back to the Problem at Hand

There is still hope. Since PspUnhandledExceptionIn SystemThread() could have instructed the system to restart the faulting instruction, there should be enough of the stack intact to obtain a stack trace containing IoDeleteDevice().  The problem is finding the context information necessary to instruct WinDbg to display that stack trace.  Specifically, we need to find the thread's context that was recorded by the trap handler when the page fault exception occurred.  This context contains the stack pointer and stack frame pointer that can be used to tell WinDbg where to start a stack trace.

Fortunately, exception filters often need this context information.  As a result, that context information is usually a parameter to an exception filter.  In fact, the first parameter to the exception filter PspUnhandledExceptionInSystem Thread() is a pointer to a structure of type EXCEPTION_POINTERS, which in turn points to a context record and an exception record.  The EXCEPTION _POINTERS structure is documented in the Platform SDK, and can be dumped using a WinDbg extension DLL from the Windows NT Support Tools.  For more information on the NT Support Tools refer to the article, "Life Support for WinDbg -- New Windows NT Support," in the Sept-Oct '98 issue of the NT Insider.

In Figure 5, we took the first argument to PspUnhandled ExceptionInSystemThread() from Figure 4, and dumped the EXCEPTION_POINTERS structure pointed to by the argument.  Then we took the context record pointer contained in the exception pointers structure and dumped the context record using the "cxr" command provided by WinDbg's default extension DLL.  Notice that we have to tell WinDbg to use the default extension DLL by qualifying the "cxr" command with "kdextx86."  Once you have loaded a second extension DLL, WinDbg is confused about which DLL is the default extension DLL.

KDx86> !kdex2x86.strct EXCEPTION_POINTERS f744bb94

Debugger extension library [kdex2x86] loaded

+0000  Structure EXCEPTION_POINTERS (Size:0x8) at 0xf744bb94:

+0000    ExceptionRecord =      f744be2c

+0004    ContextRecord =        f744bc68

KDx86>  !kdextx86.cxr f744bc68

CtxFlags: 00010017

eax=00000000 ebx=00000000 ecx=00015300 edx=f744bf6c esi=f7473d54 edi=00000000

eip=80112bf0 esp=f744bef4 ebp=f744bf1c iopl=0         nv up ei pl zr na po nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010246

0x80112bf0  f6471d08         test        byte ptr [edi+1d],08    

Figure 5 -- Exception Pointers and Context Record

Now that we have the context information, we can tell WinDbg where to begin its stack trace.  We use a special form of the "k" command as shown in Figure 6.  Specifically, we give the "k" command the stack frame pointer, stack pointer, and instruction pointer addresses from the context record.  These are the values that were in the EBP, ESP, and EIP registers at the time the thread incurred the page fault exception.  At last, we have a stack trace that includes the function where the exception occurred, IoDeleteDevice().  A quick inspection of the stack shows that a NULL pointer was passed as the first argument to IoDeleteDevice() from the function DriverUnload() in CRASH.SYS.  This should be enough information to squash the bug.

KDx86> kbvs=f744bf1c f744bef4 80112bf0

FramePtr  RetAddr   Param1   Param2   Param3   Function Name

f744bef8  f75d3526  00000000 f7473d54 f7473d54 NT!IoDeleteDevice+0x6 (FPO: [1,0,2])

f744bf1c  80172087  80713dd0 f7473d54 00000001 CRASH!DriverUnload+0x50(UKNOWN FPO TYPE) [ crash.cpp @ 149 ]

f744bf34  8010bc91  f7473d54 f744bf7c 00000000 NT!IopLoadUnloadDriver+0x17 (FPO: [1,1,3])

f744bf4c  80139a1c  00000001 00000000 00000000 NT!ExpWorkerThread+0x73 (FPO: [ebp f744bf7c] [1,0,4])

f744bf7c  8014547e  8010bc1e 00000001 00000000 NT!PspSystemThreadStartup+0x54(UKNOWN FPO TYPE)

00000000  00000000  00000000 00000000 00000000 NT!KiThreadStartup+0x16

Figure 6 -- Stack Trace Containing IoDeleteDevice

Crosschecking

There are a couple of useful items we took the liberty of skipping in getting to the crux.  First, we have not yet determined the meaning of the fourth and fifth parameters to KeBugCheckEx() when the bugcode is KMODE_ EXCEPTION_NOT_HANDLED.  Second, we did not dump the exception record pointed to by the EXCEPTION_ POINTERS structure.  As it turns out, the exception record and the parameters to KeBugCheckEx() contain the same information.

We have taken the exception record pointer from Figure 5, and used the "exr" command in WinDbg's default extension DLL to dump the exception record.  This is shown in Figure 7.  Obviously, the second parameter to KeBugCheckEx() is the ExceptionCode() field of the exception record.  The ExceptionAddress() field is the third parameter.  Referring back to Microsoft Knowledge Base Article "Q103059 - Descriptions of Bug Codes for Windows NT," it is now obvious that parameter 0 and parameter 1 from the exception RECORD are what the knowledge base article calls parameter 0 and parameter 1 of the exception.  Hence, the fourth and fifth parameters to KeBugCheckEx() are the two parameters from the exception record.

KDx86> !exr f744be2c

Exception Record @ F744BE2C:

ExceptionAddress: 80112bf0 (_IoDeleteDevice@4+0x6)

   ExceptionCode: c0000005

  ExceptionFlags: 00000000

NumberParameters: 2

   Parameter[0]: 00000000

   Parameter[1]: 0000001d

Figure 7 -- Exception Record

These two parameters from the exception record are documented in the Platform SDK.  If the ExceptionCode() is STATUS_ACCESS_VIOLATION, which it usually is for KMODE_EXCEPTION_NOT_HANDLED, then the first parameter from the exception record tells us whether or not the instruction that caused the exception was trying to read or write to memory.  The second parameter tells us what data address the instruction was trying to read or write.  If we had not obtained a reasonable stack trace in Figure 6, we could have used this information to crosscheck our debugging in WinDbg.  This would tell us whether or not we found a valid context record.

For example, if you take the faulting instruction, "test byte ptr [edi+1d],08," from the context record in Figure 5, and the data address that caused the page fault, "0000001d" from Figure 1, then you would expect the EDI register referenced by the instruction to contain the value zero.  If you inspect the context record in Figure 5, you find the EDI register did indeed contain zero when the exception occurred.  A simpler check would be to compare the third parameter to KeBugCheckEx()--as previously mentioned, this is the address of the faulting instruction--to the instruction pointer, EIP in Figure 5.  These should match as well.

Cousins of KMODE_EXCEPTION_NOT_HANDLED

If you're developing or maintaining a file system filter driver, you will be interested to know that most of the techniques presented above are equally valid for the related bugcodes FAT_FILE_SYSTEM, CDFS_FILE_SYSTEM, and NTFS _FILE_SYSTEM.  In the case of these bugcodes, the exception filter functions FatExceptionFilter(), Cd ExceptionFilter(), and NtfsExceptionFilter() call KeBug CheckEx() with their respective bugcode.  The knowledge base doesn't tell us the meaning of the parameters to KeBugCheckEx() for these bugcodes.  Fortunately, we can look in the IFS Kit to determine that the second parameter to these functions is an EXCEPTION_POINTERS structure.

If you inspect the EXCEPTION_POINTERS structure and the exception record from a FAT_FILE_SYSTEM, CDFS_ FILE_SYSTEM, or NTFS_FILE_SYSTEM crash, you will discover the meaning of most of the parameters passed to KeBugCheckEx() by FatExceptionFilter(), CdException Filter(), or NtfsExceptionFilter(). Or alternatively, you could look in the source code included with the IFS Kit.  We leave that as an exercise for the afflicted.

The True Meaning of KMODE_EXCEPTION_NOT_HANDLED

There are really two major categories of KMODE_ EXCEPTION_NOT_HANDLED crashes.  The class of KMODE_EXCEPTION_NOT_HANDLED errors divulged above results from exception filters like PspUnhandled ExceptionInSystemThread() that call KeBugCheckEx() with a bugcode of KMODE_EXCEPTION_NOT_ HANDLED.  An almost equally common breed of KMODE_ EXCEPTION_NOT_HANDLED is the kind that occurs when no exception handler was registered by the thread that incurred the exception.  Truly then the exception is not handled, since there is no exception handler to call.

These two types of KMODE_ EXCEPTION_NOT_ HANDLED crashes have some things in common.  The parameters to KeBugCheckEx() have the same meaning.  Thus, the information from the blue screen as displayed by WinDbg may be interpreted similarly.  However, for the second variety of KMODE_ EXCEPTION_NOT_ HANDLED crash, there is no exception filter function with an EXCEPTION_POINTERS structure as an argument conveniently on the stack.  Instead, you may see a stack trace similar to the one shown in Figure 8.

KDx86> kbvs

FramePtr  RetAddr   Param1   Param2   Param3   Function Name

fc55dce0  8014195d  fc55dcfc 00000000 fc55dd50 NT!KiDispatchException+0x35e(UKNOWN FPO TYPE)

fc55de4c  80127836  00000001 80703dac 00000004 NT!CommonDispatchException+0x4d (FPO: [0,80,0])

fc55ddd0  8010b8ac  fc55ddf0 807033d0 8081e318 NT!MiDispatchFault+0xb4(UKNOWN FPO TYPE)

b26f200b  00000000  00000000 00000000 00000000 NT!@ExReleaseHandleTableShared@4+0x9c(UKNOWN FPO TYPE)

Figure 8 -- Stack Trace From a Different Crashdump

This time the stack truly does look damaged, either because it is, or WinDbg is having some difficulty.  I am not surprised that MiDispatchFault() called CommonDispatchException() that in turn called KiDispatchException(), but I seriously doubt ExReleaseHandleTableShared() called MiDispatch Fault().  This notion is further strengthened by the fact that the stack frame pointer next to ExReleaseHandleTable Shared() could not possibly point to part of this thread's stack, since kernel threads cannot have stacks over 1 GB in size.  If the exception did not occur in ExReleaseHandleTable Shared(), then where did it happen?

Despite the fact that we don't have a pointer to an EXCEPTION_POINTERS structure, we should still be able to instruct WinDbg to display a stack trace starting with the function where the exception was incurred.  As mentioned above in the section on exception handling, the trap handler saves the thread's context information in the trap frame.  Therefore, somewhere on this stack we should find the trap frame that contains this thread's context information at the time of the exception.  This information can be used to create the stack trace we desire.  Luckily, Microsoft Knowledge Base Article "Q159672 - How To Find the Trap Frame If It Is Corrupt," contains some pointers for finding a trap frame on the thread's current stack.

Finding the Trap Frame

You can use a technique for finding the trap frame derived from Knowledge Base article Q159672.  First, dump the boundaries of the thread's stack using WinDbg's default extension DLL's "thread" command.  Then search the stack for the sequence of longwords "0x23 0x23" using WinDbg's built-in "s" command.  Figure 9 illustrates this technique.  The boundaries of the stack are the values the "thread" command labels "Base" and "Limit".  Notice that the stack limit is lower in memory than the base of the stack.  This is no cause for concern.  On an x86 processor, the stack grows from high addresses to low addresses.  However, WinDbg's "s" command searches memory from low addresses to high addresses.  For that reason, you must specify the limit of the stack as the beginning of the memory range to search and the base of the stack as the end of the range.  Otherwise, WinDbg will display an error instead of searching memory.  The search command found the "0x23 0x23" sequence at four different addresses.  Subtracting 0x34 from one of these four addresses will give you a pointer to the trap frame.  How do you determine which of these four values is the trap frame?

KDx86> !thread

Debugger extension library [kdextx86.dll] loaded

THREAD 807036e0  Cid 58.84  Teb: 7ffde000  Win32Thread: 00000000 RUNNING

IRP List:

    8081e2a8: (0006,0094) Flags: 00000000  Mdl: 00000000

Not impersonating

Owning Process 80703e00

WaitTime (seconds)      405372

Context Switch Count    15              

UserTime                  0:00:00.0000

KernelTime                0:00:00.0015

Start Address 0x77f052dc

Win32 Start Address 0x00401460

Stack Init fc55e000 Current fc55d9b0 Base fc55e000 Limit fc55b000 Call 0

Priority 12 BasePriority 8 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr  Args to Child

0012f420 00000000 00000000 00000000 00000000 +0xffffffff

KDx86> sd fc55b000 fc55e000 0x23 0x23

0xFC55DA44

0xFC55DBCC

0xFC55DD84

0xFC55DF38

Figure 9 -- Searching for the Trap Frame

You can use a brute force method to pinpoint the trap frame.  Subtract 0x34 from each of the four addresses at the bottom of Figure 9, then use each of the resulting addresses as arguments to WinDbg's default extension DLL's "trap" command.  The trap command dumps the contents of the trap frame which includes all the registers from when the exception happened.  Then look for a trap frame dump with a value for the instruction pointer register, designated EIP, that matches the third parameter to KeBugCheckEx(), the address of the faulting instruction.  That trap frame and the information from the blue screen where you obtain the third parameter to KeBugCheckEx() are displayed in Figure 10.  Now you have enough information to tell WinDbg how to obtain a stack trace from where the exception was incurred.

Kernel Debugger connection established for F:\dumps\Memory.dmp

Kernel Version 1381 Free loaded @ 0x80100000

Bugcheck 0000001e : c0000005 f75d33ef 00000001 00000000

KDx86> !trap 0xFC55DD50

eax=00000000 ebx=8016d401 ecx=80714450 edx=8081e2a8 esi=807274b0 edi=8081e2a8

eip=f75d33ef esp=fc55ddc4 ebp=fc55dde4 iopl=0         nv up ei pl zr na po nc

cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00010246

ErrCode = 00000002

0xf75d33ef  c60000           mov         byte ptr [eax],00       

Figure 10 -- Dumping the Trap Frame

Just as we did with a context record earlier, you can use the EBP, ESP, and EIP values from a trap frame to display a stack trace starting from the faulting instruction.  That stack trace appears in Figure 11.  The trace tells us what line of code to inspect in the OSRLOCK driver.  We should find the bug there.

KDx86> kbvs=fc55dde4 fc55ddc4 f75d33ef

FramePtr  RetAddr   Param1   Param2   Param3   Function Name

fc55dde4  80112863  807274b0 8081e2a8 807038ec OSRLOCK!IoctlDeviceDispatch+0xd2(UKNOWN FPO TYPE) [ osrlock.cpp @ 166 ]

fc55ddf8  801728c6  00000000 00000000 8016d401 NT!@IofCallDriver@8+0x37 (FPO: [0,0,2])

fc55de14  80173057  807274b0 8081e2a8 80703d68 NT!IopSynchronousServiceTail+0x6a(UKNOWN FPO TYPE)

fc55dea0  8016d442  00000050 00000000 00000000 NT!IopXxxControlFile+0x6c1

fc55ded4  80140be9  00000050 00000000 00000000 NT!NtDeviceIoControlFile+0x28(UKNOWN FPO TYPE)

fc55ded4  77f678ff  00000050 00000000 00000000 NT!KiSystemService+0xc9 (FPO: [0,0] TrapFrame @ fc55df04)

0012fea8  00000000  00000000 00000000 00000000 NTDLL!ZwDeviceIoControlFile+0xb (FPO: [10,0,0])

Figure 11 -- Useful Stack Trace

Summary

Often times when debugging a crash dump, the stack appears to be corrupt.  Other times, we are encountering limitations of our debugging tools.  In spite of these obstacles, the procedures presented above will more often than not produce the information we seek.  The challenge in developing these debugging methods is assimilating information from disparate locations.  For example, the information for this article was gleaned from the Platform SDK, the NT DDK, MSJ, The NT Insider, the Microsoft Knowledge Base, the book Inside Windows NT by Helen Custer, and the experience of looking at countless kernel dumps.