Win 7 NTDLL public symbols missing?

WINDBG
1

Seems like this shouldn’t be happening …

1: kd> .sympath
Symbol search path is: srv*D:\DebugSymbols*http://msdl.microsoft.com/download/symbols
Expanded Symbol search path is: srv*d:\debugsymbols*http://msdl.microsoft.com/download/symbols
1: kd> !sym noisy
noisy mode - symbol prompts on
1: kd> .reload /f /user

[snip]

SYMSRV: d:\debugsymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dll not found
DBGHELP: D:\Program Files\Debugging Tools for Windows (x64)\ntdll.dll - file not found
SYMSRV: D:\DebugSymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dll not found
DBGENG: F:\Windows\SYSTEM32\ntdll.dll - Couldn’t map image from disk.
DBGHELP: No debug info for ntdll.dll. Searching for dbg file
SYMSRV: d:\debugsymbols\ntdll.dbg\4BA9B8021ab000\ntdll.dbg not found
SYMSRV: http://msdl.microsoft.com/download/symbols/ntdll.dbg/4BA9B8021ab000/ntdll.dbg not found
DBGHELP: .\ntdll.dbg - file not found
DBGHELP: .\dll\ntdll.dbg - path not found
DBGHELP: .\symbols\dll\ntdll.dbg - path not found
DBGHELP: ntdll.dll missing debug info. Searching for pdb anyway
DBGHELP: Can’t use symbol server for ntdll.pdb - no header information available
DBGHELP: ntdll.pdb - file not found
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
DBGHELP: ntdll - export symbols

[snip]

.reload /f /user /v

[snip]

AddImage: F:\Windows\SYSTEM32\ntdll.dll
DllBase = 00000000`76d90000
Size = 001ab000
Checksum = 001b7506
TimeDateStamp = 4ba9b802
SYMSRV: d:\debugsymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dll not found
DBGHELP: D:\Program Files\Debugging Tools for Windows (x64)\ntdll.dll - file not found
SYMSRV: D:\DebugSymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV: http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dll not found
DBGENG: F:\Windows\SYSTEM32\ntdll.dll - Couldn’t map image from disk.
DBGHELP: No debug info for ntdll.dll. Searching for dbg file
SYMSRV: d:\debugsymbols\ntdll.dbg\4BA9B8021ab000\ntdll.dbg not found
SYMSRV: http://msdl.microsoft.com/download/symbols/ntdll.dbg/4BA9B8021ab000/ntdll.dbg not found
DBGHELP: .\ntdll.dbg - file not found
DBGHELP: .\dll\ntdll.dbg - path not found
DBGHELP: .\symbols\dll\ntdll.dbg - path not found
DBGHELP: ntdll.dll missing debug info. Searching for pdb anyway
DBGHELP: Can’t use symbol server for ntdll.pdb - no header information available
DBGHELP: ntdll.pdb - file not found
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
DBGHELP: ntdll - export symbols

[snip]

Phil

Philip D. Barila (303) 776-1264

2

Note that it’s looking for the image:

SYMSRV: d:\debugsymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV:
http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dll
not found

Is the PE header or debug directory paged out for the image? That can lead
to exactly this sort of thing, I was actually able to repro this, though I
do have a different version of ntdll than you do:

* In a user process context
0: kd> !process -1 0
PROCESS 88137030 SessionId: 1 Cid: 0c64 Peb: 7ffd7000 ParentCid: 0bdc
DirBase: 1ea32420 ObjectTable: 98d00c18 HandleCount: 86.
Image: GrooveMonitor.exe

* But no NTDLL symbols
0: kd> .reload /f ntdll.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
ntdll.dll -

* Check the header
0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
D53A4 [38] address [size] of Debug Directory
…
Debug Directories(2)
Type Size Address Pointer
Can’t read debug dir
…

* Try paging in the debug directory
0: kd> .pagein /p 88137030 ntdll+D53A4
You need to continue execution (press ‘g’ ) for the pagein to be
brought in. When the debugger breaks in again, the page will be present.

0: kd> g
Break instruction exception - code 80000003 (first chance)
nt!RtlpBreakWithStatusInstruction:
828a23b4 int 3

0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
Debug Directories(2)
Type Size Address Pointer
cv 22 d53e0 d47e0 Format: RSDS, guid, 2, ntdll.pdb
( 10) 4 d53dc d47dc
…

* Symbol load should work now
0: kd> .reload /f ntdll.dll
0: kd> !chksym ntdll

ntdll.dll
Timestamp: 4BA9B21E
SizeOfImage: 13C000
pdb: ntdll.pdb
pdb sig: 8F28FABF-69C0-45F1-93E6-2056E642DF5C
age: 2

Loaded pdb is
c:\websymbols\ntdll.pdb\8F28FABF69C045F193E62056E642DF5C2\ntdll.pdb

ntdll.pdb
pdb sig: 8F28FABF-69C0-45F1-93E6-2056E642DF5C
age: 2

MATCH: ntdll.pdb and ntdll.dll

If you don’t want to go through that pain, you could also copy the image
from the target machine to your host and set the image path appropriately to
see if that works.

-scott

Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

3

Wow, that rocked! I figured it was not really a missing pdb, just didn’t
think of paging being an issue.

Thanks,

Phil

Philip D. Barila??? (303) 776-1264

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Tuesday, November 23, 2010 10:26 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] Win 7 NTDLL public symbols missing?

Note that it’s looking for the image:

SYMSRV: d:\debugsymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV:
http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dl
l
not found

Is the PE header or debug directory paged out for the image? That can lead
to exactly this sort of thing, I was actually able to repro this, though I
do have a different version of ntdll than you do:

* In a user process context
0: kd> !process -1 0
PROCESS 88137030 SessionId: 1 Cid: 0c64 Peb: 7ffd7000 ParentCid: 0bdc
DirBase: 1ea32420 ObjectTable: 98d00c18 HandleCount: 86.
Image: GrooveMonitor.exe

* But no NTDLL symbols
0: kd> .reload /f ntdll.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
ntdll.dll -

* Check the header
0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
D53A4 [38] address [size] of Debug Directory
…
Debug Directories(2)
Type Size Address Pointer
Can’t read debug dir
…

[major snippage]

4

FWIW, those binaries should be on the public symbol server for purposes of supporting exactly this scenario. I’ll ask around as to getting it fixed.

  • S (Msft)

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Tuesday, November 23, 2010 9:26 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] Win 7 NTDLL public symbols missing?

Note that it’s looking for the image:

SYMSRV: d:\debugsymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV:
http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dll
not found

Is the PE header or debug directory paged out for the image? That can lead to exactly this sort of thing, I was actually able to repro this, though I do have a different version of ntdll than you do:

* In a user process context
0: kd> !process -1 0
PROCESS 88137030 SessionId: 1 Cid: 0c64 Peb: 7ffd7000 ParentCid: 0bdc
DirBase: 1ea32420 ObjectTable: 98d00c18 HandleCount: 86.
Image: GrooveMonitor.exe

* But no NTDLL symbols
0: kd> .reload /f ntdll.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -

* Check the header
0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
D53A4 [38] address [size] of Debug Directory
…
Debug Directories(2)
Type Size Address Pointer
Can’t read debug dir
…

* Try paging in the debug directory
0: kd> .pagein /p 88137030 ntdll+D53A4
You need to continue execution (press ‘g’ ) for the pagein to be brought in. When the debugger breaks in again, the page will be present.

0: kd> g
Break instruction exception - code 80000003 (first chance)
nt!RtlpBreakWithStatusInstruction:
828a23b4 int 3

0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
Debug Directories(2)
Type Size Address Pointer
cv 22 d53e0 d47e0 Format: RSDS, guid, 2, ntdll.pdb
( 10) 4 d53dc d47dc
…

* Symbol load should work now
0: kd> .reload /f ntdll.dll
0: kd> !chksym ntdll

ntdll.dll
Timestamp: 4BA9B21E
SizeOfImage: 13C000
pdb: ntdll.pdb
pdb sig: 8F28FABF-69C0-45F1-93E6-2056E642DF5C
age: 2

Loaded pdb is
c:\websymbols\ntdll.pdb\8F28FABF69C045F193E62056E642DF5C2\ntdll.pdb

ntdll.pdb
pdb sig: 8F28FABF-69C0-45F1-93E6-2056E642DF5C
age: 2

MATCH: ntdll.pdb and ntdll.dll

If you don’t want to go through that pain, you could also copy the image from the target machine to your host and set the image path appropriately to see if that works.

-scott

Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

—
WINDBG is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

5

Thanks, Ken.

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Skywing
Sent: Tuesday, November 23, 2010 2:59 PM
To: Kernel Debugging Interest List
Subject: RE: Re:[windbg] Win 7 NTDLL public symbols missing?

FWIW, those binaries should be on the public symbol server for purposes of
supporting exactly this scenario. I’ll ask around as to getting it fixed.

  • S (Msft)

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Tuesday, November 23, 2010 9:26 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] Win 7 NTDLL public symbols missing?

Note that it’s looking for the image:

SYMSRV: d:\debugsymbols\ntdll.dll\4BA9B8021ab000\ntdll.dll not found
SYMSRV:
http://msdl.microsoft.com/download/symbols/ntdll.dll/4BA9B8021ab000/ntdll.dl
l
not found

Is the PE header or debug directory paged out for the image? That can lead
to exactly this sort of thing, I was actually able to repro this, though I
do have a different version of ntdll than you do:

* In a user process context
0: kd> !process -1 0
PROCESS 88137030 SessionId: 1 Cid: 0c64 Peb: 7ffd7000 ParentCid: 0bdc
DirBase: 1ea32420 ObjectTable: 98d00c18 HandleCount: 86.
Image: GrooveMonitor.exe

* But no NTDLL symbols
0: kd> .reload /f ntdll.dll
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
ntdll.dll -

* Check the header
0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
D53A4 [38] address [size] of Debug Directory
…
Debug Directories(2)
Type Size Address Pointer
Can’t read debug dir
…

* Try paging in the debug directory
0: kd> .pagein /p 88137030 ntdll+D53A4
You need to continue execution (press ‘g’ ) for the pagein to be
brought in. When the debugger breaks in again, the page will be present.

0: kd> g
Break instruction exception - code 80000003 (first chance)
nt!RtlpBreakWithStatusInstruction:
828a23b4 int 3

0: kd> !dh ntdll

File Type: DLL
FILE HEADER VALUES
14C machine (i386)
5 number of sections
…
Debug Directories(2)
Type Size Address Pointer
cv 22 d53e0 d47e0 Format: RSDS, guid, 2, ntdll.pdb
( 10) 4 d53dc d47dc
…

* Symbol load should work now
0: kd> .reload /f ntdll.dll
0: kd> !chksym ntdll

ntdll.dll
Timestamp: 4BA9B21E
SizeOfImage: 13C000
pdb: ntdll.pdb
pdb sig: 8F28FABF-69C0-45F1-93E6-2056E642DF5C
age: 2

Loaded pdb is
c:\websymbols\ntdll.pdb\8F28FABF69C045F193E62056E642DF5C2\ntdll.pdb

ntdll.pdb
pdb sig: 8F28FABF-69C0-45F1-93E6-2056E642DF5C
age: 2

MATCH: ntdll.pdb and ntdll.dll

If you don’t want to go through that pain, you could also copy the image
from the target machine to your host and set the image path appropriately to
see if that works.

-scott

Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

—
WINDBG is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

—
WINDBG is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

6

Ken Johnson wrote:

FWIW, those binaries should be on the public symbol server for purposes of supporting
exactly this scenario. I’ll ask around as to getting it fixed. - S (Msft)

If that does get fixed, will I need to clear my local cache in order to see the fix? I’m running into this a lot, and it’s really quite tedious to fix each time.

7

>If that does get fixed, will I need to clear my local cache in order to see

the fix? I’m running into this a lot, and it’s really quite tedious to
>fix each time.

It shouldn’t, WinDBG will just grab the image from the symbol server.

Note that you can fix this yourself, just copy the ntdll.dll from the target
to your host and point your image search path to the location.

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

wrote in message news:xxxxx@windbg…

Ken Johnson wrote:

FWIW, those binaries should be on the public symbol server for purposes of
supporting
exactly this scenario. I’ll ask around as to getting it fixed. - S (Msft)

If that does get fixed, will I need to clear my local cache in order to see
the fix? I’m running into this a lot, and it’s really quite tedious to fix
each time.

8

> -----Original Message-----

From: xxxxx@lists.osr.com [mailto:bounce-434734-
xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Friday, December 17, 2010 7:18 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] Win 7 NTDLL public symbols missing?

>If that does get fixed, will I need to clear my local cache in order
to see
>the fix? I’m running into this a lot, and it’s really quite tedious
to
> >fix each time.

It shouldn’t, WinDBG will just grab the image from the symbol server.

Note that you can fix this yourself, just copy the ntdll.dll from the
target
to your host and point your image search path to the location.

That’s the strange thing. Both systems are identically patched. The 64 bit
NTDLL windbg on the host is using is identical to the 64 bit NTDLL the
target is using. I verified that the NTDLL in the target process is really
the same one by looking at the timestamp in the !dh NTDLL output, and
verified in Process Exploder that the one loaded into windbg is the same.
Since it’s in %SystemRoot%\System32, it’s on the path, and windbg uses the
path as the image path, does it not?

Thanks,

Phil

Philip D. Barila (303) 776-1264

9

>it’s on the path, and windbg uses the path as the image path, does it not?

No, there’s a separate image path environment variable used called
_NT_EXECUTABLE_IMAGE_PATH. WinDBG will also use your symbol search path if
your image path isn’t set, which is controlled by _NT_SYMBOL_PATH.

You can also set your image path in WinDBG from the Image File Path dialog
(Ctrl+i)

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

Hope to see you at the next OSR kernel debugging class February 14th in
Columbia, MD!

10

> -----Original Message-----

From: xxxxx@lists.osr.com [mailto:bounce-434747-
xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Friday, December 17, 2010 10:38 AM
To: Kernel Debugging Interest List
Subject: Re:[windbg] Win 7 NTDLL public symbols missing?

>it’s on the path, and windbg uses the path as the image path, does it
not?

No, there’s a separate image path environment variable used called
_NT_EXECUTABLE_IMAGE_PATH. WinDBG will also use your symbol search path
if
your image path isn’t set, which is controlled by _NT_SYMBOL_PATH.

You can also set your image path in WinDBG from the Image File Path
dialog
(Ctrl+i)

It doesn’t use the PATH if the Image Path isn’t set? I wonder why it’s
always worked before?

Oh well. Added an _NT_EXECUTABLE_IMAGE_PATH=%PATH% and that problem is
solved.

Thanks,

Phil

Philip D. Barila (303) 776-1264