Hi again, I run standard wndbg command from scripts on user and kernel dumps collected from within my network. Recently I have noticed user and kernel mode symbol errors (since about mid July 2015) - causing the scripts to fail.
I routinely run .reload /f to update the symbol files with each monthly windows update patch when debugging. No errors when reloading symbols are shown affecting the relevant modules below.
Specifically - symbol errors on
User dumps - !peb and !teb commands fail (ntdll PEB and TEB wrong symbols)
Kernel - !process command fails (nt KPRCB and KPCR wrong symbols)
(a) have I messed something up in my debug environment?
(b) is windbg matching the dump image and the symbol file correctly?
(c) is there an error with windows symbol files in recent updates?
Before July2015 I had no issues. Now I am unable to run the same scripts.
Now I get (user e.g) =
[Your debugger is not using the correct symbols
Type referenced: nt!_TEB
error InitTypeRead( TEB )…]
Using win7 x64 windbg, setup empty symbol folder, testing my problem using full user dump of explorer.exe on local x64 PC. Refreshed with new symbols using .reload /f
Does not display any datatypes for ntdll, but can see the function list.
0:000> !peb
PEB at 000007fffffd5000
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: ntdll!_PEB ***
*** ***
*************************************************************************
error 3 InitTypeRead( nt!_PEB at 000007fffffd5000)…
Could you show us the way you setup your symbol path?
Also, what is the output of the following commands:
!sym noisy
.reload /f nt
!peb
If the symbols are still not loaded properly you should get errors/warnings
when executing the “.reload /f nt” instruction.
On 16 August 2015 at 02:01, wrote:
> Using win7 x64 windbg, setup empty symbol folder, testing my problem using > full user dump of explorer.exe on local x64 PC. Refreshed with new symbols > using .reload /f > > Does not display any datatypes for ntdll, but can see the function list. > > 0:000> !peb > PEB at 000007fffffd5000 > > > > Your debugger is not using the correct symbols > > In order for this command to work properly, your symbol path > must point to .pdb files that have full type information. > > Certain .pdb files (such as the public OS symbols) do not > contain the required information. Contact the group that > provided you with these symbols if you need this command to > work. > > Type referenced: ntdll!_PEB > > > error 3 InitTypeRead( nt!_PEB at 000007fffffd5000)… > > Not sure where the problem lies. > > — > WINDBG is sponsored by OSR > > OSR is hiring!! Info at http://www.osr.com/careers > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
Hi Alexandru, thank you for replying.
Windbg symbol path: SRV*D:\symbols*symsrv*http://msdl.microsoft.com/download/symbols
(set same as environmental variable _NT_SYMBOL_PATH)
Has worked fine up until recently (noticed change in early Aug) - no abnormal errors loading symbols online or offline.
User mode debugging of iexplore.exe - results below
$ ----------------------------------------------------------------------------------------------------
0:000> !sym noisy
noisy mode - symbol prompts on
0:000> .reload /f nt
“nt” was not found in the image list.
Debugger will attempt to load “nt” at given base 00000000.
Please provide the full image name, including the extension (i.e. kernel32.dll)
for more reliable results.Base address and size overrides can be given as
.reload <image.ext>=,. DBGENG: nt - Partial symbol image load missing image info DBGHELP: No header for nt. Searching for dbg file DBGHELP: *http://msdl.microsoft.com/download/symbols\nt.dbg - The filename, directory name, or volume label syntax is incorrect. DBGHELP: .\nt.dbg - file not found DBGHELP: nt missing debug info. Searching for pdb anyway DBGHELP: Can’t use symbol server for nt.pdb - no header information available DBGHELP: *http://msdl.microsoft.com/download/symbols\nt.pdb - file not found DBGHELP: *http://msdl.microsoft.com/download/symbols\exe\nt.pdb - file not found DBGHELP: *http://msdl.microsoft.com/download/symbols\symbols\exe\nt.pdb - file not found DBGHELP: nt.pdb - file not found DBGHELP: nt - no symbols loaded Unable to add module at 00000000 0:000> !peb PEB at 7efde000 Your debugger is not using the correct symbols In order for this command to work properly, your symbol path must point to .pdb files that have full type information. Certain .pdb files (such as the public OS symbols) do not contain the required information. Contact the group that provided you with these symbols if you need this command to work. Type referenced: ntdll!_PEB
error 3 InitTypeRead( nt!_PEB at 7efde000)…
$ ---------------------------------------------------------------------------------------------------- 0:000> !lmi ntdll Loaded Module Info: [ntdll] Module: ntdll Base Address: 779f0000 Image Name: ntdll.dll Machine Type: 332 (I386) Time Stamp: 55a69e20 Thu Jul 16 03:53:36 2015 Size: 180000 CheckSum: 142a8b Characteristics: 2102 perf Debug Data Dirs: Type Size VA Pointer CODEVIEW 23, e63e0, d67e0 RSDS - GUID: {FA9C48F9-C11D-4E08-94B8-970DECD92C97} Age: 2, Pdb: wntdll.pdb CLSID 4, e63dc, d67dc [Data not mapped] Image Type: MEMORY - Image read successfully from loaded memory. Symbol Type: PDB - Symbols loaded successfully from symbol server. d:\symbols\wntdll.pdb\FA9C48F9C11D4E0894B8970DECD92C972\wntdll.pdb Load Report: public symbols , not source indexed d:\symbols\wntdll.pdb\FA9C48F9C11D4E0894B8970DECD92C972\wntdll.pdb
0:000> .reload /f ntdll
“ntdll” was not found in the image list. Debugger will attempt to load “ntdll” at given base 00000000.
Please provide the full image name, including the extension (i.e. kernel32.dll) for more reliable results.Base address and size overrides can be given as .reload <image.ext>=,. DBGENG: ntdll - Partial symbol image load missing image info DBGHELP: No header for ntdll. Searching for dbg file DBGHELP: *http://msdl.microsoft.com/download/symbols\ntdll.dbg - The filename, directory name, or volume label syntax is incorrect. DBGHELP: .\ntdll.dbg - file not found DBGHELP: ntdll missing debug info. Searching for pdb anyway DBGHELP: Can’t use symbol server for ntdll.pdb - no header information available DBGHELP: *http://msdl.microsoft.com/download/symbols\ntdll.pdb - file not found DBGHELP: *http://msdl.microsoft.com/download/symbols\exe\ntdll.pdb - file not found DBGHELP: *http://msdl.microsoft.com/download/symbols\symbols\exe\ntdll.pdb - file not found DBGHELP: ntdll.pdb - file not found DBGHELP: ntdll_0 - no symbols loaded Unable to add module at 00000000
Same result on !peb
Kernal mode, using livekd - similar result in !process command fails. Tried !sym noisy as per above - did not resolve it. Can see the function lists, but does not list any data types for ntdll (user) or ntkrnlmp (kernel)
Can list modules. Any clues?</image.ext></image.ext>
Sorry, I only now noticed we are talking about a user mode dump.
If you try “.reload /f ntdll.dll” it still does not load the correct
symbols?
On 17 August 2015 at 13:37, wrote:
> Noticed a “w” prefix on some symbol files > e.g. ntdll symbol file is wntdll.pdb > (symbol folder has ntdll and wntdll subfolders) > > also similar to > > DBGHELP: rpcrt4 - public symbols > > d:\symbols\wrpcrt4.pdb\3A41A875C5EA474889C0CA79BB8CE1322\wrpcrt4.pdb > . > DBGHELP: sspicli - public symbols > > d:\symbols\wsspicli.pdb\A08801BB042F4EB4A8162D816398B8A11\wsspicli.pdb > . > > Compared to a dump file in July, the symbol files match the image names > without the w prefixed. > > > — > WINDBG is sponsored by OSR > > OSR is hiring!! Info at http://www.osr.com/careers > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
Thanks Alex.
It looks like only lately windbg is not loading the correct symbols, or recently symbols files no longer contain data types.
We are talking about both situations. To summarise:
kernel mode - can’t list processes, !process command fails. I can’t set the process context to execute !peb. The ntkrnlmp symbols no longer lists any data structures in the default pdb.
I have the same problem. According to the dia2dump sample app, neither
of the pdb’s for the current ntkrnlmp (x64; time stamp 55a6901f) and
ntdll (x64; time stamp 55a6a196) contain type information:
C:\>dia2dump -t
C:\Symbols\ntkrnlmp.pdb\6ECA9A4801E74C44A2021EA387E737FA2\ntkrnlmp.pdb
*** TYPES
** User Defined Types
** ENUMS
** TYPEDEFS
C:\>dia2dump -t
C:\Symbols\ntdll.pdb\1EA2E1024B9149A883257AD45C8E45CB2\ntdll.pdb
*** TYPES
** User Defined Types
** ENUMS
** TYPEDEFS
I have encountered this also. I have a user dump from a client running windows 7 that I can’t analyze. The symbols on the Microsoft server match but don’t contain the necessary information. I checked my own server 2008 R2 machine and user dumps from that can’t be analyzed either. It brought back memories of 64 bit server 2003 dump analysis problems that Ivan Brugiolo helped me with years ago. It appears that a recent windows update has changed several modules and the pdb files on the symbol server lack content. If a Microsoft person could investigate it would be appreciated.
example error followed by itoldyouso and module information:
*************************************************************************
*** ***
*** ***
*** Either you specified an unqualified symbol, or your debugger ***
*** doesn’t have full symbol information. Unqualified symbol ***
*** resolution is turned off by default. Please either specify a ***
*** fully qualified symbol module!symbolname, or enable resolution ***
*** of unqualified symbols by typing “.symopt- 100”. Note that ***
*** enabling unqualified symbol resolution with network symbol ***
*** server shares in the symbol path may cause the debugger to ***
*** appear to hang for long periods of time when an incorrect ***
*** symbol name is typed or the network symbol server is down. ***
*** ***
*** For some commands to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: ntdll!_PEB ***
*** ***
*************************************************************************
1)If the test machine is changed then you should load symbols from the
internet. i.e fist time you should connect to the internet and load the
symbols.
To load the symbols:
1)!symfix
2)then set symbol path : srv*;symbolpath
2)Then set the source path.
3)Sym noisy
4).reload
5).sympath to check the symbol path.
I can confirm the same for the Windows 7 64 bit kernel update , MS stripped all data structures information killing all WinDBG functionality that used it, e.g. !devstack, !drvobj etc. The bizarre thing is that the kernel was built a month ago and MS still hasn’t fixed the issue.
Loaded symbol image file: ntkrnlmp.exe
Image path: ntkrnlmp.exe
Image name: ntkrnlmp.exe
Browse all global symbols functions data
Timestamp: Thu Jul 16 02:53:51 2015 (55A6901F)
CheckSum: 00556CB2
ImageSize: 005EB000
File version: 6.1.7601.18933
Product version: 6.1.7601.18933
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 1.0 App
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft? Windows? Operating System
InternalName: ntkrnlmp.exe
OriginalFilename: ntkrnlmp.exe
ProductVersion: 6.1.7601.18933
FileVersion: 6.1.7601.18933 (win7sp1_gdr.150715-0600)
FileDescription: NT Kernel & System
LegalCopyright: ? Microsoft Corporation. All rights reserved.
On Sun, Aug 23, 2015 at 8:17 AM, wrote: > The bizarre thing is that the kernel was built a month ago and MS still hasn’t fixed the issue.
My theory is Microsoft is so busy with the Windows 10 release that they put an intern in charge of the Windows 7 builds. He’s all alone in a basement, clueless, and everyone is ignoring his cries for help. We’re lucky anything works at all.
I am going to hazard a guess and say the msft network of support and distribution is undergoing some changes. Symbols config works OK, but the symbol files themselves have changed for Windows 7.
Sorry, I haven’t been following this thread, so this has probably already
been mentioned, but sometimes in the past when the symbol server had issues,
correct symbols could be found in the symbols download section.
I don’t even know if that section still exists, but I thought I’d throw it
out there.