How to configure the Vmware + Windbg

Hi all:

I have a very strange problem when I am using Vmware and windbg to debug the windows drivers in a single host, the process of my configuration is :

  1. add a new serial hardware and select the “use named pipe” radio box, use the default name “\.\pipe\com_1”, select “This End is server”, “other end is application” and “Yield CPU on poll” checkbox in advanced setting,

  2. edit the boot.ini, add /debugport=com1 /baudrate=115200

  3. change the parameters of com1 to 115200 in device manager

4.reboot the vmware, and select the debug mode

  1. execute the windbg in cmd mode : windbg -k com:port=\.\pipe\com_1,pipe or windbg -b -k com:pipe,port=\.\pipe\com_1,resets=0 for windbg 6.X, some guys say that use the “reconnect” instead of “resets=0” here will be ok, but it still has the same problem.

  2. add the _NT_SYMBOL_PATH into environment variable, set the value of which

srv*c:\Symbols*http://msdl.microsoft.com/download/symbols

  1. switch to the vmware, select debug mode (as step 4), and click “enter” button

  2. Vmware and windbg are all response, the windbg stops at “waiting to reconnect”, and the

Vmware keeps black screen

I try to change the name of serialport to “com2”, then the Vmware is ok, but the windbg is still with problem as above…

I see this a lot (but not every time). I don’t know what causes it, but
if you go in to windbg and do Debug | Kernel Connection | Resynchronize,
or it’s keyboard equivalent of CTRL+ALT+R, I have found that the
debugger will connect (after a short delay) and the VMware boot will
proceed.

Cheers,

–mkj

xxxxx@gmail.com wrote:

Hi all:

I have a very strange problem when I am using Vmware and windbg to debug the windows drivers in a single host, the process of my configuration is :

  1. add a new serial hardware and select the “use named pipe” radio box, use the default name “\.\pipe\com_1”, select “This End is server”, “other end is application” and “Yield CPU on poll” checkbox in advanced setting,

  2. edit the boot.ini, add /debugport=com1 /baudrate=115200

  3. change the parameters of com1 to 115200 in device manager

4.reboot the vmware, and select the debug mode

  1. execute the windbg in cmd mode : windbg -k com:port=\.\pipe\com_1,pipe or windbg -b -k com:pipe,port=\.\pipe\com_1,resets=0 for windbg 6.X, some guys say that use the “reconnect” instead of “resets=0” here will be ok, but it still has the same problem.

  2. add the _NT_SYMBOL_PATH into environment variable, set the value of which

srv*c:\Symbols*http://msdl.microsoft.com/download/symbols

  1. switch to the vmware, select debug mode (as step 4), and click “enter” button

  2. Vmware and windbg are all response, the windbg stops at “waiting to reconnect”, and the

Vmware keeps black screen

I try to change the name of serialport to “com2”, then the Vmware is ok, but the windbg is still with problem as above…


//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

You’re having some kind of serial communiction error. If you do
Ctrl+Alt+d in windbg you’ll get more info about what it’s doing. If you
only see timeout messages then windbg isn’t receiving data from the VM.
If you see resend or error messages the connection is not working
reliably.

From what I’ve heard you don’t want to use resets=0 nor reconnect when
trying to connect to a VMWare VM.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Monday, September 25, 2006 4:01 AM
To: Kernel Debugging Interest List
Subject: [windbg] How to configure the Vmware + Windbg

Hi all:

I have a very strange problem when I am using Vmware and windbg to debug
the windows drivers in a single host, the process of my configuration is
:

  1. add a new serial hardware and select the “use named pipe” radio box,
    use the default name “\.\pipe\com_1”, select “This End is server”,
    “other end is application” and “Yield CPU on poll” checkbox in advanced
    setting,

  2. edit the boot.ini, add /debugport=com1 /baudrate=115200

  3. change the parameters of com1 to 115200 in device manager

4.reboot the vmware, and select the debug mode

  1. execute the windbg in cmd mode : windbg -k
    com:port=\.\pipe\com_1,pipe or windbg -b -k
    com:pipe,port=\.\pipe\com_1,resets=0 for windbg 6.X, some guys say that
    use the “reconnect” instead of “resets=0” here will be ok, but it still
    has the same problem.

  2. add the _NT_SYMBOL_PATH into environment variable, set the value of
    which

srv*c:\Symbols*http://msdl.microsoft.com/download/symbols

  1. switch to the vmware, select debug mode (as step 4), and click
    “enter” button

  2. Vmware and windbg are all response, the windbg stops at “waiting to
    reconnect”, and the

Vmware keeps black screen

I try to change the name of serialport to “com2”, then the Vmware is ok,
but the windbg is still with problem as above…


You are currently subscribed to windbg as: xxxxx@winse.microsoft.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

I have found that with the latest version I sometimes have to exit windbg
and when I restart it, it will just work.

“Michael Jones” wrote in message
news:xxxxx@windbg…
>I see this a lot (but not every time). I don’t know what causes it, but if
>you go in to windbg and do Debug | Kernel Connection | Resynchronize, or
>it’s keyboard equivalent of CTRL+ALT+R, I have found that the debugger will
>connect (after a short delay) and the VMware boot will proceed.
>
> Cheers,
>
> --mkj
>
> xxxxx@gmail.com wrote:
>> Hi all:
>>
>> I have a very strange problem when I am using Vmware and windbg to debug
>> the windows drivers in a single host, the process of my configuration is
>> :
>>
>> 1. add a new serial hardware and select the “use named pipe” radio box,
>> use the default name “\.\pipe\com_1”, select “This End is server”,
>> “other end is application” and “Yield CPU on poll” checkbox in advanced
>> setting, 2. edit the boot.ini, add /debugport=com1 /baudrate=115200
>>
>> 3. change the parameters of com1 to 115200 in device manager
>>
>> 4.reboot the vmware, and select the debug mode
>>
>> 5. execute the windbg in cmd mode : windbg -k
>> com:port=\.\pipe\com_1,pipe or windbg -b -k
>> com:pipe,port=\.\pipe\com_1,resets=0 for windbg 6.X, some guys say that
>> use the “reconnect” instead of “resets=0” here will be ok, but it still
>> has the same problem.
>>
>> 6. add the _NT_SYMBOL_PATH into environment variable, set the value of
>> which srvc:\Symbolshttp://msdl.microsoft.com/download/symbols
>>
>> 7. switch to the vmware, select debug mode (as step 4), and click “enter”
>> button
>>
>> 8. Vmware and windbg are all response, the windbg stops at “waiting to
>> reconnect”, and the Vmware keeps black screen
>>
>> I try to change the name of serialport to “com2”, then the Vmware is ok,
>> but the windbg is still with problem as above…
>
> –
>
> //
> // Michael K. Jones
> // Stone Hill Consulting, LLC
> // http://www.stonehill.com
> //

>

Hi Bliss & Jones:
thanks for your replies. I do CTRL+ALT+D, and it displays as follow:

SYNCTARGET: Received KD_RESET ACK packet.
SYNCTARGET: Target synchronized successfully…
Done.
WRITE: Write type 2 packet
READ: Wait for ACK packet
PacketType=7, ByteCount=10f, PacketId=80800000,
READ: Received VALID data packet while waiting for ACK.
READ: Received Type 7 data packet with id = 80800000 successfully.

READ: Data packet instead of ack, stored.
READ: Wait for type 2 packet
READ: Grab packet from buffer.
READ: Unexpected Packet type 7 (Acked). Expecting Packet type 2
PacketType=5, ByteCount=0, PacketId=5f863,
READ: Received RESEND packet
READ: Target requests resend.
Resync packet id …Kd sync initial break: on

repeat this process, at last it reports the information:

Unable to read head of debugger data list
Connected to Windows XP 2600 x86 compatible target, ptr64 FALSE
Kernel Debugger connection established.
Symbol search path is: c:\symbols;srv*c:\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:

what’s wrong with it?

The RESEND means that your serial communication is not working reliably.
In cases with emulated serial hardware this is often because the
emulation doesn’t work quite like real serial hardware. resets=0 (or
leaving off resets=0) are a couple of tweaks that can help initial sync
in this case. I would recommend not specifying resets=0 if you are. If
you continue to have trouble you’d probably need to contact VMWare.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Monday, September 25, 2006 9:57 PM
To: Kernel Debugging Interest List
Subject: RE:[windbg] How to configure the Vmware + Windbg

Hi Bliss & Jones:
thanks for your replies. I do CTRL+ALT+D, and it displays as
follow:

SYNCTARGET: Received KD_RESET ACK packet.
SYNCTARGET: Target synchronized successfully…
Done.
WRITE: Write type 2 packet
READ: Wait for ACK packet
PacketType=7, ByteCount=10f, PacketId=80800000,
READ: Received VALID data packet while waiting for ACK.
READ: Received Type 7 data packet with id = 80800000 successfully.

READ: Data packet instead of ack, stored.
READ: Wait for type 2 packet
READ: Grab packet from buffer.
READ: Unexpected Packet type 7 (Acked). Expecting Packet type 2
PacketType=5, ByteCount=0, PacketId=5f863,
READ: Received RESEND packet
READ: Target requests resend.
Resync packet id …Kd sync initial break: on

repeat this process, at last it reports the information:

Unable to read head of debugger data list Connected to Windows XP 2600
x86 compatible target, ptr64 FALSE Kernel Debugger connection
established.
Symbol search path is:
c:\symbols;srv*c:\Symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:

what’s wrong with it?


You are currently subscribed to windbg as: xxxxx@winse.microsoft.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

“David J. Craig” wrote in message news:xxxxx@windbg…
>I have found that with the latest version I sometimes have to exit windbg and when I restart it, it will just work.

Then perhaps you have the same issue like I’ve seen with previous windbg
and Vmware workstation 5.5.
The reason is that VMware side of the pipe must be server.
When windbg starts first, and the pipe is not created yet,
then, for some unfathomable reason, it becomes the server, and
Vmware fails to attach to it’s end of the pipe.
Not VMware neither MS assume responsibility for this “feature”.

The workaround is, as you’ve found, not to start windbg until the VMware side has opened the pipe. And when the VM restarts,
you have to restart windbg as well.

–PA

> “Michael Jones” wrote in message news:xxxxx@windbg…
>>I see this a lot (but not every time). I don’t know what causes it, but if you go in to windbg and do Debug | Kernel
>>Connection | Resynchronize, or it’s keyboard equivalent of CTRL+ALT+R, I have found that the debugger will connect (after a
>>short delay) and the VMware boot will proceed.
>>
>> Cheers,
>>
>> --mkj
>>
>> xxxxx@gmail.com wrote:
>>> Hi all:
>>>
>>> I have a very strange problem when I am using Vmware and windbg to debug the windows drivers in a single host, the process
>>> of my configuration is :
>>>
>>> 1. add a new serial hardware and select the “use named pipe” radio box, use the default name “\.\pipe\com_1”, select “This
>>> End is server”, “other end is application” and “Yield CPU on poll” checkbox in advanced setting, 2. edit the boot.ini, add
>>> /debugport=com1 /baudrate=115200
>>>
>>> 3. change the parameters of com1 to 115200 in device manager
>>>
>>> 4.reboot the vmware, and select the debug mode
>>>
>>> 5. execute the windbg in cmd mode : windbg -k com:port=\.\pipe\com_1,pipe or windbg -b -k
>>> com:pipe,port=\.\pipe\com_1,resets=0 for windbg 6.X, some guys say that use the “reconnect” instead of “resets=0” here will
>>> be ok, but it still has the same problem.
>>>
>>> 6. add the _NT_SYMBOL_PATH into environment variable, set the value of which
>>> srvc:\Symbolshttp://msdl.microsoft.com/download/symbols
>>>
>>> 7. switch to the vmware, select debug mode (as step 4), and click “enter” button
>>>
>>> 8. Vmware and windbg are all response, the windbg stops at “waiting to reconnect”, and the Vmware keeps black screen
>>>
>>> I try to change the name of serialport to “com2”, then the Vmware is ok, but the windbg is still with problem as above…
>>
>> –
>>
>> //
>> // Michael K. Jones
>> // Stone Hill Consulting, LLC
>> // http://www.stonehill.com
>> //

>>
>
>
>

… you might want to edit boot.ini and add timeout=-1. It will wait
forever at your VMWare boot options, so you have all the time to start
WinDbg. This and other tips for VMware WinDbg were already mentioned on
this list time ago.
WBR Primoz

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: 25. september 2006 13:01
To: Kernel Debugging Interest List
Subject: [windbg] How to configure the Vmware + Windbg

Hi all:

I have a very strange problem when I am using Vmware and windbg to debug
the windows drivers in a single host, the process of my configuration is
:

  1. add a new serial hardware and select the “use named pipe” radio box,
    use the default name “\.\pipe\com_1”, select “This End is server”,
    “other end is application” and “Yield CPU on poll” checkbox in advanced
    setting,

  2. edit the boot.ini, add /debugport=com1 /baudrate=115200

  3. change the parameters of com1 to 115200 in device manager

4.reboot the vmware, and select the debug mode

  1. execute the windbg in cmd mode : windbg -k
    com:port=\.\pipe\com_1,pipe or windbg -b -k
    com:pipe,port=\.\pipe\com_1,resets=0 for windbg 6.X, some guys say that
    use the “reconnect” instead of “resets=0” here will be ok, but it still
    has the same problem.

  2. add the _NT_SYMBOL_PATH into environment variable, set the value of
    which

srv*c:\Symbols*http://msdl.microsoft.com/download/symbols

  1. switch to the vmware, select debug mode (as step 4), and click
    “enter” button

  2. Vmware and windbg are all response, the windbg stops at “waiting to
    reconnect”, and the

Vmware keeps black screen

I try to change the name of serialport to “com2”, then the Vmware is ok,
but the windbg is still with problem as above…


You are currently subscribed to windbg as: xxxxx@hermes.si
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi All:
Most grateful for your help, I set timeout= -1 , and it’s successful. :slight_smile: