Jump-start your project by learning from devs who
write Windows drivers and file systems every day.
Take an OSR seminar!

Go Back   OSR Online Lists > windbg
link COM <-> named pipe debugging in vmware workstation
Welcome, Guest
You must login to post to this list
  Message 1 of 10  
19 Jul 05 15:25
windbg member 25455
xxxxxx@bellsouth.net
Join Date:
Posts To This List: 8
COM <-> named pipe debugging in vmware workstation
I successfully connect a target macine running inside VirtualPC to a host-based windbg through a named pipe. Now I try to do the same on a VMWare workstation 5.0. I redirect virtual's COM1 or COM2 to \\.\pipe\dbgpipe which is what I use for VPC but no matter what I do with other VMWare's COM port settings (client vs server, app or VM on the other end, yield on poll), host-based windbg does not see the target [VMWare virtual] machine but is happy when the target runs inside a VPC. boot.ini's are identical on both virtual machines, with records like multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="win + COM1 + ntbtlog.txt" /fastdetect /debug /debugport=com1 /baudrate=115200 /bootlog VMWare on-line help brought nothing new. My guess is that it should work, what can be possibly wrong with a named pipe? Did anyone have this problem and solved it (or did not have this VMWare-specific problem at all, for that matter)? Maybe, it's some security policy setting that I'm missing? Any hints would be greatly appreciated. Regards, Alex Shvedov
  Message 2 of 10  
19 Jul 05 17:26
Peter Scott
xxxxxx@KernelDrivers.com
Join Date: 14 Jan 2002
Posts To This List: 4
RE: COM <-> named pipe debugging in vmware workstation
I use this method for debugging all the time. To get it working, I created the virtual COM port in the virtual machine to be a named pipe, as you did. Then I opened WInDbg and specified to connect via a named pipe, the only way I managed this was by running WinDbg from a command line specifying the following: start windbg -W Pipe -b -k com:pipe,port=\\.\pipe\dbgpipe,resets=0 Note that this will also break on entry into the system, I do this to ensure connectivity. It also uses the workspace profile I created called Pipe, which contains the symbol paths etc. I have encountered an issue with VPC that requires me to restart the debugger whenever I restart the virtual machine. This is not always the case with VMWare but it does happen. Pete Kernel Drivers Windows Filesystem and Device Driver Consulting www.KernelDrivers.com (303)546-0300 -----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex@Home Sent: Tuesday, July 19, 2005 1:25 PM To: Kernel Debugging Interest List Subject: [windbg] COM <-> named pipe debugging in vmware workstation I successfully connect a target macine running inside VirtualPC to a host-based windbg through a named pipe. Now I try to do the same on a VMWare workstation 5.0. I redirect virtual's COM1 or COM2 to \\.\pipe\dbgpipe which is what I use for VPC but no matter what I do with other VMWare's COM port settings (client vs server, app or VM on the other end, yield on poll), host-based windbg does not see the target [VMWare virtual] machine but is happy when the target runs inside a VPC. boot.ini's are identical on both virtual machines, with records like multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="win + COM1 + ntbtlog.txt" /fastdetect /debug /debugport=com1 /baudrate=115200 /bootlog VMWare on-line help brought nothing new. My guess is that it should work, what can be possibly wrong with a named pipe? Did anyone have this problem and solved it (or did not have this VMWare-specific problem at all, for that matter)? Maybe, it's some security policy setting that I'm missing? Any hints would be greatly appreciated. Regards, Alex Shvedov --- You are currently subscribed to windbg as: xxxxx@kerneldrivers.com To unsubscribe send a blank email to xxxxx@lists.osr.com
  Message 3 of 10  
20 Jul 05 07:24
Spiro Trikaliotis
xxxxxx@trikaliotis.net
Join Date: 30 Oct 2003
Posts To This List: 45
Re: COM <-> named pipe debugging in vmware workstation
Hello, * On Tue, Jul 19, 2005 at 03:25:09PM -0600 Peter Scott wrote: > I have encountered an issue with VPC that requires me to restart the > debugger whenever I restart the virtual machine. This is not always > the case with VMWare but it does happen. With VMWare, I have the following issue: When I boot the virtual machine (XP), the debugger is not allowed to be started (or the VM will not boot at all). I have to wait until after the VM has started the bootloader, I have selected the debug flags, pressed enter, and the VM's screen dimensions have changed. HTH, Spiro. -- Spiro R. Trikaliotis xxxxx@trikaliotis.net Dipl.-Inform. http://www.trikaliotis.net/ OvG-Universitaet, IVS.EUK Fon: +49-391-67-12566 PF 4120, 39016 Magdeburg Fax: +49-391-67-11161
  Message 4 of 10  
20 Jul 05 10:59
windbg member 25455
xxxxxx@bellsouth.net
Join Date:
Posts To This List: 8
Re: COM <-> named pipe debugging in vmware workstation
> the only way I managed this was by running WinDbg from a > command line specifying the following: > start windbg -W Pipe -b -k com:pipe,port=\\.\pipe\dbgpipe,resets=0 Thanks for replies, Peter and Spiro, at least I know that it _may_ work as I expected. Strange, very strange... My arrangements are exactly the same for both VPC (no problems) and VMWare, I also figured out that there is no way - for now, at least - to save the info about COM port being redirected to a pipe in windbg's workspace so that we have to use prosthetics. In my case it is a dbgpipe.cmd wich contains (I split the line) "C:\Program Files\Debugging Tools for Windows\windbg.exe" -y srv*D:\SYMBOLS*http://msdl.microsoft.com/download/symbols -k com:pipe,port=\\.\pipe\dbgpipe,resets=0,reconnect (as you see, no break on boot here, it _may_ be saved in a workspace; symbols in this script are extras also). Yet again, this script works ok with VPC but not with VMWare, and I can not spot any difference whatsoever... The VMWare-based target sits for some notable time, say 30 secs, probably waiting for windbg response, the port state is reported by VMWare as "connected" (to what ???), and then the boot continues as usual, and windbg does not see anything. Mystery. And I thought that all named pipes are created equal:-) > I have encountered an issue with VPC that requires me to restart the > debugger whenever I restart the virtual machine. Never ever happened to me, I run as many reboots as I want keeping windbg running. One more mystery... > With VMWare, I have the following issue: When I boot the virtual machine > (XP), the debugger is not allowed to be started (or the VM will not boot > at all). I have to wait until after the VM has started the bootloader, I > have selected the debug flags, pressed enter, and the VM's screen > dimensions have changed. Can you tell me what are the VMWare's and target's port settings? client/server, other end is a VM/other end is an app, yield on poll? Speed seems not to matter (or does it?). Anyway, I'll try it your way, normally I start windbg way before the target, and it's ok for VPC. Regards, Alex Shvedov ----- Original Message ----- From: "Peter Scott" <xxxxx@KernelDrivers.com> To: "Kernel Debugging Interest List" <xxxxx@lists.osr.com> Sent: Tuesday, July 19, 2005 5:25 PM Subject: RE: [windbg] COM <-> named pipe debugging in vmware workstation > > I use this method for debugging all the time. To get it working, I created > the virtual COM port in the virtual machine to be a named pipe, as you > did. > Then I opened WInDbg and specified to connect via a named pipe, the only > way > I managed this was by running WinDbg from a command line specifying the > following: > <...excess quoted lines suppressed...>
  Message 5 of 10  
23 Jul 05 04:04
Spiro Trikaliotis
xxxxxx@trikaliotis.net
Join Date: 30 Oct 2003
Posts To This List: 45
Re: COM <-> named pipe debugging in vmware workstation
Hello Alex, Alex@Home <xxxxx@bellsouth.net> schrieb: > The VMWare-based target sits for some notable time, say 30 secs, > probably waiting for windbg response, the port state is reported by > VMWare as "connected" (to what ???), and then the boot continues as > usual, and windbg does not see anything. This is exactly the behaviour I had with older WinDBGs. With newer ones, the target just doesn't sit 30seconds or so, but forever. Thus, my work-around is like this: Don't start WinDBG until the target waits. Now, start WinDBG and press Ctrl+Break (this is important, the automatic break-in does NOT work!). After this, I have control over the target. Regards, Spiro. -- Spiro R. Trikaliotis http://www.trikaliotis.net/
  Message 6 of 10  
23 Jul 05 10:32
windbg member 25455
xxxxxx@bellsouth.net
Join Date:
Posts To This List: 8
Re: COM <-> named pipe debugging in vmware workstation
> Thus, my work-around is like this: Don't start WinDBG until the target > waits. Now, start WinDBG and press Ctrl+Break (this is important, the > automatic break-in does NOT work!). After this, I have control over the > target. Thanks. The Ctrl+Break adds to the idea I extracted from your previous post. My guess is that VMWare virtual starts its COM pipe as a _server_ so it waits for someone to send in a request. Stupid, as compared to VPC, and - if I'm right - directly contradicts VM's own on-line help. > This is exactly the behaviour I had with older WinDBGs. I had all this with windbg 6.4, just got the latest tools with 6.5.3.7, let's see... Regards, Alex Shvedov ----- Original Message ----- From: "Spiro Trikaliotis" <xxxxx@trikaliotis.net> To: "Kernel Debugging Interest List" <xxxxx@lists.osr.com> Sent: Saturday, July 23, 2005 4:04 AM Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation > Hello Alex, > > Alex@Home <xxxxx@bellsouth.net> schrieb: > >> The VMWare-based target sits for some notable time, say 30 secs, >> probably waiting for windbg response, the port state is reported by >> VMWare as "connected" (to what ???), and then the boot continues as >> usual, and windbg does not see anything. > <...excess quoted lines suppressed...>
  Message 7 of 10  
23 Jul 05 22:02
Peter Scott
xxxxxx@KernelDrivers.com
Join Date: 14 Jan 2002
Posts To This List: 4
RE: COM <-> named pipe debugging in vmware workstation
Just to let you know, I posted my command line Windbg, and it works every time with VMWare. Not the case with VPC. I have the initial BP set and it hits it each time on the VMWare virtual machine. OK, every 10-15 reboots of a VMWare virtual pc I need to restart Windbg, I call that pretty good. Pete Kernel Drivers Windows Filesystem and Device Driver Consulting www.KernelDrivers.com (303)546-0300 -----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex@Home Sent: Saturday, July 23, 2005 8:32 AM To: Kernel Debugging Interest List Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation > Thus, my work-around is like this: Don't start WinDBG until the target > waits. Now, start WinDBG and press Ctrl+Break (this is important, the > automatic break-in does NOT work!). After this, I have control over the > target. Thanks. The Ctrl+Break adds to the idea I extracted from your previous post. My guess is that VMWare virtual starts its COM pipe as a _server_ so it waits for someone to send in a request. Stupid, as compared to VPC, and - if I'm right - directly contradicts VM's own on-line help. > This is exactly the behaviour I had with older WinDBGs. I had all this with windbg 6.4, just got the latest tools with 6.5.3.7, let's see... Regards, Alex Shvedov ----- Original Message ----- From: "Spiro Trikaliotis" <xxxxx@trikaliotis.net> To: "Kernel Debugging Interest List" <xxxxx@lists.osr.com> Sent: Saturday, July 23, 2005 4:04 AM Subject: Re: [windbg] COM <-> named pipe debugging in vmware workstation > Hello Alex, > > Alex@Home <xxxxx@bellsouth.net> schrieb: > >> The VMWare-based target sits for some notable time, say 30 secs, >> probably waiting for windbg response, the port state is reported by >> VMWare as "connected" (to what ???), and then the boot continues as >> usual, and windbg does not see anything. > <...excess quoted lines suppressed...> --- You are currently subscribed to windbg as: xxxxx@kerneldrivers.com To unsubscribe send a blank email to xxxxx@lists.osr.com
  Message 8 of 10  
23 Jul 05 23:45
windbg member 25455
xxxxxx@bellsouth.net
Join Date:
Posts To This List: 8
Re: COM <-> named pipe debugging in vmware workstation
> Just to let you know, I posted my command line Windbg, and it works every > time with VMWare. Not the case with VPC. Mystery. I am happy with VPC 100%. As I already said, I thought that all named pipes are created equal. They are not, some named pipes are more named than others:-) > initial BP set and it > hits it each time on the VMWare virtual machine. This is a secondary problem; in my case windbg hears VPC all right in all cases, including target reboots, and does not hear VMW at all. I stop the stuff by a DbgBreakPoint in my DriverEntry (in debug/checked build, of course). Why on earth should it behave differently, I wonder? It's just a named pipe, nothing more, after all. Mystery. A pro po: starting windbg when wmv is waiting, as Spiro suggested, does help, but wmv restart is missed by windbg once again. VMWare does insists on being a server? Regards, Alex Shvedov ----- Original Message ----- From: "Peter Scott" <xxxxx@KernelDrivers.com> To: "Kernel Debugging Interest List" <xxxxx@lists.osr.com> Sent: Saturday, July 23, 2005 10:00 PM Subject: RE: [windbg] COM <-> named pipe debugging in vmware workstation > > Just to let you know, I posted my command line Windbg, and it works every > time with VMWare. Not the case with VPC. I have the initial BP set and it > hits it each time on the VMWare virtual machine. > > OK, every 10-15 reboots of a VMWare virtual pc I need to restart Windbg, I > call that pretty good. > > Pete <...excess quoted lines suppressed...>
  Message 9 of 10  
24 Jul 05 11:23
Spiro Trikaliotis
xxxxxx@trikaliotis.net
Join Date: 30 Oct 2003
Posts To This List: 45
Re: COM <-> named pipe debugging in vmware workstation
Hello Alex, * On Sat, Jul 23, 2005 at 10:32:02AM -0400 Alex@Home wrote: > My guess is that VMWare virtual starts its COM pipe as a _server_ so > it waits for someone to send in a request. > Stupid, as compared to VPC, and - if I'm right - directly contradicts > VM's own on-line help. Well, at least in my VMWare, I can tell VMWare if I want this side to be a server or an application... Look into the settings of your serial port. Anyway, I am not sure how this would change anything? * On Sat, Jul 23, 2005 at 11:45:08PM -0400 Alex@Home wrote: > I stop the stuff by a DbgBreakPoint in my DriverEntry (in > debug/checked build, of course). You mean instead of an initial breakpoint? Yes, this works for me, too. > A pro po: starting windbg when wmv is waiting, as Spiro suggested, > does help, but wmv restart is missed by windbg once again. VMWare > does insists on being a server? I don't remeber for a VMWare restart, but I remember that going back to a snapshot in VMWare, you get an error that the pipe could not be opened (as it already is open). Perhaps, this is somehow related? Regards, Spiro. -- Spiro R. Trikaliotis http://www.trikaliotis.net/
  Message 10 of 10  
24 Jul 05 21:47
Gary Little
xxxxxx@mn.rr.com
Join Date: 20 Apr 2005
Posts To This List: 60
Re: COM <-> named pipe debugging in vmware workstation
My experience is, after about 10-15 restarts, I need to re-start WinDbg, and I'm not using virtual anything. -- The personal opinion of Gary G. Little "Peter Scott" <xxxxx@KernelDrivers.com> wrote in message news:56794@windbg... > > Just to let you know, I posted my command line Windbg, and it works every > time with VMWare. Not the case with VPC. I have the initial BP set and it > hits it each time on the VMWare virtual machine. > > OK, every 10-15 reboots of a VMWare virtual pc I need to restart Windbg, I > call that pretty good. > > Pete > <...excess quoted lines suppressed...>
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the windbg list to be able to post.

All times are GMT -5. The time now is 13:45.

Contact Us - Osr Online Homepage - Top

Copyright ©2005, OSR Open Systems Resourcs, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license