Skipping IRP stack locations and driver verifier problems

Aydan_Yumerefendi · September 29, 2008, 2:38pm

I have a TDI filter driver, which does not need to process all IRPs that it receives. For the IRPs that I do not want to filter I use the following code in my dispatch function:

IoSkipCurrentIrpStackLocation(pIrp);
status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
return status;

If the driver verifier is disabled, this works just fine. Enabling the driver verifier and I/O verification, causes an error:

WDM DRIVER ERROR: This IRP is about to run out of stack locations

Note that the error occurs on Server 2003 (both x86 and x64). It does not happen on XP x32.

This warning happens only for IRPs that have insufficient stack location (pIrp->CurrentLocation=1). Here is the callstack before calling the filtered driver

mydriver!MyDeviceExtensionDispatchInternal+0x10b
mydriver!MyDispatchInternal+0x73
nt!IovCallDriver+0x1b5
netbt!NbtDpcPostIrpToProvider+0x7e
nt!KiRetireDpcList+0x150
nt!KiDispatchInterrupt+0x4f

At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1

After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.

I have several questions:

Is this the right way to pass IRPs that I do not want to have a completion routine for down to the lower driver?
Is the driver verifier correct is pointing a problem or this is a false positive? Is the verifier confused by the use of IoSkipCurrentIrpStackLocation and if so, is there a way to get by this problem?

Thank you in advance,

–aydan

David_R_Cattley · September 29, 2008, 4:50pm

I saw similar issues with a TDI filter and eventually concluded that it was
a verifier ‘failure’ to correctly measure what is going on.

The issue is that somehow the stack got an IRP without out enough stack
locations for each driver in the stack. Since your driver is the first one
with verifier enabled (who knows, maybe it is the top of the stack, but no
matter) it is the first chance verifier has to complain about it.

TDI filtering in particular is so full of bad actors, hacks, and other
ugliness, verifier can hardly be expected to guess if an IRP might actually
succeed. However, I agree with your observation that in this case, verifier
does have enough information to know that the particular IRP *does* have
enough remaining stack locations to traverse the remainder of the device
stack.

I found that using I/O verification in a TDI (filter) stack was very much a
hassle. Let’s face it, a TDI filter has to be explicitly aware of IRP stack
exhaustion and the ‘rules’ get seriously bent to make it all work period.
Given that landscape, it is better for you to have direct instrumentation
(ASSERT()s, etc.) in your code to catch these errors instead of using
verifier to do it for you. Let’s face it. Your driver *does* have all the
information to know when to bugcheck!

Good luck,
Dave Cattley
Consulting Engineer
Systems Software Devlopment

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Monday, September 29, 2008 2:38 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Skipping IRP stack locations and driver verifier problems

I have a TDI filter driver, which does not need to process all IRPs that it
receives. For the IRPs that I do not want to filter I use the following code
in my dispatch function:

IoSkipCurrentIrpStackLocation(pIrp);
status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
return status;

If the driver verifier is disabled, this works just fine. Enabling the
driver verifier and I/O verification, causes an error:

WDM DRIVER ERROR: This IRP is about to run out of stack locations

Note that the error occurs on Server 2003 (both x86 and x64). It does not
happen on XP x32.

This warning happens only for IRPs that have insufficient stack location
(pIrp->CurrentLocation=1). Here is the callstack before calling the filtered
driver

mydriver!MyDeviceExtensionDispatchInternal+0x10b
mydriver!MyDispatchInternal+0x73
nt!IovCallDriver+0x1b5
netbt!NbtDpcPostIrpToProvider+0x7e
nt!KiRetireDpcList+0x150
nt!KiDispatchInterrupt+0x4f

At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1

After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as
expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.

I have several questions:

Is this the right way to pass IRPs that I do not want to have a
completion routine for down to the lower driver?
Is the driver verifier correct is pointing a problem or this is a false
positive? Is the verifier confused by the use of
IoSkipCurrentIrpStackLocation and if so, is there a way to get by this
problem?

Thank you in advance,

–aydan

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Aydan_Yumerefendi · October 1, 2008, 3:30pm

Dave,

Thank you for the reply. I guess we will disable IO verification and add asserts to our code instead.

Thanks,
–aydan

Bill_McKenzie-3 · October 1, 2008, 3:34pm

It sure would be nice if we had a constant, consistent and attentive avenue
for reporting this kind of stuff. As it is, I wouldn’t even try…I find
error reporting to Microsoft an exercise in futility. Not the end of the
world, but it would be nice.

Bill M.

“David R. Cattley” wrote in message news:xxxxx@ntdev…
>I saw similar issues with a TDI filter and eventually concluded that it was
> a verifier ‘failure’ to correctly measure what is going on.
>
> The issue is that somehow the stack got an IRP without out enough stack
> locations for each driver in the stack. Since your driver is the first
> one
> with verifier enabled (who knows, maybe it is the top of the stack, but no
> matter) it is the first chance verifier has to complain about it.
>
> TDI filtering in particular is so full of bad actors, hacks, and other
> ugliness, verifier can hardly be expected to guess if an IRP might
> actually
> succeed. However, I agree with your observation that in this case,
> verifier
> does have enough information to know that the particular IRP does have
> enough remaining stack locations to traverse the remainder of the device
> stack.
>
> I found that using I/O verification in a TDI (filter) stack was very much
> a
> hassle. Let’s face it, a TDI filter has to be explicitly aware of IRP
> stack
> exhaustion and the ‘rules’ get seriously bent to make it all work period.
> Given that landscape, it is better for you to have direct instrumentation
> (ASSERT()s, etc.) in your code to catch these errors instead of using
> verifier to do it for you. Let’s face it. Your driver does have all
> the
> information to know when to bugcheck!
>
> Good luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Devlopment
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> xxxxx@gmail.com
> Sent: Monday, September 29, 2008 2:38 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Skipping IRP stack locations and driver verifier problems
>
> I have a TDI filter driver, which does not need to process all IRPs that
> it
> receives. For the IRPs that I do not want to filter I use the following
> code
> in my dispatch function:
>
> IoSkipCurrentIrpStackLocation(pIrp);
> status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
> return status;
>
> If the driver verifier is disabled, this works just fine. Enabling the
> driver verifier and I/O verification, causes an error:
>
> WDM DRIVER ERROR: This IRP is about to run out of stack locations
>
> Note that the error occurs on Server 2003 (both x86 and x64). It does not
> happen on XP x32.
>
> This warning happens only for IRPs that have insufficient stack location
> (pIrp->CurrentLocation=1). Here is the callstack before calling the
> filtered
> driver
>
> mydriver!MyDeviceExtensionDispatchInternal+0x10b
> mydriver!MyDispatchInternal+0x73
> nt!IovCallDriver+0x1b5
> netbt!NbtDpcPostIrpToProvider+0x7e
> nt!KiRetireDpcList+0x150
> nt!KiDispatchInterrupt+0x4f
>
> At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1
>
> After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as
> expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.
>
> I have several questions:
> - Is this the right way to pass IRPs that I do not want to have a
> completion routine for down to the lower driver?
> - Is the driver verifier correct is pointing a problem or this is a false
> positive? Is the verifier confused by the use of
> IoSkipCurrentIrpStackLocation and if so, is there a way to get by this
> problem?
>
> Thank you in advance,
>
> --aydan
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>

Ken_Johnson · October 1, 2008, 3:59pm

Agree. I had a complex deadlock with smart card removal on an integrated (but internally USB attached) smart card reader that manifested itself across a suspend/resume in Vista x64 RTM that crossed user mode (about 4 different service processes) and kernel mode (PnP) that ultimately was caused by MS code calling LoadLibrary in DllMain (totally no-brainer stupid bug), subsequently stalling things in a PnP service notification via a concoluted chain of RPC calls.

I spent about 12 hours of my own time in kd on my own personal box triaging the issue, then I tried to report it to get fixed using one of my (at the time) MVP support incidents.

This was a total flop, even through the MVP channel (which, I assume, likely has a much higher chance of getting somewhere than your average small business PSS incident). Despite having debugged the whole damned problem, and having shown that it resulted from a bad practice that MSDN explicitly called out, I could not, for the life of me, get PSS to do anything more than acknowledge that the DllMain thing might be a bug and might get entered in a bug database for a future Windows major release.

(Of course, they could not follow up if this ever did actually happen.)

Fast forward 12 months, and a QFE hotfix is released that fixes the exact problem, (presumably because a large enough customer complained). Ugh. After PSS explicitly told me that there was no way that I could hope for anything other than a bug maybe being created for a future Windows release.

That sort of thing totally burns me up about reporting issues to Microsoft externally; unless you either 1) know the owner of the component and can get ahold of them directly, or 2) are a megacorp, you get the complete run-around and things don’t get fixed, at least not until someone sufficiently important complains.

If you know someone on the inside on the team that owns the broken component, then things are great, but the barrier of getting anything to said team from the outside world is just incredibly, ridiculously painful if you don’t have someone’s direct email to report stuff via unofficial channels.

S

-----Original Message-----
From: Bill McKenzie
Sent: Wednesday, October 01, 2008 14:37
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Skipping IRP stack locations and driver verifier problems

It sure would be nice if we had a constant, consistent and attentive avenue
for reporting this kind of stuff. As it is, I wouldn’t even try…I find
error reporting to Microsoft an exercise in futility. Not the end of the
world, but it would be nice.

Bill M.

“David R. Cattley” wrote in message news:xxxxx@ntdev…
>I saw similar issues with a TDI filter and eventually concluded that it was
> a verifier ‘failure’ to correctly measure what is going on.
>
> The issue is that somehow the stack got an IRP without out enough stack
> locations for each driver in the stack. Since your driver is the first
> one
> with verifier enabled (who knows, maybe it is the top of the stack, but no
> matter) it is the first chance verifier has to complain about it.
>
> TDI filtering in particular is so full of bad actors, hacks, and other
> ugliness, verifier can hardly be expected to guess if an IRP might
> actually
> succeed. However, I agree with your observation that in this case,
> verifier
> does have enough information to know that the particular IRP does have
> enough remaining stack locations to traverse the remainder of the device
> stack.
>
> I found that using I/O verification in a TDI (filter) stack was very much
> a
> hassle. Let’s face it, a TDI filter has to be explicitly aware of IRP
> stack
> exhaustion and the ‘rules’ get seriously bent to make it all work period.
> Given that landscape, it is better for you to have direct instrumentation
> (ASSERT()s, etc.) in your code to catch these errors instead of using
> verifier to do it for you. Let’s face it. Your driver does have all
> the
> information to know when to bugcheck!
>
> Good luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Devlopment
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> xxxxx@gmail.com
> Sent: Monday, September 29, 2008 2:38 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Skipping IRP stack locations and driver verifier problems
>
> I have a TDI filter driver, which does not need to process all IRPs that
> it
> receives. For the IRPs that I do not want to filter I use the following
> code
> in my dispatch function:
>
> IoSkipCurrentIrpStackLocation(pIrp);
> status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
> return status;
>
> If the driver verifier is disabled, this works just fine. Enabling the
> driver verifier and I/O verification, causes an error:
>
> WDM DRIVER ERROR: This IRP is about to run out of stack locations
>
> Note that the error occurs on Server 2003 (both x86 and x64). It does not
> happen on XP x32.
>
> This warning happens only for IRPs that have insufficient stack location
> (pIrp->CurrentLocation=1). Here is the callstack before calling the
> filtered
> driver
>
> mydriver!MyDeviceExtensionDispatchInternal+0x10b
> mydriver!MyDispatchInternal+0x73
> nt!IovCallDriver+0x1b5
> netbt!NbtDpcPostIrpToProvider+0x7e
> nt!KiRetireDpcList+0x150
> nt!KiDispatchInterrupt+0x4f
>
> At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1
>
> After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as
> expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.
>
> I have several questions:
> - Is this the right way to pass IRPs that I do not want to have a
> completion routine for down to the lower driver?
> - Is the driver verifier correct is pointing a problem or this is a false
> positive? Is the verifier confused by the use of
> IoSkipCurrentIrpStackLocation and if so, is there a way to get by this
> problem?
>
> Thank you in advance,
>
> --aydan
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Michal_Vodicka-2 · October 1, 2008, 4:07pm

> -----Original Message-----

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bill McKenzie
Sent: Wednesday, October 01, 2008 9:36 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Skipping IRP stack locations and driver
verifier problems

It sure would be nice if we had a constant, consistent and
attentive avenue
for reporting this kind of stuff. As it is, I wouldn’t even
try…I find
error reporting to Microsoft an exercise in futility. Not
the end of the
world, but it would be nice.

It isn’t sooo bad. In the past and this year I was able to obtain
several hotfixes for our USB problems. Well, it took months (because
there is so many bugs in Vista :). For WDK docs feedback I receive “will
be fixed” replies within few days. It isn’t completely futile but could
be definitely better.

It’d be nice if for development tools is there something similar as WDK
docs feedback. Maybe another link on the utility description page in
docs?

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

Ken_Johnson · October 1, 2008, 4:08pm

Actually, it may have been an RPC call out of DllMain and not just LoadLibrary (in case anyone here actually looked at this bug), but it was egregriously bad in any case.

S

-----Original Message-----
From: Skywing
Sent: Wednesday, October 01, 2008 15:01
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] Skipping IRP stack locations and driver verifier problems

Agree. I had a complex deadlock with smart card removal on an integrated (but internally USB attached) smart card reader that manifested itself across a suspend/resume in Vista x64 RTM that crossed user mode (about 4 different service processes) and kernel mode (PnP) that ultimately was caused by MS code calling LoadLibrary in DllMain (totally no-brainer stupid bug), subsequently stalling things in a PnP service notification via a concoluted chain of RPC calls.

I spent about 12 hours of my own time in kd on my own personal box triaging the issue, then I tried to report it to get fixed using one of my (at the time) MVP support incidents.

This was a total flop, even through the MVP channel (which, I assume, likely has a much higher chance of getting somewhere than your average small business PSS incident). Despite having debugged the whole damned problem, and having shown that it resulted from a bad practice that MSDN explicitly called out, I could not, for the life of me, get PSS to do anything more than acknowledge that the DllMain thing might be a bug and might get entered in a bug database for a future Windows major release.

(Of course, they could not follow up if this ever did actually happen.)

Fast forward 12 months, and a QFE hotfix is released that fixes the exact problem, (presumably because a large enough customer complained). Ugh. After PSS explicitly told me that there was no way that I could hope for anything other than a bug maybe being created for a future Windows release.

That sort of thing totally burns me up about reporting issues to Microsoft externally; unless you either 1) know the owner of the component and can get ahold of them directly, or 2) are a megacorp, you get the complete run-around and things don’t get fixed, at least not until someone sufficiently important complains.

If you know someone on the inside on the team that owns the broken component, then things are great, but the barrier of getting anything to said team from the outside world is just incredibly, ridiculously painful if you don’t have someone’s direct email to report stuff via unofficial channels.

- S

-----Original Message-----
From: Bill McKenzie
Sent: Wednesday, October 01, 2008 14:37
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Skipping IRP stack locations and driver verifier problems

It sure would be nice if we had a constant, consistent and attentive avenue
for reporting this kind of stuff. As it is, I wouldn’t even try…I find
error reporting to Microsoft an exercise in futility. Not the end of the
world, but it would be nice.

Bill M.

“David R. Cattley” wrote in message news:xxxxx@ntdev…
>I saw similar issues with a TDI filter and eventually concluded that it was
> a verifier ‘failure’ to correctly measure what is going on.
>
> The issue is that somehow the stack got an IRP without out enough stack
> locations for each driver in the stack. Since your driver is the first
> one
> with verifier enabled (who knows, maybe it is the top of the stack, but no
> matter) it is the first chance verifier has to complain about it.
>
> TDI filtering in particular is so full of bad actors, hacks, and other
> ugliness, verifier can hardly be expected to guess if an IRP might
> actually
> succeed. However, I agree with your observation that in this case,
> verifier
> does have enough information to know that the particular IRP does have
> enough remaining stack locations to traverse the remainder of the device
> stack.
>
> I found that using I/O verification in a TDI (filter) stack was very much
> a
> hassle. Let’s face it, a TDI filter has to be explicitly aware of IRP
> stack
> exhaustion and the ‘rules’ get seriously bent to make it all work period.
> Given that landscape, it is better for you to have direct instrumentation
> (ASSERT()s, etc.) in your code to catch these errors instead of using
> verifier to do it for you. Let’s face it. Your driver does have all
> the
> information to know when to bugcheck!
>
> Good luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Devlopment
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> xxxxx@gmail.com
> Sent: Monday, September 29, 2008 2:38 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Skipping IRP stack locations and driver verifier problems
>
> I have a TDI filter driver, which does not need to process all IRPs that
> it
> receives. For the IRPs that I do not want to filter I use the following
> code
> in my dispatch function:
>
> IoSkipCurrentIrpStackLocation(pIrp);
> status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
> return status;
>
> If the driver verifier is disabled, this works just fine. Enabling the
> driver verifier and I/O verification, causes an error:
>
> WDM DRIVER ERROR: This IRP is about to run out of stack locations
>
> Note that the error occurs on Server 2003 (both x86 and x64). It does not
> happen on XP x32.
>
> This warning happens only for IRPs that have insufficient stack location
> (pIrp->CurrentLocation=1). Here is the callstack before calling the
> filtered
> driver
>
> mydriver!MyDeviceExtensionDispatchInternal+0x10b
> mydriver!MyDispatchInternal+0x73
> nt!IovCallDriver+0x1b5
> netbt!NbtDpcPostIrpToProvider+0x7e
> nt!KiRetireDpcList+0x150
> nt!KiDispatchInterrupt+0x4f
>
> At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1
>
> After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as
> expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.
>
> I have several questions:
> - Is this the right way to pass IRPs that I do not want to have a
> completion routine for down to the lower driver?
> - Is the driver verifier correct is pointing a problem or this is a false
> positive? Is the verifier confused by the use of
> IoSkipCurrentIrpStackLocation and if so, is there a way to get by this
> problem?
>
> Thank you in advance,
>
> --aydan
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Michal_Vodicka-2 · October 1, 2008, 4:14pm

> -----Original Message-----

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Skywing
Sent: Wednesday, October 01, 2008 9:59 PM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] Skipping IRP stack locations and
driver verifier problems

That sort of thing totally burns me up about reporting issues
to Microsoft externally; unless you either 1) know the owner
of the component and can get ahold of them directly, or 2)
are a megacorp, you get the complete run-around and things
don’t get fixed, at least not until someone sufficiently
important complains.

megacorps are your customers. When a support incident is successfully
(bug admitted) finished, I fill impact info which influences the
decision when the problem will be fixed (hotfix, next SP, next OS
version). Well, I report only BSODs and when I’m sure about OS bug. For
our it usually takes months but some our customers can have QFE within a
few days.

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

Calvin_Guan-2 · October 1, 2008, 5:22pm

> 3) megacorps are your customers.

Depends how many millions pieces of hardware you sell per quarter. I worked for companies sell a lot and some other sell very few.

For the former, I wouldn’t have to worry about os bugs as long as I have convinced both msft and customers. It will get fixed sooner or later depending on how bad the problem is and how many millions of units are being affected. Tier one oems can do very good job here but they will always kick little guys butt first before they go to microsoft.

For the latter, yes I feel the pain.

Calvin

Ask a question on any topic and get answers from real people. Go to Yahoo! Answers and share what you know at http://ca.answers.yahoo.com

Michal_Vodicka-2 · October 1, 2008, 5:41pm

> -----Original Message-----

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Calvin Guan
Sent: Wednesday, October 01, 2008 11:22 PM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] Skipping IRP stack locations and
driver verifier problems

For the former, I wouldn’t have to worry about os bugs as
long as I have convinced both msft and customers.

The second can be a problem. Some customers tend to say “the problem
doesn’t occur without your device so it is you who have to solve it”.
Even when MS confirms it, some aren’t willing to open the case to speed
things up. Yes, we don’t sell many millions of units monthly.

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

Mark_S_Edwards · October 1, 2008, 6:30pm

I agree.

I do see part of Microsoft’s point. Any open access point to report
bugs, if not managed to a high degree soon gets the signal drowned
out by the noise. I would guess that many of us on here have at
times had relationships with Microsoft groups or PM’s where simply by
reputation of only ever raising something because you have conclusive
proof either of the bug or the conditions that cause the bug have
allowed you to get things fixed and changed.

Then there’s that annoying moment when you find the personnel have
been moved around and you get ignored because you have no reputation
with the new people.

I have for many years proposed that Microsoft could setup a system
which could be lightly policed. Trusted outsiders (e.g. MVP’s as
well as other classifications) get added to a portal where they can
post bugs. The noise on this portal would be limited and anyone
generating noise simply gets removed.

There was one annoying incident at the PDC when talking to someone
from Microsoft about a bug. The response being that since it hadn’t
been reported by PSS, it wasn’t on their radar. I don’t know about
anyone else but I have philosophical objections about paying
Microsoft $$$ in order to get them to acknowledge let alone fix a bug.

These days I’m pretty much with Bill, I can usually find a work
around or just document it. Life’s too short to obsess on such things.

Mark.

At 12:35 01/10/2008, Bill McKenzie wrote:

It sure would be nice if we had a constant, consistent and attentive avenue
for reporting this kind of stuff. As it is, I wouldn’t even try…I find
error reporting to Microsoft an exercise in futility. Not the end of the
world, but it would be nice.

Bill M.

“David R. Cattley” wrote in message news:xxxxx@ntdev…
> >I saw similar issues with a TDI filter and eventually concluded that it was
> > a verifier ‘failure’ to correctly measure what is going on.
> >
> > The issue is that somehow the stack got an IRP without out enough stack
> > locations for each driver in the stack. Since your driver is the first
> > one
> > with verifier enabled (who knows, maybe it is the top of the stack, but no
> > matter) it is the first chance verifier has to complain about it.
> >
> > TDI filtering in particular is so full of bad actors, hacks, and other
> > ugliness, verifier can hardly be expected to guess if an IRP might
> > actually
> > succeed. However, I agree with your observation that in this case,
> > verifier
> > does have enough information to know that the particular IRP does have
> > enough remaining stack locations to traverse the remainder of the device
> > stack.
> >
> > I found that using I/O verification in a TDI (filter) stack was very much
> > a
> > hassle. Let’s face it, a TDI filter has to be explicitly aware of IRP
> > stack
> > exhaustion and the ‘rules’ get seriously bent to make it all work period.
> > Given that landscape, it is better for you to have direct instrumentation
> > (ASSERT()s, etc.) in your code to catch these errors instead of using
> > verifier to do it for you. Let’s face it. Your driver does have all
> > the
> > information to know when to bugcheck!
> >
> > Good luck,
> > Dave Cattley
> > Consulting Engineer
> > Systems Software Devlopment
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of
> > xxxxx@gmail.com
> > Sent: Monday, September 29, 2008 2:38 PM
> > To: Windows System Software Devs Interest List
> > Subject: [ntdev] Skipping IRP stack locations and driver verifier problems
> >
> > I have a TDI filter driver, which does not need to process all IRPs that
> > it
> > receives. For the IRPs that I do not want to filter I use the following
> > code
> > in my dispatch function:
> >
> > IoSkipCurrentIrpStackLocation(pIrp);
> > status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
> > return status;
> >
> > If the driver verifier is disabled, this works just fine. Enabling the
> > driver verifier and I/O verification, causes an error:
> >
> > WDM DRIVER ERROR: This IRP is about to run out of stack locations
> >
> > Note that the error occurs on Server 2003 (both x86 and x64). It does not
> > happen on XP x32.
> >
> > This warning happens only for IRPs that have insufficient stack location
> > (pIrp->CurrentLocation=1). Here is the callstack before calling the
> > filtered
> > driver
> >
> > mydriver!MyDeviceExtensionDispatchInternal+0x10b
> > mydriver!MyDispatchInternal+0x73
> > nt!IovCallDriver+0x1b5
> > netbt!NbtDpcPostIrpToProvider+0x7e
> > nt!KiRetireDpcList+0x150
> > nt!KiDispatchInterrupt+0x4f
> >
> > At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1
> >
> > After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as
> > expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.
> >
> > I have several questions:
> > - Is this the right way to pass IRPs that I do not want to have a
> > completion routine for down to the lower driver?
> > - Is the driver verifier correct is pointing a problem or this is a false
> > positive? Is the verifier confused by the use of
> > IoSkipCurrentIrpStackLocation and if so, is there a way to get by this
> > problem?
> >
> > Thank you in advance,
> >
> > --aydan
> >
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> > http://www.osronline.com/page.cfm?name=ListServer
> >
> >
>
>
>
>—
>NTDEV is sponsored by OSR
>
>For our schedule of WDF, WDM, debugging and other seminars visit:
>http://www.osr.com/seminars
>
>To unsubscribe, visit the List Server section of OSR Online at
>http://www.osronline.com/page.cfm?name=ListServer

Calvin_Guan-2 · October 1, 2008, 6:35pm

Well, if that was the case, I’d try to bandage it with duct tape given that the problem is well understood. In many cases, a workaround is possible but it can be ugly. In the end, pleasing the customers is important or they don’t buy our chips. The drawback is that they might think a “solution” is available hence, won’t push for a real fix as hard.

Calvin

— On Wed, 10/1/08, Michal Vodicka wrote:

> From: Michal Vodicka
> Subject: RE: Re:[ntdev] Skipping IRP stack locations and driver verifier problems
> To: “Windows System Software Devs Interest List”
> Received: Wednesday, October 1, 2008, 2:41 PM
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf
> Of Calvin Guan
> > Sent: Wednesday, October 01, 2008 11:22 PM
> > To: Windows System Software Devs Interest List
> > Subject: RE: Re:[ntdev] Skipping IRP stack locations
> and
> > driver verifier problems
> >
> > For the former, I wouldn’t have to worry about os
> bugs as
> > long as I have convinced both msft and customers.
>
> The second can be a problem. Some customers tend to say
> “the problem
> doesn’t occur without your device so it is you who have
> to solve it”.
> Even when MS confirms it, some aren’t willing to open
> the case to speed
> things up. Yes, we don’t sell many millions of units
> monthly.
>
> Best regards,
>
> Michal Vodicka
> UPEK, Inc.
> [xxxxx@upek.com, http://www.upek.com]
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars
> visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online
> at http://www.osronline.com/page.cfm?name=ListServer

__________________________________________________________________
Reclaim your name @ymail.com or @rocketmail.com. Get your new email address now! Go to http://ca.promos.yahoo.com/jacko/

Michal_Vodicka-2 · October 1, 2008, 7:01pm

> -----Original Message-----

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Calvin Guan
Sent: Thursday, October 02, 2008 12:35 AM
To: Windows System Software Devs Interest List
Subject: RE: Re:[ntdev] Skipping IRP stack locations and
driver verifier problems

Well, if that was the case, I’d try to bandage it with duct
tape given that the problem is well understood. In many
cases, a workaround is possible but it can be ugly. In the
end, pleasing the customers is important or they don’t buy
our chips. The drawback is that they might think a “solution”
is available hence, won’t push for a real fix as hard.

Yes, this is what I do sometimes. In parallel with support request
waiting in MS priority queue. The ugly part is driver checking installed
USB stack version and enabling/disabling workarounds accordingly

Future driver will be UMDF which has the advantage than most workarounds
are simply impossible and there is no discussion about BSOD ‘owner’

Best regards,

Michal Vodicka
UPEK, Inc.
[xxxxx@upek.com, http://www.upek.com]

David_R_Cattley · October 1, 2008, 7:27pm

Keep in mind that verifier is flagging what it thinks is a potential problem
from the perspective that it does not *know* that everytime you process this
particular IRP that you *always* will skip & forward. The big question not
answered is why you got a short IRP to begin with. Was it from a component
that did not allocate the IRP correctly or are you getting into the stack
after the client has created its IRP pool? Just be sure that you can
explain the scenario and handle it robustly and I recommend that you add
runtime checking of your own to help replace the I/O Verification that
Verifier was trying to do.

Good Luck,
Dave Cattley
Consulting Engineer
Systems Software Development

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Wednesday, October 01, 2008 3:30 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Skipping IRP stack locations and driver verifier
problems

Dave,

Thank you for the reply. I guess we will disable IO verification and add
asserts to our code instead.

Thanks,
–aydan

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Bill_McKenzie-3 · October 1, 2008, 9:37pm

Take your experience and multiply it by about 10 or 12 and that is exactly
what I went through with 1394 back when I was an active MVP. Except that
some of the bugs I hit never got fixed AFAIK. Some got fixed because big
important customers ran into them just like you saw. Glad I am not alone
anyway

Bill M.

“Skywing” wrote in message
news:xxxxx@ntdev…

Agree. I had a complex deadlock with smart card removal on an integrated
(but internally USB attached) smart card reader that manifested itself
across a suspend/resume in Vista x64 RTM that crossed user mode (about 4
different service processes) and kernel mode (PnP) that ultimately was
caused by MS code calling LoadLibrary in DllMain (totally no-brainer stupid
bug), subsequently stalling things in a PnP service notification via a
concoluted chain of RPC calls.

I spent about 12 hours of my own time in kd on my own personal box triaging
the issue, then I tried to report it to get fixed using one of my (at the
time) MVP support incidents.

This was a total flop, even through the MVP channel (which, I assume, likely
has a much higher chance of getting somewhere than your average small
business PSS incident). Despite having debugged the whole damned problem,
and having shown that it resulted from a bad practice that MSDN explicitly
called out, I could not, for the life of me, get PSS to do anything more
than acknowledge that the DllMain thing might be a bug and might get entered
in a bug database for a future Windows major release.

(Of course, they could not follow up if this ever did actually happen.)

Fast forward 12 months, and a QFE hotfix is released that fixes the exact
problem, (presumably because a large enough customer complained). Ugh.
After PSS explicitly told me that there was no way that I could hope for
anything other than a bug maybe being created for a future Windows
release.

That sort of thing totally burns me up about reporting issues to Microsoft
externally; unless you either 1) know the owner of the component and can get
ahold of them directly, or 2) are a megacorp, you get the complete
run-around and things don’t get fixed, at least not until someone
sufficiently important complains.

If you know someone on the inside on the team that owns the broken
component, then things are great, but the barrier of getting anything to
said team from the outside world is just incredibly, ridiculously painful if
you don’t have someone’s direct email to report stuff via unofficial
channels.

- S

-----Original Message-----
From: Bill McKenzie
Sent: Wednesday, October 01, 2008 14:37
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Skipping IRP stack locations and driver verifier
problems

It sure would be nice if we had a constant, consistent and attentive avenue
for reporting this kind of stuff. As it is, I wouldn’t even try…I find
error reporting to Microsoft an exercise in futility. Not the end of the
world, but it would be nice.

Bill M.

“David R. Cattley” wrote in message news:xxxxx@ntdev…
>I saw similar issues with a TDI filter and eventually concluded that it was
> a verifier ‘failure’ to correctly measure what is going on.
>
> The issue is that somehow the stack got an IRP without out enough stack
> locations for each driver in the stack. Since your driver is the first
> one
> with verifier enabled (who knows, maybe it is the top of the stack, but no
> matter) it is the first chance verifier has to complain about it.
>
> TDI filtering in particular is so full of bad actors, hacks, and other
> ugliness, verifier can hardly be expected to guess if an IRP might
> actually
> succeed. However, I agree with your observation that in this case,
> verifier
> does have enough information to know that the particular IRP does have
> enough remaining stack locations to traverse the remainder of the device
> stack.
>
> I found that using I/O verification in a TDI (filter) stack was very much
> a
> hassle. Let’s face it, a TDI filter has to be explicitly aware of IRP
> stack
> exhaustion and the ‘rules’ get seriously bent to make it all work period.
> Given that landscape, it is better for you to have direct instrumentation
> (ASSERT()s, etc.) in your code to catch these errors instead of using
> verifier to do it for you. Let’s face it. Your driver does have all
> the
> information to know when to bugcheck!
>
> Good luck,
> Dave Cattley
> Consulting Engineer
> Systems Software Devlopment
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of
> xxxxx@gmail.com
> Sent: Monday, September 29, 2008 2:38 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Skipping IRP stack locations and driver verifier problems
>
> I have a TDI filter driver, which does not need to process all IRPs that
> it
> receives. For the IRPs that I do not want to filter I use the following
> code
> in my dispatch function:
>
> IoSkipCurrentIrpStackLocation(pIrp);
> status = IoCallDriver(pDeviceExtension->pLowerDeviceObject, pIrp);
> return status;
>
> If the driver verifier is disabled, this works just fine. Enabling the
> driver verifier and I/O verification, causes an error:
>
> WDM DRIVER ERROR: This IRP is about to run out of stack locations
>
> Note that the error occurs on Server 2003 (both x86 and x64). It does not
> happen on XP x32.
>
> This warning happens only for IRPs that have insufficient stack location
> (pIrp->CurrentLocation=1). Here is the callstack before calling the
> filtered
> driver
>
> mydriver!MyDeviceExtensionDispatchInternal+0x10b
> mydriver!MyDispatchInternal+0x73
> nt!IovCallDriver+0x1b5
> netbt!NbtDpcPostIrpToProvider+0x7e
> nt!KiRetireDpcList+0x150
> nt!KiDispatchInterrupt+0x4f
>
> At that time: pIrp->StackCount = 2 and pIrp->CurrentLocation = 1
>
> After calling IoSkipCurrentIrpStackLocation, pIrpCurrentLocation is 2 (as
> expected), and pIrp->Tail.Overlay.CurrentStackLocation has changed.
>
> I have several questions:
> - Is this the right way to pass IRPs that I do not want to have a
> completion routine for down to the lower driver?
> - Is the driver verifier correct is pointing a problem or this is a false
> positive? Is the verifier confused by the use of
> IoSkipCurrentIrpStackLocation and if so, is there a way to get by this
> problem?
>
> Thank you in advance,
>
> --aydan
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Aydan_Yumerefendi · October 1, 2008, 9:58pm

I get a “short” IRP because at the time my filter driver is installed other drivers (e.g., netbt) have been loaded and they do not update their estimate for the depth of the IRP stack until the machine is rebooted. In the case of netbt every time I access a network share all IRPs come “short”. On reboot everything works fine, but I cannot force our users to reboot their machines after installation. In fact, running without a reboot has been a major marketing goal and that has caused me a lot of headaches.

What I tried to determine with this post was if other people have observed problems with the driver verifier and short IRPs. Based on the responses I would assume that the answer is: yes.

Thanks,
–aydan

Pavel_A1 · October 1, 2008, 10:40pm

Calvin Guan wrote:

Well, if that was the case, I’d try to bandage it with duct tape given that the problem is well understood. In many cases, a workaround is possible but it can be ugly. In the end, pleasing the customers is important or they don’t buy our chips. The drawback is that they might think a “solution” is available hence, won’t push for a real fix as hard.

The poor driver engineer is always guilty for everything.
You find a workaround for buggy hardware, and they blame you for being
too slow fixing bugs in the driver.
You find a workaround for OS bug - same story. You’re in the middle…

Regards,
–PA

anton_bassov · October 2, 2008, 5:37am

> kernel mode (PnP) that ultimately was caused by MS code calling LoadLibrary in

DllMain (totally no-brainer stupid bug)

“Flames” seem to be all over the place…and I am not even participating in this thread…

Anton Bassov

Lyndon_J_Clarke-2 · October 3, 2008, 8:30pm

No time to lose, anton, no time to lose

wrote in message news:xxxxx@ntdev…
>> kernel mode (PnP) that ultimately was caused by MS code calling
>> LoadLibrary in
>> DllMain (totally no-brainer stupid bug)
>
> “Flames” seem to be all over the place…and I am not even
> participating in this thread…
>
>
> Anton Bassov
>

Lukas_Rypacek · October 8, 2008, 5:35pm

Hi, I don’t know if the original topic is already lost in rant, but I have also seen this netbt bug and I assume everyone who tried to deploy a TDI filter must have already dealt with it. It can be avoided by issuing a newly allocated IRP with enough stack locations instead of the short one, or skipping the location, if that is the option.

As I understand it, skipping the current stack location is OK, however, when driver verifier is enabled, is uses the stack location your driver would otherwise use (but didn’t) for its own completion routine. In effect, even if your driver would call IoSkipCurrentIrpStackLocation( ), that stack location would be used for driver verifier. From this point of view it is not a false positive, since the IRP is truly too short to run with Verifier

An experienced driver developer from Microsoft has suggested to me that we should check for the Verifier presence and only call IoSkipCurrentIrpStackLocation( ) if Verifier is not loaded.

Lukas R.