I am writing a KMDF 1.9 driver in which I use a continuous reader on one of the USB endpoints. As the data comes in through the EvtUsbTargetPipeReadComplete callback, I would like to throw the WDFMEMORY handle (after taking a reference of coure) into a queue and process it on a PASSIVE_LEVEL thread so that I can continue to receive data. Is it more appropriate to submit a reusuable WDFWORKITEM whenever the data comes in, or should I instead use a dedicated PsCreateSystemThread that contintinually waits (via a semaphore) for data to become available in the queue?
I know that system worker threads are a valuable resource that should be used sparingly, so I wasn’t sure which is the more appropriate approach. The driver will typically only be receiving data when there is a user operating the device - probably a few minutes worth per hour - but there will be a lot of data coming in during those bursts. However, I also noticed that, in the WDK samples, PsCreateSystemThread is only ever used in WDM code, so I wasn’t sure whether its use in KMDF drivers is discouraged, or if there is a better approach for dealing with this situation.
Also, if a WDFWORKITEM’s EvtWorkItemFunc is alread executing, and the driver calls WdfWorkItemEnqueue(), is it guaranteed that the work item will be enqueued and its EvtWorkItemFunc called again, or does this only occur if the previous EvtWorkItemFunc has finished?
With regard to the queue, is it more appropriate to use a WDFCOLLECTION or a doubly linked list? I had figured that using the friendly WDFCOLLECTION would be the preferred approach for a KMDF driver, so I was surprised to see that the doubly linked list came so highly recommended in this post: http://www.osronline.com/showThread.cfm?link=222275. What are the tradeoffs between the two?
All good questions.
In general, most engineers will agree that if you have just a FEW requests that need to be performed “out of line and at IRQL PASIVE_LEVEL” you should use a work item. Beyond that, the question of whether you create your won thread pool or use the system worker thread pool (that is, work item) becomes one more of art than science, to be perfectly honest.
At OSR, we generally lean toward creating our own thread pool if a driver has regular, on-going, or long-running requirements for use of a thread. We’ve come to this conclusion somewhat reluctantly over the years after dealing with weird deadlocks and strange edge-conditions caused by using the system worker threads. Plus, when you “roll your own” you get the advantage of being able to create exactly the priority and concurrency conditions that YOU want for your work threads. Sometimes this matters, sometimes not. But it can play a role in making the decision.
There’s no limit on the number of system threads that can be created, so it’s not *really* correct to say “they are a valuable resource that should be used sparingly.” If you need one, just PsCreateSystemThread and go for it.
I’m also a big fan of WDFCOLLECTION. It has the advantage of retaining a reference to the item while it’s queued, also. I wouldn’t take a native WDF item and queue it with a LIST_ENTRY, unless there was a very specific gain to be had by doing so. The other discussion you cite was… well… about other stuff (that guy started with using named pipes, for goodness sakes) and I suspect few of us in that thread were aware he made reference to WDF in one of his posts… I know I wasn’t.
Peter
OSR
>over the years after dealing with weird deadlocks and strange edge-conditions caused by using the
The general rule is that you should not use IoQueueWorkItem below Cc - i.e. the noncached paths of the FSD or the block storage stack.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
“The general rule is that you should not use IoQueueWorkItem below Cc - i.e. the
noncached paths of the FSD or the block storage stack.”
Unfortunately, nobody told that to MPIO folks.
Another hazard of using system workitem threads is that you should not rely on your workitems to complete for transition to S3/S4/S5. And that may be VERY difficult or impossible to comply with. Typically, workitem routines are allowed to touch pageable code/data, which may require the paging stack to be on. But during the system transition to low power, the paging stack may become to shut down earlier.
This statement is correct if your device has not set the DO_POWER_PAGABLE bit. If your device is power pagable, as long as you run down the work items during power down, you will be fine.
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@broadcom.com
Sent: Friday, April 06, 2012 9:40 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] WDF work item or system thread? WDF collection or doubly linked list?
Another hazard of using system workitem threads is that you should not rely on your workitems to complete for transition to S3/S4/S5. And that may be VERY difficult or impossible to comply with. Typically, workitem routines are allowed to touch pageable code/data, which may require the paging stack to be on. But during the system transition to low power, the paging stack may become to shut down earlier.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Thanks for your responses. I guess I should steer away from WDFWORKITEMs, then. It seems like their behavior is a bit difficult to understand. Also, in my driver, I suppose the callback could potentially be long-running if there is a lot of data streaming in.
Nonetheless, I haven’t quite given up hope on WDFWORKITEMs. I am still interested to know whether a work item’s callback will be executed again if the work item is re-enqueued while there is currently an instance of its callback already executing?
What do the msdn docs tell you?
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@cspeed.com
Sent: Friday, April 06, 2012 10:51 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] WDF work item or system thread? WDF collection or doubly linked list?
Thanks for your responses. I guess I should steer away from WDFWORKITEMs, then. It seems like their behavior is a bit difficult to understand. Also, in my driver, I suppose the callback could potentially be long-running if there is a lot of data streaming in.
Nonetheless, I haven’t quite given up hope on WDFWORKITEMs. I am still interested to know whether a work item’s callback will be executed again if the work item is re-enqueued while there is currently an instance of its callback already executing?
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
It says:
“For versions 1.7 and later of KMDF, if your driver reuses its work-item objects, the driver can call WdfWorkItemEnqueue again for the same work item before a system worker thread has dequeued the work item and called the driver?s EvtWorkItem callback function. However, KMDF will not add the work item to the queue if it is already there. Therefore, your EvtWorkItem callback function must process all queued work each time that it is called.”
So, if the EvtWorkItem is currently running, I presume this implies the work item has already been dequeued. That is the part I’m unsure of – the docs say “A system worker thread subsequently removes the work item from the queue and calls the work item’s EvtWorkItem callback function”, but I’m not sure whether “and” suggests order. Anyway, assuming the work item has already been dequeued by the time the first EvtWorkItem runs, then enqueuing it again means that the EvtWorkItem should be called again, even if the previous callback from that work item is still running?
Correct, if the work-item is still queued, calling WdfWorkItemEnqueue will not enqueue another one. Thus your work-item callback must be able to process all queued work.
Correct also for the second case, if your work-item is running, you can queue another work-item. Note that the new work-item’s callback can even run before the first one completes.
Egi.
Thanks again for all the help!
I think that I’ll start off by using work items, because I would like to try to stick with the high-level abstractions where available. If I later feel like I’m abusing my work items by having them run too long, or have issues with power states, maybe I will go back and create a primitive thread that continuously waits for data in the queue and processes it accordingly when some becomes available.
(And using a WDF collection seems like the clear answer to my data queue question.)
WRT to collection vs LIST_ENTRY, the collection is a nice abstraction BUT it allocates memory on insertion, which means it can fail and introduce another point of failure. You can use a LIST_ENTRY without allocating more memory by specifying a context on the WDFMEMORY when you create the continuous reader. That context can contain the LIST_ENTRY. So while a little bit less of a nice abstraction, it is slightly less error prone as well
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@cspeed.com
Sent: Friday, April 06, 2012 1:14 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] WDF work item or system thread? WDF collection or doubly linked list?
Thanks again for all the help!
I think that I’ll start off by using work items, because I would like to try to stick with the high-level abstractions where available. If I later feel like I’m abusing my work items by having them run too long, or have issues with power states, maybe I will go back and create a primitive thread that continuously waits for data in the queue and processes it accordingly when some becomes available.
(And using a WDF collection seems like the clear answer to my data queue question.)
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer