After teaching tens of thousands of students how to write device drivers using WDM, we have a pretty good idea how to guide beginners and get them through the initial complexities. This made the advent of WDF both exciting and challenging. Exciting because we could dream of the day when we no longer had to discuss the differences between bus-first and function-first IRPs, and challenging because we needed to learn to not let that WDM way of thinking poison our approach to teaching the new Framework.

After having the opportunity to teach our course, Writing WDF Drivers for Windows, several times now, we've learned a few things. One of the most important of which is that the WDFQUEUE may be the single most underappreciated and misunderstood of the Framework objects. So, when is a Queue not just a queue? When it's a WDFQUEUE, of course!

Should have been called a WDF_FUNNEL?
The problem is clearly the fact that most devs have a pre-conceived notion of the meaning of the word "queue". When they hear this term, they immediately think of linked lists. If they're WDM driver writers, they think of LIST_ENTRY.  However, WDFQUEUEs are more like a funnel than a standard queue or linked list. In a WDFQUEUE, Requests get loaded into the funnel at the top (i.e. the Framework) and the manner in which the Requests come out at the bottom (i.e. your Framework driver) is configured by you.

Queue Basics
Before we can discuss the more interesting aspects of Queues, we need to get the basics down. The first concept is the various choices your driver has when it comes to picking a dispatch type for a Queue.

Queue Dispatch Types
When you create a Queue, you must specify its dispatch type. Dispatch type essentially controls a spigot at the bottom of the funnel. Your choices are:

1. Sequential: Requests are presented to your driver one at a time. The next Request does not arrive until your driver completes or forwards (discusses later) the current Request.
2. Parallel: Requests are presented to your driver, up to a maximum number set by you.  By default, the maximum number is  "unlimited."
3. Manual: Requests are explicitly retrieved from the Queue by your driver. 

Minor Aside: On the Queue versus Presented
Now is probably a good time to cover a subtle point: WDFQUEUEs are the funnel for WDFREQUESTs, and there are two states the WDFREQUESTs may be in relative to the funnel. The Request may be in the funnel waiting to be let through the spigot, which we call "on the queue", or might have been let through the spigot and given to the driver, which we call "queue presented" or simply "presented".

Another important thing to note is that Requests from a manual Queue are not presented automatically by the Framework to your driver.  Rather, they are removed by your driver when you need them. However, once removed from the Queue they are in the "presented" state and all of the same rules apply.

Queue States
A WDFQUEUE may be in one of several different states:

1. Started: Queue is operational and following the rules of its dispatch type.
2. Stopped: Requests may be inserted on the Queue but Requests are not being presented to the driver.
3. Purged: Queue is empty and attempts to insert subsequent Requests are immediately failed by the Framework.

Power Managed
By default WDFQUEUEs are "power managed", meaning that Requests going into the Queue may only be presented to the driver if the device is in the fully powered state.  When a Queue is power managed, the Framework automatically moves the Queue into the stopped state when the device transitions into a lower power state and back into the started state on resume.   Of course, you can override this behavior by setting the Queue to non-power managed (in which case, the state of the Queue isn't changed automatically based on the associated device's D-state).

Incoming Request Queues
A driver is free to create as many Queues as it likes. However, at least one of these Queues will need to be an "Incoming Request Queue", which is to say a Queue through which the Framework will present Requests to the driver.

A driver may create an Incoming Request Queue by creating a default Queue for its driver. Alternatively, the driver can create a non-default Queue and indicate to the Framework that it would like to convert it into an Incoming Request Queue for specific request types.  This is done by making one or more calls to WdfDeviceConfigureRequestDispatching, passing the handle to the WDFQUEUE and the Request type to be forwarded to the Queue by the Framework.

Other Driver Created Queues
As we mentioned, you're allowed to create as many Queues as you wish, and not all of them need be Incoming Requests Queues. You might, for example, create a manual Queue that your driver uses to hold Requests that are waiting for some event to occur. In this case, your driver would forward Requests presented through one of your Incoming Request Queues to your manual Queue.

Queue Events and Event Processing Callbacks
Next up on the basics chopping block are the events, and their associated Event Processing Callbacks, that WDFQUEUEs raise. The first events we will look at are those specifically related to processing I/O Requests. These include:

1. EvtIoRead: Raised by the Framework when a read Request arrives at the Queue.
2. EvtIoWrite: Raised by the Framework on arrival of a write Request.
3. EvtIoDeviceControl: Raised by the Framework when a device control Request arrives at the Queue
4. EvtIoInternalDeviceControl: Raised by the Framework when an internal device control Request arrives at the Queue.

There are also two events that relate to Queue state changes:

1. EvtIoStop: Queue is transitioning into the stopped state. This event is raised for each Request that has been presented to the driver but has not yet been completed or forwarded to another Queue. This call is important as it is your job to clean up any presented Requests that your driver is holding. You can do this by either completing the outstanding Requests or calling WdfRequestStopAcknowledge. Failure to do so will lead to the stop operation hanging indefinitely, preventing device removal or system shutdown.
2. EvtIoResume: Queue is transitioning out of the stopped state. This event is also raised for each Request that has been presented to the driver but has not yet been completed or forwarded to another Queue.

There is also one event that is raised for Request cancellation. If a Request is currently on a Queue and is aborted (either as a result of the user attempting to cancel it or the thread that issued it attempting to exit), the Queue raises an EvtIoCanceledOnQueue event. Note that this event is only raised when a Request is canceled while on a Queue (or immediately before being queued), thus it's not likely to provide much value in the case of a parallel Queue.

Different Uses For Queues
Now that we know a bit about Queues, we can talk about some of the more interesting applications of them in device drivers.

Queues As Long Term Storage
Drivers often need the ability to "park" Requests within the driver to be completed only when some asynchronous event occurs. For example, on the OSR USB FX2 device there is a switch pack that generates an interrupt with the state of the switches when they are toggled. Instead of having the application poll the driver to determine if something has changed, it would be nice to let the application send asynchronous IOCTLs that get completed when the switch pack changes.

This is a perfect fit for a manual Queue. Requests to read the switch pack arrive at one of your Incoming Request Queues and are then promptly forwarded to your manual Queue. When the device interrupts, you simply drain your manual Queue and complete all Requests with the state of the switch pack. Clean and simple, with the added benefit of cancellation of the Requests being completely handled for you!

Splitting Power Managed and Non-Power Managed Requests
Many device drivers have two classes of I/O operations: I/O operations that require device access and I/O operations that don't.  Examples of Requests that might fall into the second category are IOCTLs that simply gather statistical or configuration information. Because we'd all like to be green, and WDF makes aggressive power management so much easier, it might make sense to split these Requests into two different incoming Queues with different power management settings.

The Queue that receives reads and writes can be power managed, thus requiring the Framework to power our device back up before presenting the Request to us. However, our IOCTL Queue could be set to not be power managed, leaving our device to power down idle and save energy while we perform our IOCTL processing.

Serializing a Subset of I/O Operations
You might find yourself in a situation where your driver has three IOCTLs, we'll call them A, B, and C. Now, A and B can safely run in parallel, but your driver can only process one C IOCTL at a time. While there are many ways to solve this, why not leverage the Framework to do your serialization for you? Simply create a parallel Incoming Request Queue and a serial secondary Queue. Within your Incoming Request Queue's EvtIoDeviceControl handler, process A and B inline and forward any C Request to your serial Queue.

Queue Gotchas
Of course, there are going to be some gotchas that you might run into sooner or later. In order to give your later the benefit of our sooner, here are some things to note when writing your driver:

1. If a Request is presented to your driver from a sequential Queue and you forward that Request to a secondary Queue, the spigot on your sequential Queue is now open. Thus, if your device only supports one I/O Request at a time, parking the in progress Request on a manual Queue is not an option.
2. Don't use the dispatch type as a cheap means of serialization. While it might at first seem tempting to set your Incoming Request Queues to sequential and never worry about locking, that's not really the spirit of the sequential Queue. Sequential Queues should only be used when your device can only support one operation at a time, WDF's synchronization scope (not covered in this article) should be used in all other cases.
3. You cannot complete Requests while they are on a Queue.
4. There is no way to directly remove a specific WDFREQUEST from a Queue (i.e. there is no WdfIoQueueRemoveThisRequest function call).