Hi all,
At first, I have to give an apology for asking a this low level question here…
I’ve studied some articles about IRPs,
but I’m confused for asynchronous IO with following article, http://www.osronline.com/article.cfm?id=83
The step 5 in figure 8(Asynchronous case),
“The third driver completes the I/O operation. It calls IoCompleteRequest() and initiates first stage completion.”, its description of step 5.
My question,
Does this step need to call IoMarkIrpPending() for the IRP before calling IoCompleteRequest()?
If yes, I think the application should get a STATUS_PENDING message before the system handles the IO completion request stage#2?
Good for you trying to understand Windows (native, WDM-level) I/O Completion. It’s one of THE hardest concepts in all of Windows driver development.
No, because the third driver isn’t returning STATUS_PENDING. It’s completing the IRP, after filling in Irp->IoStatus.Status and Irp->IoStatus.Information.
But it doesn’t matter. You go on to write:
It could, in any case… if the first or second drivers “go asynchronous” and don’t call IoCallDriver from their dispatch routine. In this case, either of these drivers would queue the IRP (somewhere) to send it later, call IoMarkIrpPending and return STATUS_PENDING from their dispatch routine. Later, that same driver would dequeue the IRP, and send it to the next driver using IoCallDriver.
The article in question was written some 17 years ago (it was the first time Windows I/O Completion was described in detail ANYwhere by ANYone), and while it is still correct I think it could use some editing for clarification.
If a driver returns STATUS_PENDING, it MUST call IoMarkIrpPending before completing the IRP, or in its completion routine. If the driver explicitly returns STATUS_PENDING, it calls IoMarkIrpPending unconditionally. If the driver just propagates the result returned by the lower driver, it checks PendingReturned in its completion routine, and only calls IoMarkIrpPending if it’s set.
In case if STATUS_PENDING returned, the IRP may be completed or not before the return.
If the driver explicitly returns status other than STATUS_PENDING, the IRP must be completed before that, it MUST NOT call IoMarkIrpPending, Irp->IoStatus.Status MUST match the returned status.
I still can’t understand why the Completion Routine needs to check if Irp->ReturnedPending is true or false to determine if it needs to call IoMarkIrpPending().
Assume there is an upper driver, it has a dispatch routine doing these following steps,
1.IoSetCompletionRoutine() to set its own completion routine in the IRP that is prepared to forward to the next lower driver.
2.IoMarkIrpPending() for IRP.
3.Queue its IRP.
4.IoCallDriver() for relinquishing IRP ownership to next lower driver.
5.return STATUS_PENDING.
My question,
Is the upper driver possible to execute its Completion Routine before the step 5 return STATUS_PENDING to caller?
IoMarkIrpPending sets a flag in the current IO_STACK_LOCATION. The requirement is that this flag MUST match STATUS_PENDING returned by this driver.
On every level of the stack, the return value and the flag must match. In the end, it only matters on the topmost level, but the PendingReturned propagation mechanism needs them to match on all levels.
The driver may not know if it’s returning STATUS_PENDING, because it may just return whatever IoCallDriver returned. In this case, the only way to set the flag properly is to check PendingReturned in the completion routine and call IoMarkIrpPending based on that condition.
http://www.osronline.com/article.cfm?id=83
In asynchronous case,
I don’t understand the difference between step 5 and step 11.
If the Completion Routine of all drivers has been executed in step 5, 6, 7.
Why step 11 needs to execute IoCompleteRequest() again?
If so, I’m quite confident the text is correct as written.
Again, here is a description of each of the labeled steps:
1) The application program issues an I/O request (read, write, etc.) to the I/O Manager.
2) The I/O Manager dispatches the I/O request by building an IRP and passing it to the first driver in the chain.
3) The first driver continues processing by sending the I/O request to the second driver.
4) The second driver continues processing by sending the I/O request to the third driver.
5) The third driver completes the I/O operation. It calls IoCompleteRequest() and initiates first stage completion.
6) During first stage completion the second driver?s completion routine (if any) gets called back.
7) During first stage completion the first driver?s completion routine (if any) gets called back.
8) Next, IoCompleteRequest() queues the I/O completion APC to handle second stage I/O completion. First stage I/O completion is done.
9) The I/O completion APC is dequeued and second stage I/O completion starts. Note that this is done in the context of the thread which originated the I/O.
10) The I/O status and data is copied to the user address space.
11) Next IoCompleteRequest() returns back to the third driver.
12) The third driver returns a status value (e.g., SUCCESS or PENDING) to the second driver.
13) The second driver returns a status value to the first driver.
14) The first driver returns a status value to the I/O Manager.
15) The I/O operation is now complete and control returns to the caller.
In this case (the “asynchronous I/O” case) the second stage I/O completion processing (steps (9) and (10)) occur in an arbitrary order relative to the return of control from IoCompleteRequest() back to the application program (steps (11) through (14)).
Is this the list we’re talking about?
If so… if you’d kindly explain your question/concern I’ll try to explain the sequence further.
The important point about step 11 (actually 11 through 14) is explained in the sentence following the list, which I why I copied it above.
This is hard enough to explain in person, that it usually takes me almost 90 minutes to describe this process in class… and then, I’m only successful in communicating what’s going on to about half my students
Every driver in the WDK would pretty much qualify… at least every driver that doesn’t call IoCompleteRequest from its dispatch entry point without first sending the IRP to another driver with IoCallDriver.
This is the kind of thing that I was referring to previously when I indicated that the article could use some clarification. We’ll try to do that as part of moving and updating some of these venerable older articles from osronline.com to www.osr.com to make room for the new community site we’re building.
If the application uses multi-thread to send the asynchronous IO request by FILE_FLAG_OVERLAPPED.
Who is responsible to queue these IRPs? IO manager?
Because the Figure 8 in article didn’t indicate any IRP queue,
if there is no IRP queue, is it possible to implement asynchronous IO?
“When every driver has completed its corresponding sub-request, the I/O request is complete. The I/O Manager retrieves the status of the request from the Irp->IoStatus.Status field and retrieves the number of bytes transferred from the Irp->IoStatus.Information field.”, quotation from Handling IRPs(Completing an IRP).
I think the Completion Routine for device stack has been executed by step 6~7 in figure 8(Asynchronous case),
My understanding is as follows.
IoCompleteRequest() stage#1 gives Driver#2 and Driver#1 the ownership of IRP to execute their completion routine,
then IoCompleteRequest() stage#1 takes back the ownership of IRP as the Driver#1 completely executed its completion routine.
Then the IRP would leave in IoCompleteRequest() stage#1 to wait for its IoCompletion APC is completed in IoCompleteRequest() stage#2.
Finally, IoCompleteRequest() stage#1 starts to return the IRP to drivers, and the IoManager delete the IRP as it explicitly retrieve the status of request.
If the application uses multi-thread to send the asynchronous IO request by FILE_FLAG_OVERLAPPED.
Who is responsible to queue these IRPs? IO manager?
There’s no queue on the application level.
If a file object is created without FILE_FLAG_OVERLAPPED,
only one request can be in at any time.
Other threads trying to send a request just will block until
the active request completes.
if there is no IRP queue, is it possible to implement asynchronous IO?
Yes, in a trivial way (the driver immediately fails all requests,
until the active one completes).
The general good practice is to avoid blocking (KeWaitXxx or such) in a dispatch routine, and queue the IRP instead.
The only exception if this is NOT overlapped IO (IoIsOperationSynchronous). In this case, you can block (IO manager will block anyway on return from you).
But, if the operation does not require blocking, like “query values” IOCTLs, you can do it in the dispatch routine without a queue.
“Marion Bond” wrote in message news:xxxxx@ntdev… There is no good reason not to implement IRP queuing for most drivers. KMDF makes this trivial and supports cancel too. In general, drivers in windows are ASSUMED to have this support and applications either use overlapped IO or don’t. In modern Windows, a driver that does not support overlapped IO and IRP queuing is almost unheard of; especially a new driver that isn’t for some custom embedded solution
Sent from Surface Pro
From: Pavel A. Sent: Tuesday, August 26, 2014 5:41 AM To: Windows System Software Devs Interest List
On 26-Aug-2014 04:51, xxxxx@hotmail.com wrote: > If the application uses multi-thread to send the asynchronous IO request by FILE_FLAG_OVERLAPPED. > Who is responsible to queue these IRPs? IO manager?
There’s no queue on the application level. If a file object is created without FILE_FLAG_OVERLAPPED, only one request can be in at any time. Other threads trying to send a request just will block until the active request completes.
> if there is no IRP queue, is it possible to implement asynchronous IO?
Yes, in a trivial way (the driver immediately fails all requests, until the active one completes).
