NextLuRequest for scsi miniport driver

NTDEV
1

Hi,

When I am doing DTM test for my scsi miniport driver, I found that SCSI
verify will pop an error 0xF1. From WLK doc I get the error message is ’
The miniport driver called ScsiPortNotification to ask for
NextLuRequest, but an untagged request is still active’. Does that means
scsi miniport driver can just get one scsi command from scsi port driver
at a time?

I am developing Xen windows pv drivers. My scsi miniport driver send/get
data to/from backend point by a ring buffer. Take write command as an
example. In startio routine, when the data in write command are send to
ring buffer, I call ScsiPortNotification(NextLuRequest,…) to notify
scsi port driver that I can deal with another scsi command. In interrupt
routine, if scsi command processed properly, I will call
ScsiPortNotification(RequestComplete,…) and
ScsiPortNotification(NextLuRequest,…). This approach works fine (I mean
common Read/Write) but cannot pass WHQL test. If scsi miniport driver
can just deal with one scsi command at a time, I think ring buffer
cannot run fully and it’s inefficiency. Any way to improve that except
transfer to stor miniport driver?

Thanks
Wayne

2

> The miniport driver called ScsiPortNotification to ask for

NextLuRequest, but an untagged request is still active’. Does that means
scsi miniport driver can just get one scsi command from scsi port driver
at a time?

Maybe only 1 untagged request per LUN, and not per adapter?

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

3

Could you please give me a detail explain for ‘untagged request’? Any flag on SRB need to set or some ScsiPortxxx need to call for request?

Thanks
Wayne

4

> Could you please give me a detail explain for ‘untagged request’?

This is the very old core SCSI concept.

The thing is that the SCSI request (as delivered to the LUN over the cable) has the “queue tag” field in it. The field is 1 byte, can be 255 (untagged) or any value of [0…255) (tagged).

The LUN can execute in parallel either a) 1 and only 1 untagged request OR b) any number of tagged requests, they must have different queue tag values.

This feature is similar to SATA’s NCQ (allows the disk drive to order the IO requests itself by being aware on several of them at once), and was traditionally one of the 2 large advantages of SCSI over IDE (disconnected operations was the second, this ceased after IDE became SATA with no shared controller/cabling hardware among the drives).

In Windows, it is the class driver who makes a decision on whether the request is tagged (allows concurrent execution with other similar requests) or untagged (requires the LUN in monopoly mode till will be completed by the LUN).

IIRC this decision is made by using the “queue action enable” flag on the SRB.

As about assigning the queue tag value itself - this is done inside SCSIPORT between the class driver and the miniport.

As about your error - I assume that there is a rule on SCSIPORT’s miniport to never call NextLuRequest if there is an untagged request currently in progress. I was not aware about this, but looks like it is so, though SCSIPORT would definitely be able of handling this safely.

Once more on NextRequest and NextLuRequest:

  • after calling your StartIo, SCSIPORT will no more call StartIo for the next request till either a) the current request will be completed or b) Next(Lu)Request will be called. NextRequest actually does nothing else.
  • NextLuRequest, besides doing the same as NextRequest, also carries the second functionality. Namely - SCSIPORT will not call StartIo for the next request for a particular LUN until either the current request is completed or NextLuRequest is called. This is the additional rule to the one listed above.

So, NextLuRequest is NextRequest + opening also the per-LUN “logical lock”. The miniport can never use them - this mean that it will have only 1 outstanding SRB per controller, this is slow but should work.

If the miniport wants parallelism, then it’s StartIo path should check _whether there are enough resources (DMA mailslots, controller registers or such) to execute one more request, and, in this case, call NextRequest from StartIo.

More so, if the miniport wants per-LUN parallelism (i.e. queue tags) - then it’s StartIo, for a tagged request, should check whether there are enough resources for one more request for this LUN, and, if yes, call NextLuRequest instead of NextRequest.

In your case, I would suggest to never call NextLuRequest for SP_UNTAGGED SRB. Just allow it to complete the usual way, no other requests can go to this LUN before this anyway.

The StartIo logic should be:

  • submit the request for execution
  • if( EnoughGlobalAdapterResourcesForOneMoreRequest )
    {
    if( ( Srb->QueueTag != SP_UNTAGGED ) && EnoughPerLUNResourcesForOneMoreRequest )
    ScsiPortNotification(NextLuRequest);
    else
    ScsiPortNotification(NextRequest);
    }

Looks like NextLuRequest should be called only in StartIo path for the tagged request.

Try this.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

5

SCSIPORT is pretty particular about not letting the miniport violate the basic SCSI-2 semantics which includes mixing of tagged and untagged requests (this is a no-no that will blow a large number of SCSI-2 devices and some controllers out of the water).

I think you can set MulitpleRequestPerLu in your adapter settings in order to tell SCSIPORT that you’ll take care of the protocol requirements. However untagged requests are also used as a gate to ensure that only one request is running so you may need to consider supporting the semantic.

Are all of the requests you get untagged? If so are you setting the flag in your INQUIRY data which says you can support command queuing?

-p

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Friday, February 06, 2009 10:38 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] NextLuRequest for scsi miniport driver

Could you please give me a detail explain for ‘untagged request’?

This is the very old core SCSI concept.

The thing is that the SCSI request (as delivered to the LUN over the cable) has the “queue tag” field in it. The field is 1 byte, can be 255 (untagged) or any value of [0…255) (tagged).

The LUN can execute in parallel either a) 1 and only 1 untagged request OR b) any number of tagged requests, they must have different queue tag values.

This feature is similar to SATA’s NCQ (allows the disk drive to order the IO requests itself by being aware on several of them at once), and was traditionally one of the 2 large advantages of SCSI over IDE (disconnected operations was the second, this ceased after IDE became SATA with no shared controller/cabling hardware among the drives).

In Windows, it is the class driver who makes a decision on whether the request is tagged (allows concurrent execution with other similar requests) or untagged (requires the LUN in monopoly mode till will be completed by the LUN).

IIRC this decision is made by using the “queue action enable” flag on the SRB.

As about assigning the queue tag value itself - this is done inside SCSIPORT between the class driver and the miniport.

As about your error - I assume that there is a rule on SCSIPORT’s miniport to never call NextLuRequest if there is an untagged request currently in progress. I was not aware about this, but looks like it is so, though SCSIPORT would definitely be able of handling this safely.

Once more on NextRequest and NextLuRequest:

  • after calling your StartIo, SCSIPORT will no more call StartIo for the next request till either a) the current request will be completed or b) Next(Lu)Request will be called. NextRequest actually does nothing else.
  • NextLuRequest, besides doing the same as NextRequest, also carries the second functionality. Namely - SCSIPORT will not call StartIo for the next request for a particular LUN until either the current request is completed or NextLuRequest is called. This is the additional rule to the one listed above.

So, NextLuRequest is NextRequest + opening also the per-LUN “logical lock”. The miniport can never use them - this mean that it will have only 1 outstanding SRB per controller, this is slow but should work.

If the miniport wants parallelism, then it’s StartIo path should check _whether there are enough resources (DMA mailslots, controller registers or such) to execute one more request, and, in this case, call NextRequest from StartIo.

More so, if the miniport wants per-LUN parallelism (i.e. queue tags) - then it’s StartIo, for a tagged request, should check whether there are enough resources for one more request for this LUN, and, if yes, call NextLuRequest instead of NextRequest.

In your case, I would suggest to never call NextLuRequest for SP_UNTAGGED SRB. Just allow it to complete the usual way, no other requests can go to this LUN before this anyway.

The StartIo logic should be:

  • submit the request for execution
  • if( EnoughGlobalAdapterResourcesForOneMoreRequest )
    {
    if( ( Srb->QueueTag != SP_UNTAGGED ) && EnoughPerLUNResourcesForOneMoreRequest )
    ScsiPortNotification(NextLuRequest);
    else
    ScsiPortNotification(NextRequest);
    }

Looks like NextLuRequest should be called only in StartIo path for the tagged request.

Try this.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

> Once more on NextRequest and NextLuRequest:

  • after calling your StartIo, SCSIPORT will no more call StartIo for
    the
    next request     till either a) the current request will be completed or
    b)
    Next(Lu)Request will be called. NextRequest actually does nothing
    else.
  • NextLuRequest, besides doing the same as NextRequest, also carries
    the
    second functionality. Namely - SCSIPORT will not call StartIo for the
    next
    request for a particular LUN until either the current request is
    completed or NextLuRequest is called. This is the additional rule to
    the
    one listed above.

When you are saying “current request is completed” are you implying that
RequestComplete is called? I’m pretty sure that StartIo is only called
after NextRequest or NextLuRequest is called, RequestComplete does not
tell scsiport that it is okay to send another request, only that the
current one is done.

That’s the way I always understood it and I can’t see how my code could
work if it was any other way. The examples I have read have a
‘NextRequest’ call immediately after their ‘RequestComplete’ calls too,
which is what I am doing unless I don’t want another request.

Thanks

James

7

> - submit the request for execution

  • if( EnoughGlobalAdapterResourcesForOneMoreRequest )
    {
    if( ( Srb->QueueTag != SP_UNTAGGED ) && EnoughPerLUNResourcesForOneMoreRequest )
    ScsiPortNotification(NextLuRequest);
    else
    ScsiPortNotification(NextRequest);
    }

Looks like NextLuRequest should be called only in StartIo path for the tagged request.

Try this.

Thanks for your message. It’s very helpful for me. I have done on my
code as what you said. It seems can work.
When I do some test for my driver I found that when I use IoMeter to do
performance test for that block device which scsi miniport driver
serviced for, I cannot get any untagged SRB even with more then one
outstanding I/O request. But when I use SDStress to do stress test with
more then one thread, driver get lots of untagged SRB. So now my
question is which scenario can create untagged SRB?

Thanks
Wayne