Choosing MSI over MSI-X

Hi. It’s my understanding that when presented with a device that supports MSI and MSI-X, Windows will choose MSI-X if the driver attempts to enable MSI. Is that set in stone or can that behavior be altered?

I have a device that supports MSI and MSI-X. Unfortunately, the hardware guys designed the device reset mechanism such that it also wipes out the MSI-X table. Now, there are a number of reasons why the driver might need to reset the device during normal operation so this is problematic since there appears to be no obvious way to access and repopulate the MSI-X table from a function driver. Thus, it seems that the driver would need to delete the old interrupt object and create a new one each time the device is reset. That seems…unclean. (Is there a better way?)

For this reason, I’d like to use MSI interrupts since they’re unaffected by device resets. Can Windows be forced to use MSI even if a device claims to also support MSI-X?

JM

Hi JM, I have the same problem. I need to use MSI other than MSI-X on
the hardware which claims to support both. Have you find the way?


*From*: xxxxx@gmail.com
*Sent*: Thu, 25 Sep 2014 13:45:03 -0400 (EDT)
*To*: Windows System Software Devs Interest List
Subject: [ntdev] Choosing MSI over MSI-X

> Hi. It’s my understanding that when presented with a device that supports MSI and MSI-X, Windows will choose MSI-X if the driver attempts to enable MSI. Is that set in stone or can that behavior be altered?
>
> I have a device that supports MSI and MSI-X. Unfortunately, the hardware guys designed the device reset mechanism such that it also wipes out the MSI-X table. Now, there are a number of reasons why the driver might need to reset the device during normal operation so this is problematic since there appears to be no obvious way to access and repopulate the MSI-X table from a function driver. Thus, it seems that the driver would need to delete the old interrupt object and create a new one each time the device is reset. That seems…unclean. (Is there a better way?)
>
> For this reason, I’d like to use MSI interrupts since they’re unaffected by device resets. Can Windows be forced to use MSI even if a device claims to also support MSI-X?
>
> JM
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Do you have the PCIe config descriptors programmable by NVRAM?

Did you mean the things such as PCI_MSI_CAPABILITY or
PCI_MSIX_CAPABILITY? I thought we don’t need to change that directly.
Are there any functions or registry configurations?


*From*: xxxxx@broadcom.com
*Sent*: Sat, 11 Oct 2014 16:12:04 -0400 (EDT)
*To*: Windows System Software Devs Interest List
*Subject*: RE:[ntdev] Choosing MSI over MSI-X

Do you have the PCIe config descriptors programmable by NVRAM?


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

re: NVRAM settings

Unfortunately, no. I unfortunately cannot tweak the EEPROM and I’ve run into considerable pushback from the hardware guys at the mere suggestion that they change the EEPROM to advertise only MSI.

re: my idea of deleting the old interrupt handler and re-creating a new one after a device reset

My initial attempts at this have also failed. It seems KMDF forbids WdfInterruptCreate after EvtDeviceAdd has completed. Is there a way around this? Can I induce KMDF to quietly delete and re-create the device?

(In my situation, driver-initiated device resets occur only rarely (but they do occur) so recreating the device and going through the EvtDeviceAdd and EvtPrepareHardware would probably be tolerable. But it needs to happen under the covers without disturbing any applications which might be trying to use the device. So unloading and reloading the driver is not an option.)

I guess I’m a little surprised that there’s no way to force Windows to ignore advertised MSI-X functionality and use MSI.

>I unfortunately cannot tweak the EEPROM and I’ve run into
considerable pushback from the hardware guys at the mere suggestion that they
change the EEPROM to advertise only MSI.

This is where the management should get involved. If EEPROM change can rescue the project from FUBAR, then it should be done.

If they don’t want to change EEPROM, they got to fix the chip reset.

This is categorically incorrect.

It is *very* common to call WdfInterruptCreate from your EvtDevicePrepareHardware Event Processing Callback.

To be honest, I’m sort of surprised as well.

If I remember, I’ll try to look into this a bit when I’m back in the office. No guarantees… but it seems logical to me that my driver should be able to force a “fall back” to MSI when MSI-x is available. Though, frankly, if you have a device that implements MSI-x correctly, and your device moves *any* significant amount of data or uses lots of interrupt lines, the advantages you gain with MSI-X make it a far better choice than MSI. If I were the hw dev, I wouldn’t want to remove MSI-x support either.

Peter
OSR
@OSRDrivers

Curious. I just tried to move my WdfInterruptCreate stuff to my PrepareHardware callback and received 0xC0000184 (STATUS_INVALID_DEVICE_STATE). In fact I’ve received that return code every time I’ve tried to invoke it anywhere outside of my EvtDriverDeviceAdd callback. Do I need to use some additional magic?

JM

Calling WdfInterruptCreate in EvtDevicePrepareHardware is supported in KMDF 1.11+.

… and the current version is 1.13 …

You definitely want to “move on up” to the latest and greatest WDK.

Peter
OSR
@OSRDrivers

On 10/20/2014 11:52 AM, xxxxx@osr.com wrote:

… and the current version is 1.13 …

You definitely want to “move on up” to the latest and greatest WDK.

Is this the case if you still need to target XP? I’ve been sticking
with the Windows 7 WDK and KMDF version 1.11 in those cases…is that
the wrong way to go about it these days?

– mkj


//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

XP? I remember that… it was so long ago

No, Mr. Jones. I think if you have to target XP you’re almost certainly better off avoiding a headache and staying with the Win 7 WDK at this point. It’s sad, but it’s true.

I would, however, strongly counsel you to use SAL V2 and do occasional validation builds using the newest tool chain. The static analysis tools (code analysis and SDV) have gone from annoying to indispensable in the past couple of releases, especially when you combine them with the capabilities of SAL V2 (to validate things like locking, for example).

Peter
OSR
@OSRDrivers

On 10/20/2014 12:41 PM, xxxxx@osr.com wrote:

XP? I remember that… it was so long ago

I know, right? But the client wants what the client wants. At least
I’m not getting calls to support NT 3.5 or Win9x anymore :slight_smile:

No, Mr. Jones. I think if you have to target XP you’re almost certainly better off avoiding a headache and staying with the Win 7 WDK at this point. It’s sad, but it’s true.

I would, however, strongly counsel you to use SAL V2 and do occasional validation builds using the newest tool chain. The static analysis tools (code analysis and SDV) have gone from annoying to indispensable in the past couple of releases, especially when you combine them with the capabilities of SAL V2 (to validate things like locking, for example).

Seems like it should be safe to install VS2013 and WDK 8.1 alongside
VS2010 and the Windows 7 WDK…any gotchas I should be wary of?

– mkj


//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

It’s worked for me, in multiple installs so far. I haven’t HEARD of any problems.

Peter
OSR
@OSRDrivers

On Oct 20, 2014, at 10:57 AM, xxxxx@osr.com wrote:

It’s worked for me, in multiple installs so far. I haven’t HEARD of any problems.

At present, my development machine contains the XP DDK (2600), the 2003 DDK (3790, both releases), the Vista WDK (6000), the Win 7 WDK (7600), the Win 8 WDK (9200), and the Win 8.1 WDK (9600). Plus, Visual Studio 98, 2002, 2003, 2005, 2008, 2010, 2012, and 2013. They all play together very nicely.

Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I wanted to close the loop on this topic:

I am not aware of any facility that allows you to force selection of MSI over MSI-X capabilities, for a device that has valid MSI-X capabilities.

Peter
OSR
@OSRDrivers

Thanks for checking, Peter.

I do have a workaround for my situation. It turns out the MSI-X table *can* be modified and, thus, restored by a function driver: The GUID_MSIX_TABLE_CONFIG_INTERFACE contains an accessor routine “SetTableEntry” that does the dirty work. Using this function to completely restore an erased MSI-X table is probably not one of the use cases Microsoft had in mind but it does seem to work in my limited testing.

Nice!

Glad to hear you got things sorted.

I think it would be useful to be able to elect MSI support over MSI-X support, for driver testing purposes.

Peter
OSR
@OSRDrivers