EvtDevicePrepareHardware not getting called

Michael_Jones · February 7, 2009, 6:57pm

I’m writing a driver for a PCI device using KMDF 1.7. I’m installing on
32-bit XP (SP2), using the “Found new hardware” wizard. The install
hangs with the WdfCoInstaller01007.dll step complete (I *believe* it has
completed that step; the WDF setup log file looks happy).

This happens to be a Core2 Duo (fortunately), since Task Manager shows
one processor spinning away in “System”. If I break into the debugger
and looks at the threads for the system process, I see one which is
active, and it has an interesting stack:

ChildEBP RetAddr Args to Child
bacfb78c 80544109 00000001 00000000 000000d1 nt!RtlpBreakWithStatusInstruction (FPO: [1,0,0])
bacfb78c 805dc3f1 00000001 00000000 000000d1 nt!KeUpdateSystemTime+0x175 (FPO: [0,2] TrapFrame @ bacfb7a0)
bacfb81c 805dc546 e17525d8 0000c000 00000000 nt!RtlpIsRangeAvailable+0x89 (FPO: [11,0,0])
bacfb874 ba774f54 e1b0a8a0 00001000 00000000 nt!RtlFindRange+0xfc (FPO: [12,6,4])
bacfb8bc ba76dd12 e138a54c e13cb310 e138a54c pci!ArbFindSuitableRange+0x9e (FPO: [2,1,4])
bacfb8d4 ba773c6f e138a54c e13cb310 e138a54c pci!ario_FindSuitableRange+0xb6 (FPO: [2,0,0])
bacfb8f0 ba7749a6 e138a54c 003cb000 bacfb9ac pci!ArbAllocateEntry+0x8d (FPO: [2,0,4])
bacfb910 ba773ac0 0000000f e13a90a4 bacfb9ac pci!ArbTestAllocation+0xb2 (FPO: [2,1,4])
bacfb92c 80594982 e138a54c 00000000 bacfb94c pci!ArbArbiterHandler+0x32 (FPO: [3,0,4])
bacfb95c 80595c01 bacfb9ac e10c81c0 00000028 nt!IopTestConfiguration+0x42 (FPO: [1,4,4])
bacfb98c 80596fde e10c8008 0000000b ffffffff nt!IopFindBestConfiguration+0x3d (FPO: [3,6,4])
bacfb9e0 805974b2 e1b9f810 e1b2a2f8 00000000 nt!IopRebalance+0x1da (FPO: [2,13,0])
bacfba38 8059a09a bacfba98 01cfba74 00000000 nt!IopAllocateResources+0x1bc (FPO: [5,15,0])
bacfba90 8059a202 00000001 e1b2a2f8 00000001 nt!IopAssignResourcesToDevices+0x100 (FPO: [4,13,0])
bacfbac8 80590e26 89bb75e8 00000000 bacfbb15 nt!IopProcessAssignResources+0xd6 (FPO: [3,5,0])
bacfbd24 805914ba 89bb75e8 00000001 00000000 nt!PipProcessDevNodeTree+0xa6 (FPO: [8,143,4])
bacfbd54 804f66bc 00000003 8055a5c0 8056375c nt!PiRestartDevice+0x80 (FPO: [1,0,0])
bacfbd7c 80537757 00000000 00000000 89c318b8 nt!PipDeviceActionWorker+0x168 (FPO: [1,4,0])
bacfbdac 805ce794 00000000 00000000 00000000 nt!ExpWorkerThread+0xef (FPO: [1,6,0])
bacfbddc 805450ce 80537668 00000001 00000000 nt!PspSystemThreadStartup+0x34 (FPO: [Non-Fpo])
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

It sure looks like the system is trying to assign some resources to this
thing, and running around in circles. Any suggestions for what to look
for? If it helps, here is the output from !pci. The only thing that
looks a little odd to me is that ‘Command’ does not have IoSpaceEn set,
but BAR1 contains an I/O space (BAR0 and BAR1 map the same set of
registers, one in memory and one I/O, both 4KB in size). I’m not sure
what my next step is…

PCI Configuration Space (Segment:0000 Bus:06 Device:05 Function:00)
Common Header:
00: VendorID 10e3 Tundra Semiconductor Corp.
02: DeviceID 0000
04: Command 0006 MemSpaceEn BusInitiate
06: Status 0200 DEVSELTiming:1
08: RevisionID 02
09: ProgIF 00
0a: SubClass 80 Other Bridge
0b: BaseClass 06 Bridge Device
0c: CacheLineSize 0000
0d: LatencyTimer 20
0e: HeaderType 00
0f: BIST 00
10: BAR0 d7fff000
14: BAR1 0000e001
18: BAR2 00000000
1c: BAR3 00000000
20: BAR4 00000000
24: BAR5 00000000
28: CBCISPtr 00000000
2c: SubSysVenID 0000
2e: SubSysID 0000
30: ROMBAR 00000000
34: CapPtr 00
3c: IntLine 0b
3d: IntPin 01
3e: MinGnt 03
3f: MaxLat 00
Device Private:
40: 00000000 00000000 00000000 00000000
50: 00000000 00000000 00000000 00000000
60: 00000000 00000000 00000000 00000000
70: 00000000 00000000 00000000 00000000
80: 00000000 00000000 00000000 00000000
90: 00000000 00000000 00000000 00000000
a0: 00000000 00000000 00000000 00000000
b0: 00000000 00000000 00000000 00000000
c0: 00000000 00000000 00000000 00000000
d0: 00000000 00000000 00000000 00000000
e0: 00000000 00000000 00000000 00000000
f0: 00000000 00000000 00000000 00000000

–

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

Calvin_Guan-2 · February 7, 2009, 11:46pm

Hi Michael,

I can’t tell fer sure what went wrong. I’m suspecting the pci system is running short of port io space. Can you post the output of “!arbiter 1”?

I’d try to disable the IO BAR to confirm whether or not it’s indeed causing the problem. I’d also confirm how much the card is actually asking for, and what alignment requirement is being requested, i.e. blow all-one to bar1 and then read it back to see what you got. You can do it with windbag or using a dos program.

Although it’s not related to the problem you are seeing, it’s weird that the device doesn’t seem to support PM CAP. All pci devices should have supported d0 and d3 at least.

Happy hacking!

–
Calvin Guan
Broadcom Corp.
Connecting Everything(r)

— On Sat, 2/7/09, Michael Jones wrote:

> From: Michael Jones
> Subject: [ntdev] EvtDevicePrepareHardware not getting called
> To: “Windows System Software Devs Interest List”
> Date: Saturday, February 7, 2009, 3:56 PM
> I’m writing a driver for a PCI device using KMDF 1.7.
> I’m installing on 32-bit XP (SP2), using the “Found
> new hardware” wizard. The install hangs with the
> WdfCoInstaller01007.dll step complete (I believe it has
> completed that step; the WDF setup log file looks happy).
>
> This happens to be a Core2 Duo (fortunately), since Task
> Manager shows one processor spinning away in
> “System”. If I break into the debugger and looks
> at the threads for the system process, I see one which is
> active, and it has an interesting stack:
>
> > ChildEBP RetAddr Args to Child bacfb78c
> 80544109 00000001 00000000 000000d1
> nt!RtlpBreakWithStatusInstruction (FPO: [1,0,0])
> > bacfb78c 805dc3f1 00000001 00000000 000000d1
> nt!KeUpdateSystemTime+0x175 (FPO: [0,2] TrapFrame @
> bacfb7a0)
> > bacfb81c 805dc546 e17525d8 0000c000 00000000
> nt!RtlpIsRangeAvailable+0x89 (FPO: [11,0,0])
> > bacfb874 ba774f54 e1b0a8a0 00001000 00000000
> nt!RtlFindRange+0xfc (FPO: [12,6,4])
> > bacfb8bc ba76dd12 e138a54c e13cb310 e138a54c
> pci!ArbFindSuitableRange+0x9e (FPO: [2,1,4])
> > bacfb8d4 ba773c6f e138a54c e13cb310 e138a54c
> pci!ario_FindSuitableRange+0xb6 (FPO: [2,0,0])
> > bacfb8f0 ba7749a6 e138a54c 003cb000 bacfb9ac
> pci!ArbAllocateEntry+0x8d (FPO: [2,0,4])
> > bacfb910 ba773ac0 0000000f e13a90a4 bacfb9ac
> pci!ArbTestAllocation+0xb2 (FPO: [2,1,4])
> > bacfb92c 80594982 e138a54c 00000000 bacfb94c
> pci!ArbArbiterHandler+0x32 (FPO: [3,0,4])
> > bacfb95c 80595c01 bacfb9ac e10c81c0 00000028
> nt!IopTestConfiguration+0x42 (FPO: [1,4,4])
> > bacfb98c 80596fde e10c8008 0000000b ffffffff
> nt!IopFindBestConfiguration+0x3d (FPO: [3,6,4])
> > bacfb9e0 805974b2 e1b9f810 e1b2a2f8 00000000
> nt!IopRebalance+0x1da (FPO: [2,13,0])
> > bacfba38 8059a09a bacfba98 01cfba74 00000000
> nt!IopAllocateResources+0x1bc (FPO: [5,15,0])
> > bacfba90 8059a202 00000001 e1b2a2f8 00000001
> nt!IopAssignResourcesToDevices+0x100 (FPO: [4,13,0])
> > bacfbac8 80590e26 89bb75e8 00000000 bacfbb15
> nt!IopProcessAssignResources+0xd6 (FPO: [3,5,0])
> > bacfbd24 805914ba 89bb75e8 00000001 00000000
> nt!PipProcessDevNodeTree+0xa6 (FPO: [8,143,4])
> > bacfbd54 804f66bc 00000003 8055a5c0 8056375c
> nt!PiRestartDevice+0x80 (FPO: [1,0,0])
> > bacfbd7c 80537757 00000000 00000000 89c318b8
> nt!PipDeviceActionWorker+0x168 (FPO: [1,4,0])
> > bacfbdac 805ce794 00000000 00000000 00000000
> nt!ExpWorkerThread+0xef (FPO: [1,6,0])
> > bacfbddc 805450ce 80537668 00000001 00000000
> nt!PspSystemThreadStartup+0x34 (FPO: [Non-Fpo])
> > 00000000 00000000 00000000 00000000 00000000
> nt!KiThreadStartup+0x16
>
> It sure looks like the system is trying to assign some
> resources to this thing, and running around in circles. Any
> suggestions for what to look for? If it helps, here is the
> output from !pci. The only thing that looks a little odd to
> me is that ‘Command’ does not have IoSpaceEn set,
> but BAR1 contains an I/O space (BAR0 and BAR1 map the same
> set of registers, one in memory and one I/O, both 4KB in
> size). I’m not sure what my next step is…
>
> PCI Configuration Space (Segment:0000 Bus:06 Device:05
> Function:00)
> Common Header:
> 00: VendorID 10e3 Tundra Semiconductor Corp.
> 02: DeviceID 0000
> 04: Command 0006 MemSpaceEn BusInitiate
> 06: Status 0200 DEVSELTiming:1
> 08: RevisionID 02
> 09: ProgIF 00
> 0a: SubClass 80 Other Bridge
> 0b: BaseClass 06 Bridge Device
> 0c: CacheLineSize 0000
> 0d: LatencyTimer 20
> 0e: HeaderType 00
> 0f: BIST 00
> 10: BAR0 d7fff000
> 14: BAR1 0000e001
> 18: BAR2 00000000
> 1c: BAR3 00000000
> 20: BAR4 00000000
> 24: BAR5 00000000
> 28: CBCISPtr 00000000
> 2c: SubSysVenID 0000
> 2e: SubSysID 0000
> 30: ROMBAR 00000000
> 34: CapPtr 00
> 3c: IntLine 0b
> 3d: IntPin 01
> 3e: MinGnt 03
> 3f: MaxLat 00
> Device Private:
> 40: 00000000 00000000 00000000 00000000
> 50: 00000000 00000000 00000000 00000000
> 60: 00000000 00000000 00000000 00000000
> 70: 00000000 00000000 00000000 00000000
> 80: 00000000 00000000 00000000 00000000
> 90: 00000000 00000000 00000000 00000000
> a0: 00000000 00000000 00000000 00000000
> b0: 00000000 00000000 00000000 00000000
> c0: 00000000 00000000 00000000 00000000
> d0: 00000000 00000000 00000000 00000000
> e0: 00000000 00000000 00000000 00000000
> f0: 00000000 00000000 00000000 00000000
>
> –
> //
> // Michael K. Jones
> // Stone Hill Consulting, LLC
> // http://www.stonehill.com
> //
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars
> visit: http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online
> at http://www.osronline.com/page.cfm?name=ListServer

Michael_Jones · February 9, 2009, 2:04pm

The output from !arbiter was too big (the Lyris list manager rejected by
reply). I’ve stuck it on my web site (http://www.stonehill.com/output.txt).

The file starts with the result of writing all ones to BAR1; it wants a
4KB chunk for I/O space (which agrees with the chip spec), but which I
won’t even use, since BAR0 provides access to the same chip registers in
memory. I’ve asked the BIOS guy if we can disable BAR1.

I don’t see anything obvious in the output from !arbiter, other than it
does not appear to have allocated any I/O space to the device, but I
admit that I don’t really know what to look for.

It seems to me 4KB is a lot of I/O space to ask for; perhaps the “stuck”
thread is not so much stuck as it is trying valiantly to find 4K of I/O
space by rejiggering everything else; I could imagine that taking some time.

The fact that the device is a chip soldered on the main board might be
why the the PM cap is not set; I’ve asked the BIOS guy about that too.

Cheers,

–mkj

Calvin Guan wrote:

Hi Michael,

I can’t tell fer sure what went wrong. I’m suspecting the pci system is running short of port io space. Can you post the output of “!arbiter 1”?

I’d try to disable the IO BAR to confirm whether or not it’s indeed causing the problem. I’d also confirm how much the card is actually asking for, and what alignment requirement is being requested, i.e. blow all-one to bar1 and then read it back to see what you got. You can do it with windbag or using a dos program.

Although it’s not related to the problem you are seeing, it’s weird that the device doesn’t seem to support PM CAP. All pci devices should have supported d0 and d3 at least.

Happy hacking!

–
Calvin Guan
Broadcom Corp.
Connecting Everything(r)

–

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

David_J_Craig · February 9, 2009, 3:59pm

A 64KB bar is not unusual and works well.

David Craig
Broadcom Corp.
Connecting Everything(r)

“Michael Jones” wrote in message
news:xxxxx@ntdev…
> The output from !arbiter was too big (the Lyris list manager rejected by
> reply). I’ve stuck it on my web site
> (http://www.stonehill.com/output.txt).
>
> The file starts with the result of writing all ones to BAR1; it wants a
> 4KB chunk for I/O space (which agrees with the chip spec), but which I
> won’t even use, since BAR0 provides access to the same chip registers in
> memory. I’ve asked the BIOS guy if we can disable BAR1.
>
> I don’t see anything obvious in the output from !arbiter, other than it
> does not appear to have allocated any I/O space to the device, but I admit
> that I don’t really know what to look for.
>
> It seems to me 4KB is a lot of I/O space to ask for; perhaps the “stuck”
> thread is not so much stuck as it is trying valiantly to find 4K of I/O
> space by rejiggering everything else; I could imagine that taking some
> time.
>
> The fact that the device is a chip soldered on the main board might be why
> the the PM cap is not set; I’ve asked the BIOS guy about that too.
>
> Cheers,
>
> --mkj
>
>
>
> Calvin Guan wrote:
>> Hi Michael,
>>
>> I can’t tell fer sure what went wrong. I’m suspecting the pci system is
>> running short of port io space. Can you post the output of “!arbiter 1”?
>> I’d try to disable the IO BAR to confirm whether or not it’s indeed
>> causing the problem. I’d also confirm how much the card is actually
>> asking for, and what alignment requirement is being requested, i.e. blow
>> all-one to bar1 and then read it back to see what you got. You can do it
>> with windbag or using a dos program. Although it’s not related to the
>> problem you are seeing, it’s weird that the device doesn’t seem to
>> support PM CAP. All pci devices should have supported d0 and d3 at least.
>> Happy hacking!
>>
>> –
>> Calvin Guan
>> Broadcom Corp.
>> Connecting Everything(r)
>
> –
>
> //
> // Michael K. Jones
> // Stone Hill Consulting, LLC
> // http://www.stonehill.com
> //
>

Maxim_S_Shatskih · February 9, 2009, 4:47pm

>A 64KB bar is not unusual and works well.

Memory - maybe, but IO space is limited to 64K total, so, 4K IO BAR is a doubtful thing.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

OSR_Community_User · February 9, 2009, 5:08pm

According to the PCI spec, “Devices that map control functions into I/O
Space may not consume more than 256 bytes per I/O base register.” Besides
that, even if the BIOS or OS allowed you to map this much, the bridges
don’t forward the last 768 bytes of each 1k unless the ISA Enable bit in
the Bridge Control register is cleared, and I have seen many BIOSes
actually set the bit.

Michael Jones
Sent by: xxxxx@lists.osr.com
02/09/2009 02:05 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
Re:[ntdev] EvtDevicePrepareHardware not getting called

The output from !arbiter was too big (the Lyris list manager rejected by
reply). I’ve stuck it on my web site (http://www.stonehill.com/output.txt
).

The file starts with the result of writing all ones to BAR1; it wants a
4KB chunk for I/O space (which agrees with the chip spec), but which I
won’t even use, since BAR0 provides access to the same chip registers in
memory. I’ve asked the BIOS guy if we can disable BAR1.

I don’t see anything obvious in the output from !arbiter, other than it
does not appear to have allocated any I/O space to the device, but I
admit that I don’t really know what to look for.

It seems to me 4KB is a lot of I/O space to ask for; perhaps the “stuck”
thread is not so much stuck as it is trying valiantly to find 4K of I/O
space by rejiggering everything else; I could imagine that taking some
time.

The fact that the device is a chip soldered on the main board might be
why the the PM cap is not set; I’ve asked the BIOS guy about that too.

Cheers,

–mkj

Calvin Guan wrote:
> Hi Michael,
>
> I can’t tell fer sure what went wrong. I’m suspecting the pci system is
running short of port io space. Can you post the output of “!arbiter 1”?
>
> I’d try to disable the IO BAR to confirm whether or not it’s indeed
causing the problem. I’d also confirm how much the card is actually asking
for, and what alignment requirement is being requested, i.e. blow all-one
to bar1 and then read it back to see what you got. You can do it with
windbag or using a dos program.
>
> Although it’s not related to the problem you are seeing, it’s weird that
the device doesn’t seem to support PM CAP. All pci devices should have
supported d0 and d3 at least.
>
> Happy hacking!
>
> –
> Calvin Guan
> Broadcom Corp.
> Connecting Everything(r)

–

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

David_J_Craig · February 9, 2009, 7:46pm

Who besides PS/2 and some ports for BIOS control still use PIO? Microsoft
seems to be pushing for everyone to use MMIO. It works much better and
faster. There is only the initial config reads and writes needed if the BAR
is not mapped until the driver loads.

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>A 64KB bar is not unusual and works well.

Memory - maybe, but IO space is limited to 64K total, so, 4K IO BAR is a
doubtful thing.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

Jake_Oshins · February 9, 2009, 8:50pm

If you don’t want your device to consume I/O space, even if there is an I/O
BAR, just say so in your INF.

See http://msdn.microsoft.com/en-us/library/aa906255.aspx. Look
specifically for “Needs=PciIoSpaceNotRequired”.

–
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Team

This post implies no warranties and confers no rights.

“Michael Jones” wrote in message
news:xxxxx@ntdev…
> The output from !arbiter was too big (the Lyris list manager rejected by
> reply). I’ve stuck it on my web site
> (http://www.stonehill.com/output.txt).
>
> The file starts with the result of writing all ones to BAR1; it wants a
> 4KB chunk for I/O space (which agrees with the chip spec), but which I
> won’t even use, since BAR0 provides access to the same chip registers in
> memory. I’ve asked the BIOS guy if we can disable BAR1.
>
> I don’t see anything obvious in the output from !arbiter, other than it
> does not appear to have allocated any I/O space to the device, but I admit
> that I don’t really know what to look for.
>
> It seems to me 4KB is a lot of I/O space to ask for; perhaps the “stuck”
> thread is not so much stuck as it is trying valiantly to find 4K of I/O
> space by rejiggering everything else; I could imagine that taking some
> time.
>
> The fact that the device is a chip soldered on the main board might be why
> the the PM cap is not set; I’ve asked the BIOS guy about that too.
>
> Cheers,
>
> --mkj
>
>
>
> Calvin Guan wrote:
>> Hi Michael,
>>
>> I can’t tell fer sure what went wrong. I’m suspecting the pci system is
>> running short of port io space. Can you post the output of “!arbiter 1”?
>> I’d try to disable the IO BAR to confirm whether or not it’s indeed
>> causing the problem. I’d also confirm how much the card is actually
>> asking for, and what alignment requirement is being requested, i.e. blow
>> all-one to bar1 and then read it back to see what you got. You can do it
>> with windbag or using a dos program. Although it’s not related to the
>> problem you are seeing, it’s weird that the device doesn’t seem to
>> support PM CAP. All pci devices should have supported d0 and d3 at least.
>> Happy hacking!
>>
>> –
>> Calvin Guan
>> Broadcom Corp.
>> Connecting Everything(r)
>
> –
>
> //
> // Michael K. Jones
> // Stone Hill Consulting, LLC
> // http://www.stonehill.com
> //
>

Tim_Roberts · February 9, 2009, 9:05pm

Jake Oshins wrote:

If you don’t want your device to consume I/O space, even if there is
an I/O BAR, just say so in your INF.

See http://msdn.microsoft.com/en-us/library/aa906255.aspx. Look
specifically for “Needs=PciIoSpaceNotRequired”.

Cool concept. For reference sake, I’ll point out that this only works
for Vista and later. At least, my XP copy of machine.inf doesn’t have
that section.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

OSR_Community_User · February 9, 2009, 11:25pm

Well, most BIOS boot drivers run in 16 bit real mode, and using memory
mapped I/O in an Expansion ROM BIOS driver is a real pain and subject to
all kinds of bugs in the System BIOS.

“David Craig”
Sent by: xxxxx@lists.osr.com
02/09/2009 07:46 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
Re:[ntdev] EvtDevicePrepareHardware not getting called

Who besides PS/2 and some ports for BIOS control still use PIO? Microsoft

seems to be pushing for everyone to use MMIO. It works much better and
faster. There is only the initial config reads and writes needed if the
BAR
is not mapped until the driver loads.

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>A 64KB bar is not unusual and works well.

Memory - maybe, but IO space is limited to 64K total, so, 4K IO BAR is a

doubtful thing.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

OSR_Community_User · February 9, 2009, 11:28pm

Good idea, but it only works in Vista and beyond.

“Jake Oshins”
Sent by: xxxxx@lists.osr.com
02/09/2009 08:50 PM
Please respond to
“Windows System Software Devs Interest List”

To
“Windows System Software Devs Interest List”
cc

Subject
Re:[ntdev] EvtDevicePrepareHardware not getting called

If you don’t want your device to consume I/O space, even if there is an
I/O
BAR, just say so in your INF.

See http://msdn.microsoft.com/en-us/library/aa906255.aspx. Look
specifically for “Needs=PciIoSpaceNotRequired”.

–
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Team

This post implies no warranties and confers no rights.

--------------------------------------------

“Michael Jones” wrote in message
news:xxxxx@ntdev…
> The output from !arbiter was too big (the Lyris list manager rejected by

> reply). I’ve stuck it on my web site
> (http://www.stonehill.com/output.txt).
>
> The file starts with the result of writing all ones to BAR1; it wants a
> 4KB chunk for I/O space (which agrees with the chip spec), but which I
> won’t even use, since BAR0 provides access to the same chip registers in

> memory. I’ve asked the BIOS guy if we can disable BAR1.
>
> I don’t see anything obvious in the output from !arbiter, other than it
> does not appear to have allocated any I/O space to the device, but I
admit
> that I don’t really know what to look for.
>
> It seems to me 4KB is a lot of I/O space to ask for; perhaps the “stuck”

> thread is not so much stuck as it is trying valiantly to find 4K of I/O
> space by rejiggering everything else; I could imagine that taking some
> time.
>
> The fact that the device is a chip soldered on the main board might be
why
> the the PM cap is not set; I’ve asked the BIOS guy about that too.
>
> Cheers,
>
> --mkj
>
>
>
> Calvin Guan wrote:
>> Hi Michael,
>>
>> I can’t tell fer sure what went wrong. I’m suspecting the pci system is

>> running short of port io space. Can you post the output of “!arbiter
1”?
>> I’d try to disable the IO BAR to confirm whether or not it’s indeed
>> causing the problem. I’d also confirm how much the card is actually
>> asking for, and what alignment requirement is being requested, i.e.
blow
>> all-one to bar1 and then read it back to see what you got. You can do
it
>> with windbag or using a dos program. Although it’s not related to the
>> problem you are seeing, it’s weird that the device doesn’t seem to
>> support PM CAP. All pci devices should have supported d0 and d3 at
least.
>> Happy hacking!
>>
>> –
>> Calvin Guan
>> Broadcom Corp.
>> Connecting Everything(r)
>
> –
>
> //
> // Michael K. Jones
> // Stone Hill Consulting, LLC
> // http://www.stonehill.com
> //
>

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Calvin_Guan-2 · February 10, 2009, 1:06am

Dude,

Run “!arbiter 1” or go to device manager&click “view resources by type”, see how much IO space is used by the system. 64K is the *entire* addressable IO space that a modern x86 architecture have. AT has only 10 address pins (1k) for I/O if I remembered right at the time I had to design stupid ISA cards for coursework at school.

4k of IO space is sizable. It could be worse if he has aggressive alignment requirement as I said.

Calvin Guan
Broadcom Corp.
Connecting Everything(r)

— On Mon, 2/9/09, David Craig wrote:

From: David Craig
Subject: Re:[ntdev] EvtDevicePrepareHardware not getting called
To: “Windows System Software Devs Interest List”
Date: Monday, February 9, 2009, 4:46 PM

Who besides PS/2 and some ports for BIOS control still use PIO?? Microsoft
seems to be pushing for everyone to use MMIO.? It works much better and
faster.? There is only the initial config reads and writes needed if the BAR
is not mapped until the driver loads.

“Maxim S. Shatskih” wrote in message
news:xxxxx@ntdev…
>A 64KB bar is not unusual and works well.

Memory - maybe, but IO space is limited to 64K total, so, 4K IO BAR is a
doubtful thing.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

—
NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Michael_Jones · February 10, 2009, 7:39am

Both the size and the alignment alignment are 4KB, so it really appears
that I am SOL. Is the stack I originally posted consistent with this
(i.e. the system searching for a hole, or trying to make one)?

The INF trick is nice, and I do have to support Vista, but I also have
to support XP. Any ideas there? Is there anything we can do in the
BIOS that would prevent Windows from trying to assign I/O space?
Alternatively, is there anything I can do in EvtDriverAddDevice?

The chip is what it is; it’s from a third party, so we have no hope of
changing it. It’s always going to have the two BARs, one memory and one
I/O. We can control which one comes first (there’s a pin that is
sampled on reset). It is currently wired so the memory BAR comes first,
but all we can do is switch it so the I/O bar comes first, and I’m
guessing that won’t help at all.

Cheers,

–mkj

Calvin Guan wrote:

Dude,

Run “!arbiter 1” or go to device manager&click “view resources by type”,
see how much IO space is used by the system. 64K is the *entire*
addressable IO space that a modern x86 architecture have. AT has only 10
address pins (1k) for I/O if I remembered right at the time I had to
design stupid ISA cards for coursework at school.

4k of IO space is sizable. It could be worse if he has aggressive
alignment requirement as I said.

Calvin Guan
Broadcom Corp.
Connecting Everything(r)

— On *Mon, 2/9/09, David Craig //* wrote:
>
>
> From: David Craig
> Subject: Re:[ntdev] EvtDevicePrepareHardware not getting called
> To: “Windows System Software Devs Interest List”
> Date: Monday, February 9, 2009, 4:46 PM
>
> Who besides PS/2 and some ports for BIOS control still use PIO?
> Microsoft
> seems to be pushing for everyone to use MMIO. It works much better and
> faster. There is only the initial config reads and writes needed if
> the BAR
> is not mapped until the driver loads.
>
> “Maxim S. Shatskih” > > wrote in message
> news:xxxxx@ntdev…
> >A 64KB bar is not unusual and works well.
>
> Memory - maybe, but IO space is limited to 64K total, so, 4K IO
> BAR is a
> doubtful thing.
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>

–

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//

Jake_Oshins · February 10, 2009, 11:25am

Yes, it was added with PCI Express support, as PCI Express tended to be so
deeply bridged that it’s difficult to guarantee that there will always be
available I/O space.

–
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Team

This post implies no warranties and confers no rights.

“Tim Roberts” wrote in message news:xxxxx@ntdev…
> Jake Oshins wrote:
>> If you don’t want your device to consume I/O space, even if there is
>> an I/O BAR, just say so in your INF.
>>
>> See http://msdn.microsoft.com/en-us/library/aa906255.aspx. Look
>> specifically for “Needs=PciIoSpaceNotRequired”.
>
> Cool concept. For reference sake, I’ll point out that this only works
> for Vista and later. At least, my XP copy of machine.inf doesn’t have
> that section.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>

Jake_Oshins · February 10, 2009, 11:34am

Yes. See EvtDeviceFilterRemoveResourceRequirements.

–
Jake Oshins
Hyper-V I/O Architect
Windows Kernel Team

This post implies no warranties and confers no rights.

“Michael Jones” wrote in message
news:xxxxx@ntdev…
> Both the size and the alignment alignment are 4KB, so it really appears
> that I am SOL. Is the stack I originally posted consistent with this
> (i.e. the system searching for a hole, or trying to make one)?
>
> The INF trick is nice, and I do have to support Vista, but I also have to
> support XP. Any ideas there? Is there anything we can do in the BIOS
> that would prevent Windows from trying to assign I/O space? Alternatively,
> is there anything I can do in EvtDriverAddDevice?
>
> The chip is what it is; it’s from a third party, so we have no hope of
> changing it. It’s always going to have the two BARs, one memory and one
> I/O. We can control which one comes first (there’s a pin that is sampled
> on reset). It is currently wired so the memory BAR comes first, but all
> we can do is switch it so the I/O bar comes first, and I’m guessing that
> won’t help at all.
>
> Cheers,
>
> --mkj
>
> Calvin Guan wrote:
>> Dude,
>>
>> Run “!arbiter 1” or go to device manager&click “view resources by type”,
>> see how much IO space is used by the system. 64K is the entire
>> addressable IO space that a modern x86 architecture have. AT has only 10
>> address pins (1k) for I/O if I remembered right at the time I had to
>> design stupid ISA cards for coursework at school.
>>
>> 4k of IO space is sizable. It could be worse if he has aggressive
>> alignment requirement as I said.
>>
>> Calvin Guan
>> Broadcom Corp.
>> Connecting Everything(r)
>>
>> — On Mon, 2/9/09, David Craig // wrote:
>>
>>
>> From: David Craig
>> Subject: Re:[ntdev] EvtDevicePrepareHardware not getting called
>> To: “Windows System Software Devs Interest List”
>>
>> Date: Monday, February 9, 2009, 4:46 PM
>>
>> Who besides PS/2 and some ports for BIOS control still use PIO?
>> Microsoft
>> seems to be pushing for everyone to use MMIO. It works much better
>> and
>> faster. There is only the initial config reads and writes needed if
>> the BAR
>> is not mapped until the driver loads.
>>
>> “Maxim S. Shatskih” >> > wrote in message
>> news:xxxxx@ntdev…
>> >A 64KB bar is not unusual and works well.
>>
>> Memory - maybe, but IO space is limited to 64K total, so, 4K IO
>> BAR is a
>> doubtful thing.
>>
>> –
>> Maxim S. Shatskih
>> Windows DDK MVP
>> xxxxx@storagecraft.com
>> http://www.storagecraft.com
>>
>>
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>>
>
> –
>
> //
> // Michael K. Jones
> // Stone Hill Consulting, LLC
> // http://www.stonehill.com
> //
>

OSR_Community_User · February 10, 2009, 11:58am

Michael,

Yes, the stack you posted is consistent with the system valiantly trying to satisfy a request for 4KB of I/O. And that request is basically guaranteed to fail. The reason is the continued existence of 10 bit decoding devices, which take a chunk out of every 4KB in the system. VGA is the main case for this, but I will spare you the details.

Jake is right that the I/O reduction feature in Vista was designed to handle this. Jake is also right that you *might* be able to do this by filtering resource requirements on XP. You can remove the I/O requirement (and the device private requirement that follows it) from the list entirely.

However, this is dangerous and might not work depending on your device. By removing the requirement from the list, you will cause XP to leave the I/O BAR untouched. It will continue to be programmed to the address it had at boot. XP will then set the I/O space enable bit, regardless of whether the device actually asked for any I/O. This means you will be decoding a random address.

You can’t change XP’s behavior to unconditionally set the I/O space enable bit. All you can do is tell XP what to program into your BAR. Is there an address that the BAR treats as “invalid”? 0 maybe? If programming a 0 into the BAR will cause the device to decode nothing (rather than decoding address 0), then you can filter a resource entry of 0 back into the list. This will cause XP to program the BAR to address 0. The decode will be enabled, but your device won’t be decoding anything, and everything will be happy.

That is a number of if’s, of course, which is precisely why we implemented richer support for this in Vista. In addition to the INF support, Vista will detect resource requirements that have been filtered to CmResourceTypeNull. If all resources of a type have been filtered in this way, Vista will leave the associated decode bit off.

Dave

Michael_Jones · February 10, 2009, 12:27pm

Should have R’d the FM. Thanks, Jake; this is just what I need!

Cheers,

–mkj

Jake Oshins wrote:

Yes. See EvtDeviceFilterRemoveResourceRequirements.

–

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

Michael_Jones · February 10, 2009, 12:48pm

Questions in-line…

xxxxx@microsoft.com wrote:

Yes, the stack you posted is consistent with the system valiantly
trying to satisfy a request for 4KB of I/O. And that request is
basically guaranteed to fail. The reason is the continued existence
of 10 bit decoding devices, which take a chunk out of every 4KB in
the system. VGA is the main case for this, but I will spare you the
details.

Although I don’t need to know the details, I am curious, so if it’s
possible to give a quick thumbnail, I’d be interested.

Jake is right that the I/O reduction feature in Vista was designed to
handle this. Jake is also right that you *might* be able to do this
by filtering resource requirements on XP. You can remove the I/O
requirement (and the device private requirement that follows it) from
the list entirely.

You mean filtering by using the the
EvtDeviceFilterRemoveResourceRequirements, right? Not sure what you
mean by also removing the “device private requirement that follows it”.

However, this is dangerous and might not work depending on your
device. By removing the requirement from the list, you will cause XP
to leave the I/O BAR untouched. It will continue to be programmed to
the address it had at boot. XP will then set the I/O space enable
bit, regardless of whether the device actually asked for any I/O.
This means you will be decoding a random address.

The BIOS sets the address, and I can (well, I can have someone else) set
the I/O BAR to whatever we want. However, we’re not going to use the
I/O bar ourselves, so that’s not a problem. But when you say “decoding
random address”, you mean if our I/O BAR is set to overlap some other
device’s I/O space (which it almost certainly does), then our device
will also decode it, and read/write one of it’s own registers at the
same time as the other device (the one that legitimately owns that
particular address), yes? That would obviously be a Bad Thing. A quick
scan of the datasheet doesn’t seem to say if there’s a value that is not
decoded, so I’ll have to ask the company.

You can’t change XP’s behavior to unconditionally set the I/O space
enable bit. All you can do is tell XP what to program into your BAR.
Is there an address that the BAR treats as “invalid”? 0 maybe? If
programming a 0 into the BAR will cause the device to decode nothing
(rather than decoding address 0), then you can filter a resource
entry of 0 back into the list. This will cause XP to program the BAR
to address 0. The decode will be enabled, but your device won’t be
decoding anything, and everything will be happy.

You’re also saying that even if we remove the I/O requirement from the
resource list, XP will still set the I/O enabled bit? The memory space
that the chip decodes actually covers the PCI config space. If I clear
the I/O enable bit after I map the memory region, does that do the
trick? Or is it more complicated than that?

That is a number of if’s, of course, which is precisely why we
implemented richer support for this in Vista. In addition to the INF
support, Vista will detect resource requirements that have been
filtered to CmResourceTypeNull. If all resources of a type have been
filtered in this way, Vista will leave the associated decode bit off.

Dave

Thanks a bunch,

– mkj

//
// Michael K. Jones
// Stone Hill Consulting, LLC
// http://www.stonehill.com
//_______________________________________________

OSR_Community_User · February 10, 2009, 1:18pm

Michael,

I have two specific responses to your issue, and then a long treatise on the history of I/O space that explains why you’re having to deal with this in gory detail.

First the specific responses:

if you use "!devnode

e" to look at the format of the resource requirements list that gets created for your device, you will notice that each BAR requirement is followed by one of CmResourceTypeDevicePrivate. This contains data associated with the requirement preceding it. They have to be removed as a pair.
2) re: your suggestion to modify the I/O enable bit manually - this is also bad. The PCI driver believes it owns this bit, and will set and clear it at the appropriate times. Your code would have to keep up with all of these times, and avoid synchronization issues that come from two drivers trying to manage the same bit in different ways. This is the kind of thing that would work until it doesn't. It would be much, much, much better if there were an un-decoding address that the BAR could be set to.

----

Now that's out of the way, here's the story with I/O space. Back in the ISA days, there were devices that did 10 bit decoding. This caused them to ignore the top 6 bits of an I/O address, and therefore would decode a range in every 0x400 bytes. VGA controllers were among the devices that did this. Then the PCI-PCI bridge spec came along, and decided to make the minimum granularity for a bridge's I/O window 0x1000 bytes. Any ISA device would be guaranteed to be in conflict with any bridge window location, as a result. To handle this situation, the ISA Enable bit in the Bridge Control register was invented. This bit should be named the "Make this bridge compatible with ISA devices bit." When set, it causes a bridge to only decode the first 0x100 bytes of every 0x400 byte range in its window. The reasoning was that the first 0x100 bytes were reserved for motherboard resources like the speaker port that did full 16 bit decoding. It was the other ranges used for ISA cards (like VGA controllers) that were the 10 bit decoding problems. So this limits the I/O space behind a bridge to 1/4 of its true available space, but avoids conflicts.

Then PCI video controllers started coming out. They needed to support legacy VGA ranges in addition to this fancy new BAR concept for decoding resources (for boot, crash, no driver present, etc). PCI video controllers needed to be placed behind PCI-PCI bridges in some cases, so bridges needed to be built to handle these legacy ranges. The VGA Enable bit in the Bridge Control register was born. This bit tells a bridge to decode legacy VGA resources (memory address 0xa0000, I/O ports 0x3b0-3bb, etc) in addition to its native BARs and resource windows. The spec says that when this bit is set, these ranges are to be decoded in 10 bit mode, meaning the bridge decodes 0x3b0-0x3bb *and all of its 10 bit aliases.* This was to handle the old 10 bit ISA VGA controllers that were being converted to PCI and still did some 10 bit decoding.

Eventually PCI video controllers were changes so that they did full 16 bit decoding of the legacy VGA ranges, but the definition of the PCI-PCI bridge VGA Enable bit remained. When AGP was invented, a video controller in every system was behind a PCI-PCI bridge, and so this bit was set on a bridge in every AGP system out there. As a result, there was a 10 bit decoding agent in every mainstream PC, even though ISA had gone away.

Finally, the PCI-PCI bridge spec was revised to have a 16 bit decoding version of the VGA Enable bit. This happened after XP shipped, so XP still has the 10 bit decoding behavior. This means that any other PCI-PCI bridges in the system must have the ISA Enable bit set in order to operate properly in the presence of an AGP bridge (or PCIe root port), limiting the available I/O space to 1/4 of the theoretical 64k. It also means that the effective maximum size for a single I/O BAR is 0x100 bytes. Anything beyond that is essentially guaranteed to fail.

Dave

Tim_Roberts · February 10, 2009, 1:18pm

Michael Jones wrote:

Questions in-line…

xxxxx@microsoft.com wrote:
> Yes, the stack you posted is consistent with the system valiantly
> trying to satisfy a request for 4KB of I/O. And that request is
> basically guaranteed to fail. The reason is the continued existence
> of 10 bit decoding devices, which take a chunk out of every 4KB in
> the system. VGA is the main case for this, but I will spare you the
> details.

Although I don’t need to know the details, I am curious, so if it’s
possible to give a quick thumbnail, I’d be interested.

The original IBM PC only used 10 bits for I/O port addressing, so you
could use from 0 to 3FF. Many chip designers grew lazy, and only
decoded the low-order 10 bits of the addresses, since the upper bits
didn’t matter.

Sadly, this carried forward into newer designs. Even after I/O port
address space was expanded to 16 bits, these chips still only decoded
the low-order 10 bits. So, a VGA card that was listening on port 03D0
would also trap cycles to port 07D0, 0BD0, 0FD0, 13D0, and so on. The
IBM 8514/A display adapter (which S3 cloned), among other devices,
actually took advantage of this, by piling all of its I/O ports on a
single 10-bit value.

Because there are STILL old devices today that do lazy 10-bit decoding,
a PCI BIOS cannot assign you, for example, the I/O port block from 3000
to 3FFF, because 33D0 (and 37D0 and 3BD0 and 3FD0) will be claimed by
that old VGA.

You’re also saying that even if we remove the I/O requirement from the
resource list, XP will still set the I/O enabled bit? The memory
space that the chip decodes actually covers the PCI config space. If
I clear the I/O enable bit after I map the memory region, does that do
the trick? Or is it more complicated than that?

If your device does not claim the cycles (as it probably won’t after you
clear the enable bit), then there should be no problem.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.