My question comes from the standpoint of drivers for “common” hardware such as a SATA HBA versus something more specific (fill in the _________ with whatever). How would one determine if the SATA HBA is capable of accessing 64-bit regions of memory?
I’m trying to track down potential causes for a rather intermittent bug I’m seeing. I’ve just finished reading the article, “Driver Basics - DMA Concepts,” (a very good article by the way). This article tracks with what the author of this driver told me about the design. He told me that the user memory buffers are “pinned into kernel memory” and the DMA takes place from there. *If* the system is 64-bit capable (the SATA HBA, and the target drive, really), then it would seem that there is no reason to move/copy data through the “map registers” into regions the device can read from/write to.
After time, this driver returns that it could not allocate resources when doing DMA. I’m trying to rule out that memory fragmentation is limiting the size of individual transfers.
You read the data sheet and/or programmers guide and hope that the
manufacturer bothered to explain the dma model used.
Mark Roddy
On Tue, May 23, 2017 at 11:22 AM, wrote:
> Hi, > > My question comes from the standpoint of drivers for “common” hardware > such as a SATA HBA versus something more specific (fill in the _________ > with whatever). How would one determine if the SATA HBA is capable of > accessing 64-bit regions of memory? > > I’m trying to track down potential causes for a rather intermittent bug > I’m seeing. I’ve just finished reading the article, “Driver Basics - DMA > Concepts,” (a very good article by the way). This article tracks with what > the author of this driver told me about the design. He told me that the > user memory buffers are “pinned into kernel memory” and the DMA takes place > from there. If the system is 64-bit capable (the SATA HBA, and the > target drive, really), then it would seem that there is no reason to > move/copy data through the “map registers” into regions the device can read > from/write to. > > After time, this driver returns that it could not allocate resources when > doing DMA. I’m trying to rule out that memory fragmentation is limiting > the size of individual transfers. > > — > NTDEV is sponsored by OSR > > Visit the list online at: http:> showlists.cfm?list=ntdev> > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:></http:>
You’d have to tell us WHICH function is failing, and what the return status is.
Also, the advice to enable WDF Verifier and Driver Verifier is always good. Note that there is a DMA verifier mode that will cause all DMA operations to be intermediately buffered.
And, aside from that, look to see the state of pool when your driver fails. !poolused can be your friend here.
In general if the device accepts 64-bit DMA addresses, you can (probably) assume it’s 64-bit DMA capable.
If the device only accepts 32-bit DMA addresses, you can assume it’s only 32-bit DMA capable.
If the device accepts both then, as Mark pointed out, you read the data sheet or industry standard that describes the device’s registers. Something in there should tell you how you put the device in 64-bit mode (if it’s something your s/w can control) or how you determine which types of addresses the device supports (if it’s something controlled by the device).
There is no globally defined “I do 64-bit XYZ” bit.
-p
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Mark Roddy
Sent: Tuesday, May 23, 2017 8:39 AM
To: Windows System Software Devs Interest List Subject: Re: [ntdev] How does one determine if a device is capable of 64 bit DMA
You read the data sheet and/or programmers guide and hope that the manufacturer bothered to explain the dma model used.
Mark Roddy
On Tue, May 23, 2017 at 11:22 AM, > wrote: Hi,
My question comes from the standpoint of drivers for “common” hardware such as a SATA HBA versus something more specific (fill in the _________ with whatever). How would one determine if the SATA HBA is capable of accessing 64-bit regions of memory?
I’m trying to track down potential causes for a rather intermittent bug I’m seeing. I’ve just finished reading the article, “Driver Basics - DMA Concepts,” (a very good article by the way). This article tracks with what the author of this driver told me about the design. He told me that the user memory buffers are “pinned into kernel memory” and the DMA takes place from there. If the system is 64-bit capable (the SATA HBA, and the target drive, really), then it would seem that there is no reason to move/copy data through the “map registers” into regions the device can read from/write to.
After time, this driver returns that it could not allocate resources when doing DMA. I’m trying to rule out that memory fragmentation is limiting the size of individual transfers.
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at http:>
To unsubscribe, visit the List Server section of OSR Online at http:>
— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at</http:></http:></http:>
The call may fail because of iinvalid arameters, or insufficient memory. In the latter case, most likely of memory leak, which has nothing to do with 64-bitness. Unless somebody forgets to call PutScatterGatherList, and bounce buffers are leaking.
To add what has already been said, there also are some devices that are a mixture of 32/64 bit capable. For example, they might accept 64 physical addresses for a data buffer, but the address of a command/response ring is 32-bits. I’ve also seen devices that support some functionality with 64-bit addresses (like a command buffer), but only if all the 64-bit addresses for that functionality have the same top 32-bits. An example of the hardware interface that could cause this: the device has a command fifo where you write the physical address of the next command descriptor, except this fifo is only 32-bit wide, and you have a global upper 32-bit register that’s paired with it. To use this fifo, you have to set the upper 32-bits to a fixed value, and then you write to the fifo for each command. You could set the upper 32-bit register to 0 and act as if only 32-bit addresses are supported, or you could allocate a pool of command buffers above 4GB, all in the same 4GB range, and set the high 32-bits to this 4GB range.
It can take very careful reading of a chip interface spec to really understand the capabilities and limitations. Hardware also sometimes (often?) has bugs, and does not actually work like the docs say.
There are some tests a driver developer can do around 32/64 address. One can force all physical addresses to be 32-bits by how you set the DMA adapter initialization. You used to be able to force things to be 64-bit address by a boot option “nolowmem”, although https://msdn.microsoft.com/en-us/library/windows/hardware/ff542202(v=vs.85).aspx says that’s not supported in Windows 8 and later.
My question comes from the standpoint of drivers for “common” hardware such as a SATA HBA versus something more specific (fill in the _________ with whatever). How would one determine if the SATA HBA is capable of accessing 64-bit regions of memory?
I’m trying to track down potential causes for a rather intermittent bug I’m seeing. I’ve just finished reading the article, “Driver Basics - DMA Concepts,” (a very good article by the way). This article tracks with what the author of this driver told me about the design. He told me that the user memory buffers are “pinned into kernel memory” and the DMA takes place from there. If the system is 64-bit capable (the SATA HBA, and the target drive, really), then it would seem that there is no reason to move/copy data through the “map registers” into regions the device can read from/write to.
After time, this driver returns that it could not allocate resources when doing DMA. I’m trying to rule out that memory fragmentation is limiting the size of individual transfers.