Windows characterizes DMA device and driver architectures as being either "packet-based" or "common buffer." When a driver programs a device with the base and length of each buffer fragment in response to a transfer request from a user, and instructs the device to begin the transfer, that driver is implementing a "packet-based" DMA scheme. Contrast this with the approach in which the driver and the device share a (physically contiguous) buffer in host memory, referred to as a "common buffer." The layout and contents of this buffer are understood by both the driver the firmware running in the device. In a pure common buffer approach, the driver copies write data to the shared buffer area, typically to a series of ring buffers that also contain control information. To process this write request, the device transfers the data from that common buffer (and not from the original requestor's buffer) to the device. Reads are handled similarly, with the driver being responsible for moving the data received into the common buffer to the requestor's data buffer. One of the hallmarks of a common buffer design is that, after the shared data area has been established in main memory and the device has been apprised of its location, the device performs transfers to/from this buffer as needed, without any specific instruction being required from the driver. The device uses control information stored in the common buffer to know what to transfer and when.

More common these days is the hybrid packet-based and common buffer design. In this design, which is used by most network cards and many storage controllers, the driver allocates a (physically contiguous) block of host memory that it shares with the device, and that block of memory has a mutually agreed structure, just as in the common buffer approach. However, in a hybrid design, the driver does not need to move the data between the requestor's buffer and the common buffer as it would in a pure common buffer design. Rather, when it receives a request to perform a transfer operation, the driver places into the common buffer information that describes the base and length of each fragment of the requestor's buffer, just like it would program the device in a pure packet-based design. The device then performs the transfer directly in to or out of the requestor's data buffer using the information it reads from the common buffer.

Windows is designed to be an extraordinarily flexible operating system. In fact, Windows view of DMA is so flexible, that it doesn't presume much of anything about the underlying machine topology. But instead of forcing the DMA device driver to deal directly with all this flexibility and the details of the underlying machine topology directly, Windows nicely abstracts many of the details for the driver developer. Thus, when you write a driver for a DMA device on Windows, you write your driver not to the specific details of the underlying machine hardware, but rather you write your driver to match the non-changing Windows DMA abstraction.

The advantages of this approach are plain: When you write a Windows driver for a DMA device, you get to concentrate on the details of programming your device. You don't have to worry about the hardware details of the system that your driver might be running on, such as the physical path your data takes as it travels from your device to main memory. You don't end-up with conditionals in your code to handle various system hardware layouts.

You write your driver once. You write it to the Windows DMA abstraction. After that, your driver just needs to be compiled for each processor architecture (x86, AMD-64, and Itanic) and it'll work regardless of the underlying system hardware configuration. Sweet, huh?

What the Windows approach does require -- first and foremost -- is that before developing a driver for a DMA device, a developer very clearly understand the Windows DMA abstraction. After all, the DMA abstraction defines the environment in which your driver will run.

While it might not be immediately obvious, this part of the DMA abstraction has some very important and far-reaching implications for driver developers. The single most important of these is that under Windows, DMA is never performed directly using physical memory addresses. Not ever. Rather, when your driver programs your device with the base and length of each buffer fragment in preparation for a DMA transfer, the base address provided to your device must be a device bus logical address that you have received from the DMA subsystem. This means that it is a serious design error to determine the physical address of a user data buffer (such as by calling MmGetPhysicalAddress) and program your DMA device with that address. If you haven't gotten the address from the DMA subsystem, it's not a valid device bus logical address!

So, in Windows, the steps a driver will typically take to perform a packet-based DMA transfer are as follows:

Another interesting feature of the map register design is the fact that, in the Windows DMA abstraction, all devices can always reach all of physical memory. Because device bus logical addresses are "relocated" to physical memory addresses using map registers, your device that only supports 32-bit DMA operations will still work on systems with 4GB or more of physical memory. Cool, huh?

Architecturally, this is no different than how PAE works in memory management on an x86 system. Even though an application is limited to 32-bits of virtual address space, the memory management registers on a system supporting PAE can relocate that 32-bit address space to a larger physical address space.

Map registers can also be used to increase the throughput (and decrease the complexity) of drivers for DMA devices that do not support scatter/gather. If, as part of initialization, a driver indicates that its device does not have scatter/gather support, the DMA subsystem will try to allocate a block of map registers that are contiguous in device bus logical address space. If this contiguous map register allocation is successful, the driver will receive a single base device bus logical address with which the device can be programmed. This single base address will be sufficient to describe the entire transfer. This saves the driver from having to break the overall DMA transaction (for a buffer that spans multiple, non-contiguous, physical fragments) into multiple individual DMA transfers. Not only is this convenient, but it also increases device throughput and utilization.

Don't believe me? Well, how about if you write a driver for a device that only supports 32-bit DMA. Your customer buys this device and installs your driver knowing full-well that it only has 32-bit DMA support. Things work fine. The customer is happy: The device is relatively inexpensive, and it works just fine in his Windows server system with 2GB of main memory.

1. If you ignored the Windows DMA abstraction, and didn't worry about supporting address spaces greater than 32-bits, your device fails to operate properly. Worse, your customer's system ceases to work properly. Ooops! Hope he didn't throw away those old DIMMs.
2. If you ignored the Windows DMA abstraction, but being clever, you wrote code to handle 64-bit transfers manually by allocating an intermediate buffer, you're in luck! Well, SORT of. Up to this point, your driver hasn't actually been running any of that 64-bit handling, intermediate buffering, code. Hope you tested it really thoroughly, huh? Because if you didn't... yeah... same result as item 1, above.
3. If you USED the Windows DMA abstraction, your device continues to function without any problems. Problem solved, with no effort at all on your part.

Well, of course, how it's implemented depends on the underlying processor hardware (much hand waving and histrionics here). I mean, it is possible that some motherboard somewhere could, you know, maybe, possibly, implement map registers in hardware... as a direct physical translation of the Windows abstraction. There have been systems with hardware map registers in history. Both the PDP-11 and the VAX worked this way (surely a coincidence). In fact, there have even been a few systems that ran Windows NT that had hardware map registers.

The DMA subsystem will allocate an intermediate buffer, and use that buffer, when either (a) your driver indicates that your device does not support scatter/gather, or (b) the device your driver supports is not capable of directly accessing all fragments of the user's data buffer (because the device has only 32-bit DMA support and one or more of the buffer fragments are located at physical address 4GB or above). When such an intermediate buffer is used, your device winds-up transferring data to/from the intermediate buffer (the "map register", so to speak), and the Windows DMA subsystem is responsible for moving the data between that intermediate buffer and the actual requestor's data buffer. As we often say in class when we describe this, "this results in you getting all the cost and complexity of DMA, with all the CPU overhead of programmed I/O!"