My Latin teacher in high school was a real pain when grading translations on exams. No credit would be given if you used the obvious English word for which the Latin word was the root. So, for example, if you translated the word, "mandatum" to "mandate", zero points. Had to be "order", "instruction" or something else with an equivalent meaning. You basically had to be a walking Latin to English thesaurus in order to make it out of the class alive. Much like Latin itself, this was of course amazingly annoying. However, some good did come from this suffering. For one, it made me realize that I was glad that Latin was a dead language, and for two I'm now overly cautious in my use of circular definitions.

So, why share this anecdote in an article about MDLs? Well, because I'm fairly certain that whoever wrote the DDK documentation for the MDL APIs never took my high school Latin class. Exhibit A will be the entry for MmMapLockedPagesSpecifyCache:

The MmMapLockedPagesSpecifyCache routine maps the physical pages that are described by an MDL, and allows the caller to specify the cache behavior of the mapped memory.

So, according to this, MmMapLockedPagesSpecifyCache maps the pages in the MDL and allows you to specify the cache type...Zero points.

Let's Try to Improve Those Definitions

Based on the number of questions I get about MDL issues, I'm guessing that I'm not the only one who finds the documentation in the DDK on this subject to be woefully inadequate. In this article I hope to demystify several of the common MDL operations such as probing, locking, and mapping. I will also provide insight into several of the common (and some not so common) APIs that transcends just turning the API name into a sentence.

First things first: In order to get anything out of this article you need to understand the basics of virtual memory. There is so much material available on this topic that I won't even attempt to describe the concepts of virtual memory here. If you don't know your ass from a kernel virtual address, read the background articles from past issues of The NT Insider  (Windows NT Virtual Memory Part I, Mar-Apr 1998; Windows NT Virtual Memory Part II, Jan-Feb 1999).

Since this is going to be an article about MDLs, let's start with my attempt at a reasonable definition of an MDL:

An MDL is a structure that describes the fixed physical memory locations that comprise a contiguous data buffer in virtual memory.

Here are a few significant facts you should take special notice of in this definition:

Each MDL can only describe a single virtually contiguous data buffer.

The data buffer that the MDL describes can be in either a kernel virtual address space, user virtual address space, or both.

The MDL describes the data buffer at a fixed position in physical memory. In other words, the data buffer an MDL describes will always be paged in (resident), and its pages will be locked-down ("pinned"). This means the data buffer can neither be paged out nor moved. These pages will remain locked for the lifetime of the MDL.

The data buffer that the MDL describes does not need to be page aligned, nor does it need to be an integral number of pages in length.

I know how to properly use the word "comprise" and have not confused it with the word "composed."

The Structure Itself

An article on MDLs wouldn't be complete without a cursory glance at the structure itself. Note that the MDL is documented to be fully opaque, which means you should never dereference its fields directly nor should you make any assumptions about its internal structure. However, as is true with much in the kernel, just because you should not rely on undocumented details in your drivers does not mean that you should not explore in order to help your debugging skills or overall understanding.

Here is the structure definition from the Server 2003 SP1 DDK. Above the structure definition is a great comment that describes a lot of the fields for us beautifully, so I'm just going to paste it in its entirety in Figure 1.

Realize that process context is important when calling this DDI on an MDL that describes a buffer within a user virtual address space. One of the purposes of this function is to bring in the pages described by the virtual address range passed to IoAllocateMdl and lock them into memory. This is done by translating the virtual addresses that make up the range into the individual PFN entries and referencing them. If this occurs in a process context other than the original, the page tables will be different and the pages brought in may not be what you expected.

MmProbeAndLockPages is one of a select few DDIs that throws an exception on error. As such, any calls to this function must be wrapped in a structured exception handling block.

Step 3: Mapping

This stage is potentially an optional stage. However, for now let's assume your driver wants a new virtual address for accessing the underlying pages.

At this point you have an MDL that is setup for the proper access and that has the physical pages that back it pinned into memory. The data buffer described by the MDL is only mapped into the address space (or spaces) in which it originally resided. If you want to map the buffer that the MDL describes into another virtual address range, use the MmMapLockedPagesSpecifyCache DDI:

PVOID

MmMapLockedPagesSpecifyCache (

     __in PMDL MemoryDescriptorList,

     __in KPROCESSOR_MODE AccessMode,

     __in MEMORY_CACHING_TYPE CacheType,

     __in_opt PVOID RequestedAddress,

     __in ULONG BugCheckOnFailure,

     __in MM_PAGE_PRIORITY Priority

     );

The most common use for this DDI is to map a data buffer into the non-process dependent part of the kernel virtual address space. This allows the data in the data buffer to be accessed by your driver in an arbitrary process context such as within the driver's DPC. To map the buffer described by the MDL into kernel virtual address space, specify KernelMode as the AccessMode parameter when you call this function.  No need to worry about page faults when you access the buffer, right?  It's physical pages are locked!

Alternatively, a particularly interesting way to use MmMapLockedPagesSpecifyCache is to map the buffer specified by an MDL into the context of a user process. Note that because the MDL describes a set of fixed physical pages, the MDL can be referred to in any process context. To map the data buffer described by the MDL into the user virtual address space of the current process, call MmMapLockedPagesSpecifyCache and specify UserMode as the AccessMode parameter.

Assuming you want a kernel virtual address for this MDL, (normal case), you can fight off carpel tunnel for a few more days by using the MmGetSystemAddressForMdlSafe macro shown in Figure 2.

 Figure 2 - MmGetSystemAddressForMdlSafe Macro

Practical Use

Imagine that your driver wants to read some data from the disk and you decided the best way to do this is by sending an IRP that you built. Since the disk driver is a METHOD_DIRECT driver, you need an MDL for the buffer. In this situation you know at least three things:

1. You're about to write new code so it will be awful and probably corrupt your disk. With much trepidation, you must trust that the mere fact that it compiled is a miracle and therefore it will work through divine intervention.
2. It's your driver, so you know where the memory came from and whether or not it is pageable.
3. You know the address is context independent.

In this case, you're about to put on your I/O manager hat and send another driver an MDL that describes the buffer. It is worth mentioning what state the I/O manager guarantees an MDL to be in before sending it to a METHOD_DIRECT driver.

Operation MDL Cleanup

So far this article has spent a lot of time talking about building MDLs for different situations, but has not addressed the issues of cleaning up after MDLs. This discussion has been left to the end of the article for a good reason. It seems to me that MDLs have this mystical air about them that is completely unwarranted. Once you understand what the DDIs used to build them actually do, a discussion about tearing them down should be fairly simple.

Let's look at the DDIs that have been used so far and list the "undo" DDIs for each operation.

IoAllocateMdl

Whenever you allocate an MDL with IoAllocateMdl, at some point you must call IoFreeMdl.

MmProbeAndLockPages

For every call to MmProbeAndLockPages, there must be a call to MmUnlockPages. When you think of MmProbeAndLockPages as taking out a reference to the underlying pages, then the idea that there must always be a matching MmUnlockPages to drop the reference makes sense.

MmBuildMdlForNonPagedPool

Because this DDI doesn't need to take out any references to the underlying pages in order to make them resident, no undo operation is required.

MmMapLockedPagesSpecifyCache

That's Enough...For Now

One of the great things about working in the kernel is that no matter what bit of information you choose to explore, there's always something more to learn. MDLs are no exception. Hopefully this article has led you to a better understanding of common operations that will result in far less painful development and debugging experiences in the future.

MDLs Sidebar