I’m curious if anyone can explain the reasoning behind a behavior that I’ve
noticed recently. I plan to do some more research on it, but would prefer
to save time if someone has a good explanation for what I’m seeing. Here’s
the synopsis:
When allocating memory at an explicit address in the context of a given
process, ZwAllocateVirtualMemory seems to actually allocate in 16 page
intervals instead of single page intervals. For example, let’s say that I
am calling ZwAllocateVirtualMemory in the context of a process and telling
it to allocate a region of memory that is one page in size at 0x00014000.
What I have witnessed when enumerating the process’ VAD after successful
return from ZwAllocateVirtualMemory is that the region that is actually
listed in the tree starts at 0x00010000 and ends at 0x00014000 (the end
includes the 0x00014000 page). These 5 pages are all committed, however,
the rest of the pages in the 16 page block (0x00015000 -> 0x0001f000) seem
to be marked as reserved (I’m not 100% sure about this, but it’s the only
explanation I’ve been able to come up with thus far). The reason I say they
appear to be reserved is because when I try to allocate memory at
0x00016000, for example, ZwAllocateVirtualMemory returns
STATUS_CONFLICTING_ADDRESSES, even though the region that would contain
0x00016000 (as it appears in the process’ VAD) is unoccupied. Please keep
in mind that this 16 page alignment is entirely transparent to the caller,
as the base address that is returned is simply that which was requested.
So, given what I have noted, why is it that ZwAllocateVirtualMemory chooses
to align allocations on a 16 page boundary instead of on single page
boundaries? Is it meant to conserve entries in the process’ VAD such that
it can be enumerated more quickly and thus reduce the maximum depth of the
tree? I can see that as being a valid reason, but it leads to annoyances
when trying to allocate memory in the upper region of the 16 page block that
was illustrated in the example above. Calling ZwAllocateVirtualMemory on
0x00016000 with MEM_COMMIT (and excluding MEM_RESERVE) causes
STATUS_CONFLICTING_ADDRESSES to be returned as well. Is this the expected
behavior, and, if so, how should one properly work around it?
The reason I’m wondering these things is because a driver that I’m
developing has the specific requirement of allocating memory along page
boundaries in the context of a given process. I *can* allocate in 16 page
boundaries (and doing this works perfectly fine), but would prefer to not
waste so much memory if at all possible. Given this requirement, the 16
page aligned behavior that has been exhibited by ZwAllocateVirtualMemory is
making things a little more annoying than I would have thought. Is there
perhaps a better way to do such granular allocations in the context of a
process from kernel mode? I’d prefer documented ways, if at all possible 
Matt
> These 5 pages are all committed, however,
the rest of the pages in the 16 page block (0x00015000 -> 0x0001f000) seem
to be marked as reserved (I’m not 100% sure about this, but it’s the only
explanation I’ve been able to come up with thus far). The reason
I say they
appear to be reserved is because when I try to allocate memory at
0x00016000, for example, ZwAllocateVirtualMemory returns
STATUS_CONFLICTING_ADDRESSES, even though the region that would contain
0x00016000 (as it appears in the process’ VAD) is unoccupied.
Following bad form and posting right away to my post, one point of
clarification. Another reason that I considered (which is probably the
correct answer) is that the reason it is returning
STATUS_CONFLICTING_ADDRESSES is because it is trying to align the 0x00016000
allocation to a 16 page boundary (0x00010000) which is indeed consumed in
the process’ VAD by the previous allocation. Thus it returns
STATUS_CONFLICTING_ADDRESSES. The question remains, though, why the 16 page
limitation, and how does one get around it to do more ganular allocations?
The short answer is because this is the way VAD reservation works.
Allocation of actual pages within that VA range can probably be done at
page granularity, but reserving the actual address space occurs in 64k
chunks (on x86 - it might be 128k on 8kb page systems).
Typically one doesn’t allocate memory within the address space of the
calling process. If you need page-aligned buffers you would generally
either require that the driver provide them, or allocate them out of
pool and copy from the user-buffers to the page-aligned pool buffer.
-p
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Matt Miller
Sent: Monday, September 13, 2004 2:46 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] ZwAllocateVirtualMemory seems to align
allocations on 16 page boundaries
I’m curious if anyone can explain the reasoning behind a
behavior that I’ve noticed recently. I plan to do some more
research on it, but would prefer to save time if someone has
a good explanation for what I’m seeing. Here’s the synopsis:
When allocating memory at an explicit address in the context
of a given process, ZwAllocateVirtualMemory seems to actually
allocate in 16 page intervals instead of single page
intervals. For example, let’s say that I am calling
ZwAllocateVirtualMemory in the context of a process and
telling it to allocate a region of memory that is one page in
size at 0x00014000.
What I have witnessed when enumerating the process’ VAD after
successful return from ZwAllocateVirtualMemory is that the
region that is actually listed in the tree starts at
0x00010000 and ends at 0x00014000 (the end includes the
0x00014000 page). These 5 pages are all committed, however,
the rest of the pages in the 16 page block (0x00015000 ->
0x0001f000) seem to be marked as reserved (I’m not 100% sure
about this, but it’s the only explanation I’ve been able to
come up with thus far). The reason I say they appear to be
reserved is because when I try to allocate memory at
0x00016000, for example, ZwAllocateVirtualMemory returns
STATUS_CONFLICTING_ADDRESSES, even though the region that
would contain 0x00016000 (as it appears in the process’ VAD)
is unoccupied. Please keep in mind that this 16 page
alignment is entirely transparent to the caller, as the base
address that is returned is simply that which was requested.
So, given what I have noted, why is it that
ZwAllocateVirtualMemory chooses to align allocations on a 16
page boundary instead of on single page boundaries? Is it
meant to conserve entries in the process’ VAD such that it
can be enumerated more quickly and thus reduce the maximum
depth of the tree? I can see that as being a valid reason,
but it leads to annoyances when trying to allocate memory in
the upper region of the 16 page block that was illustrated in
the example above. Calling ZwAllocateVirtualMemory on
0x00016000 with MEM_COMMIT (and excluding MEM_RESERVE) causes
STATUS_CONFLICTING_ADDRESSES to be returned as well. Is this
the expected behavior, and, if so, how should one properly
work around it?
The reason I’m wondering these things is because a driver
that I’m developing has the specific requirement of
allocating memory along page boundaries in the context of a
given process. I *can* allocate in 16 page boundaries (and
doing this works perfectly fine), but would prefer to not
waste so much memory if at all possible. Given this
requirement, the 16 page aligned behavior that has been
exhibited by ZwAllocateVirtualMemory is making things a
little more annoying than I would have thought. Is there
perhaps a better way to do such granular allocations in the
context of a process from kernel mode? I’d prefer documented
ways, if at all possible
Matt
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@windows.microsoft.com To unsubscribe send a blank
email to xxxxx@lists.osr.com
Thanks for the response Peter.
The short answer is because this is the way VAD reservation works.
Allocation of actual pages within that VA range can probably be done at
page granularity, but reserving the actual address space occurs in 64k
chunks (on x86 - it might be 128k on 8kb page systems).
Out of morbid curiosity, why is this? Is it a performance related issue (to
keep the max depth in a process’ VAD low)? Also, given this behavior, is
there no way make use of the memory at 0x00016000 after allocationg memory
at 0x00014000 (which implicitly creates a region starting at 0x00010000)
without explicitly allocating all 16 pages in the region? The answer seems
to be no, unfortunately. Perhaps there is a supported means to extend the
region?
Typically one doesn’t allocate memory within the address space of the
calling process. If you need page-aligned buffers you would generally
either require that the driver provide them, or allocate them out of
pool and copy from the user-buffers to the page-aligned pool buffer.
Indeed, unfortunately the driver I’m working on (as far as I know, correct
me if I’m wrong in my assumption) doesn’t have the luxury of being able to
do this as it must allocate regions at specific addresses in the context of
3rd party applications that are unaware of what is occurring (sounds ugly, I
know, but necessary for what I’m trying to accomplish). I suppose I could
potentially use a pagefile-backed section, but I think it would have much
the same problem as far as the VAD reservation is concerned. All told, it
just pains me to have to allocate 16 pages when in reality I only need one.
Matt
> From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Matt Miller
Sent: Monday, September 13, 2004 3:42 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ZwAllocateVirtualMemory seems to align
allocations on 16 page boundaries
Thanks for the response Peter.
> The short answer is because this is the way VAD reservation works.
> Allocation of actual pages within that VA range can
probably be done at
> page granularity, but reserving the actual address space
occurs in 64k
> chunks (on x86 - it might be 128k on 8kb page systems).
Out of morbid curiosity, why is this? Is it a performance
related issue (to
keep the max depth in a process’ VAD low)?
That is probably true. VADs are stored in a simple binary tree.
The tree is not balanced. Microsoft could replace the simple VAD binary tree
with
more advanced data structures and have page granularity for allocating
virtual addresses.
It seems that the number of apps/drivers that would benefit from that change
is extremely small.
Perhaps there is a supported means
to extend the region?
I really doubt that.
Dmitriy Budko, VMware
I think you should be able to commit within that range once it’s
reserved. From your mail what I see is that you can’t reserve the range
again, which I would expect since it’s already been reserved. If MM let
you re-reserve without any warning then it would be harder for multiple
disconnected components within a process to safely reserve virtual
memroy (they might think they’d managed to reserve some memory when in
fact it was already reserved by someone else for their use).
As to the design you’re working with. I hope you’re coding your driver
so that it doesn’t assume these ranges will remain allocated (the
application could release them or even reuse them out from under you)
and that you expect the application to change their contents. If you
don’t do what’s necessary to ensure this then you’re going to let an
arbitrary application that uses your driver potentially crash the
operating system.
-p
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Matt Miller
Sent: Monday, September 13, 2004 3:42 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ZwAllocateVirtualMemory seems to align
allocations on 16 page boundaries
Thanks for the response Peter.
> The short answer is because this is the way VAD reservation works.
> Allocation of actual pages within that VA range can
probably be done
> at page granularity, but reserving the actual address space
occurs in
> 64k chunks (on x86 - it might be 128k on 8kb page systems).
Out of morbid curiosity, why is this? Is it a performance
related issue (to keep the max depth in a process’ VAD low)?
Also, given this behavior, is there no way make use of the
memory at 0x00016000 after allocationg memory at 0x00014000
(which implicitly creates a region starting at 0x00010000)
without explicitly allocating all 16 pages in the region?
The answer seems to be no, unfortunately. Perhaps there is a
supported means to extend the region?
> Typically one doesn’t allocate memory within the address
space of the
> calling process. If you need page-aligned buffers you
would generally
> either require that the driver provide them, or allocate
them out of
> pool and copy from the user-buffers to the page-aligned pool buffer.
Indeed, unfortunately the driver I’m working on (as far as I
know, correct me if I’m wrong in my assumption) doesn’t have
the luxury of being able to do this as it must allocate
regions at specific addresses in the context of 3rd party
applications that are unaware of what is occurring (sounds
ugly, I know, but necessary for what I’m trying to
accomplish). I suppose I could potentially use a
pagefile-backed section, but I think it would have much the
same problem as far as the VAD reservation is concerned. All
told, it just pains me to have to allocate 16 pages when in
reality I only need one.
Matt
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as:
xxxxx@windows.microsoft.com To unsubscribe send a blank
email to xxxxx@lists.osr.com
> When allocating memory at an explicit address in the context of a given
process, ZwAllocateVirtualMemory seems to actually allocate in 16 page
This is documented, IIRC documented even in Win32, and this 64KB granularity is
SYSTEM_INFO::dwAllocationGranularity.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> I think you should be able to commit within that range once it’s
reserved. From your mail what I see is that you can’t reserve the range
again, which I would expect since it’s already been reserved. If MM let
you re-reserve without any warning then it would be harder for multiple
disconnected components within a process to safely reserve virtual
memroy (they might think they’d managed to reserve some memory when in
fact it was already reserved by someone else for their use).
I tried this (passing ZwAllocateVirtualMemory w/ just MEM_COMMIT for the
second allocation) and STATUS_CONFLICTING_ADDRESSES is still returned. I
think this has to do with what I alluded to in the previous E-mail, where
some chunk of code in ZwAllocateVirtualMemory 16 page aligns the address I’m
attempting to allocate and then walks the process’ VAD to see if there is a
range that conflicts with that region. In the example from before, the
first allocation at 0x00014000 is 16 page aligned to 0x00010000, thus an
entry is put into the process’ VAD that has the Start set to 0x00010000 and
the End set to 0x00014000. All 5 pages are marked as committed, and none
are left uncommitted (I passed MEM_COMMIT | MEM_RESERVE for this
allocation). The second allocation follows the same code path, but this
time it attempts to allocate memory at 0x00016000. The same code 16 page
aligns the address to 0x00010000 and walks the process’ VAD. At this point
the badness happens where it encounters a region that is already mapped at
that Start address (and, more importantly, overlaps with the region that is
to be allocated), and thus it returns STATUS_CONFLICTING_ADDRESSES. This
seems like the best explanation (I don’t have windbg logs handy to give a
better illustration of what I’m saying, so I hope it makes sense), and it’s
what leads me to believe that it’s not related to MEM_RESERVE in this
specific case. However, it’s always possible that I’m heading down the
wrong path, so set me straight if that seems like what’s happening
As to the design you’re working with. I hope you’re coding your driver
so that it doesn’t assume these ranges will remain allocated (the
application could release them or even reuse them out from under you)
and that you expect the application to change their contents.
I definitely agree in principal; however, given the context of what my
driver is trying to do (apologies for being vague…mum’s the word it
seems), I can assume that these regions will not be altered or deallocated
after I allocate them.
If you
don’t do what’s necessary to ensure this then you’re going to let an
arbitrary application that uses your driver potentially crash the
operating system.
This is definitely a concern in general, and I’m being careful to not allow
for scenarios where this could happen. We’ll see how it goes
Matt
I checked with the owner of the memory manager. He says that you will
be able to commit individual pages within that region if you align the
base address and length to multiples of 64k when you call
ZwAllocateVirtualMemory, rather than letting it realign the allocation
for you.
-p
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Wieland
Sent: Monday, September 13, 2004 4:22 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] ZwAllocateVirtualMemory seems to align
allocations on 16 page boundaries
I think you should be able to commit within that range once
it’s reserved. From your mail what I see is that you can’t
reserve the range again, which I would expect since it’s
already been reserved. If MM let you re-reserve without any
warning then it would be harder for multiple disconnected
components within a process to safely reserve virtual memroy
(they might think they’d managed to reserve some memory when
in fact it was already reserved by someone else for their use).
As to the design you’re working with. I hope you’re coding
your driver so that it doesn’t assume these ranges will
remain allocated (the application could release them or even
reuse them out from under you) and that you expect the
application to change their contents. If you don’t do what’s
necessary to ensure this then you’re going to let an
arbitrary application that uses your driver potentially crash
the operating system.
-p
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Matt Miller
> Sent: Monday, September 13, 2004 3:42 PM
> To: Windows System Software Devs Interest List
> Subject: RE: [ntdev] ZwAllocateVirtualMemory seems to align
> allocations on 16 page boundaries
>
> Thanks for the response Peter.
>
> > The short answer is because this is the way VAD reservation works.
> > Allocation of actual pages within that VA range can
> probably be done
> > at page granularity, but reserving the actual address space
> occurs in
> > 64k chunks (on x86 - it might be 128k on 8kb page systems).
>
> Out of morbid curiosity, why is this? Is it a performance related
> issue (to keep the max depth in a process’ VAD low)?
> Also, given this behavior, is there no way make use of the
memory at
> 0x00016000 after allocationg memory at 0x00014000 (which implicitly
> creates a region starting at 0x00010000) without explicitly
allocating
> all 16 pages in the region?
> The answer seems to be no, unfortunately. Perhaps there is a
> supported means to extend the region?
>
> > Typically one doesn’t allocate memory within the address
> space of the
> > calling process. If you need page-aligned buffers you
> would generally
> > either require that the driver provide them, or allocate
> them out of
> > pool and copy from the user-buffers to the page-aligned
pool buffer.
>
> Indeed, unfortunately the driver I’m working on (as far as I know,
> correct me if I’m wrong in my assumption) doesn’t have the
luxury of
> being able to do this as it must allocate regions at specific
> addresses in the context of 3rd party applications that are
unaware of
> what is occurring (sounds ugly, I know, but necessary for what I’m
> trying to accomplish). I suppose I could potentially use a
> pagefile-backed section, but I think it would have much the same
> problem as far as the VAD reservation is concerned. All
told, it just
> pains me to have to allocate 16 pages when in reality I
only need one.
>
> Matt
>
>
>
> —
> Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as:
> xxxxx@windows.microsoft.com To unsubscribe send a blank email to
> xxxxx@lists.osr.com
>
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
You are currently subscribed to ntdev as: unknown lmsubst tag
argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com