Hi,
I am trying to debug a memory corruption issue, but I’m running into a
problem:
I have a block of memory that I am allocating in a driver with
MmAllocatePagesForMdl, then mapping into userspace with
MmMapLockedPagesSpecifyCache. I pass the user-mode address to an
application via an IoCtl and use the shared memory to pass messages between
the application and driver.
Everything has been working great, but now my memory is getting corrupted
occasionally. I think a piece of user-mode code is misbehaving and writing
into my shared memory, but I am trying to track down exactly what is doing
it. I tried using IBM Rational Purify, but it did not red-flag anything, so
I’m trying another method.
My plan is to set the PAGE_GUARD flag for my shared memory pages and latch a
structured exception handler to check for bad data and trigger a breakpoint
so I can see the call stack of the thread that is corrupting the memory.
It seems like this method should do the trick, but I am unable to set the
PAGE_GUARD flag for the shared memory. I have tried using VirtualProtect in
the user-mode code, but it returns ERROR_INVALID_ADDRESS, even though a call
to VirtualQuery with the same address succeeds and returns valid information
about the memory. I tried setting the flag in the driver using
MmProtectMdlSystemAddress, but it returns STATUS_INVALID_PAGE_PROTECTION
when I add the PAGE_GUARD flag. I also tried using MmSecureVirtualMemory,
but it just returns a NULL handle when I add the PAGE_GUARD flag.
Is there any way to set this flag from the driver? Or, why can’t I set the
flag from the user-mode application? Does it matter which memory caching
mode I specify in MmMapLockedPagesSpecifyCache?
I will also need to be able to re-set the PAGE_GUARD flag from within the
exception handler, because it is automatically disabled every time the
exception is raised.
Any ideas?
Thanks in advance,
~Leif
You should specify the same caching. Ideally, should be cached. There is no reason for using non-cached memory, except in very obscure circumstances.
Right. I don’t think the caching mode has anything to do with the
PAGE_GUARD flag. I guess I’m just looking for reasons why this isn’t
working.
On Fri, May 20, 2011 at 8:15 PM, wrote:
> You should specify the same caching. Ideally, should be cached. There is no
> reason for using non-cached memory, except in very obscure circumstances.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
There is no way to change protection of pages mapped to user space by MmMapLockedPagesSpecifyCache. These mappings are always RWX.
Thanks,
Pavel
Caching mode mismatch may cause your data corruption. But I heard that since Vista+ Windows will make sure a page is always mapped with the same mode. But if you unmap a page and map it again with different caching, that may be a bad idea.
Pavel, please confirm.
The fundamental rule (imposed by the hardware, not the OS) is that a given physical page cannot be mapped with two different caching attributes at the same time. Violating this rule results in undefined behavior. For example it can cause a fatal machine check on some CPUs.
Windows therefore has to ensure user space programs can’t break the rule. For example, cache attributes for shared sections can only be specified when the section is created (rather than when each view is mapped), and can’t be changed afterwards using VirtualProtect.
Drivers have a lot more flexibility when it comes to accessing physical memory so Windows can’t always ensure there will be no conflicts. The general rule here is that as long as you are operating on physical pages that have been properly locked down (which means you allocated them as non-pageable, or probe-and-locked them) you are safe. Most of the time the system will actually ignore the CacheType parameter you pass to MmMapLockedPagesSpecifyCache and automatically figure out the cache type appropriate for each page (see MmMapLockedPagesSpecifyCache docs for details). But if you start mapping arbitrary physical pages whose lifetime you have no control over (e.g. using MmMapIoSpace or Device\PhysicalMemory) then all bets are off.
Thanks,
Pavel
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@broadcom.com
Sent: Saturday, May 21, 2011 7:00 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Cannot set the PAGE_GUARD flag for memory mapped in kernel mode
Caching mode mismatch may cause your data corruption. But I heard that since Vista+ Windows will make sure a page is always mapped with the same mode. But if you unmap a page and map it again with different caching, that may be a bad idea.
Pavel, please confirm.
I suspect MmAllocatePagesForMdl have the same dangers as arbitrary physical pages mapping.
I’m not sure Windows will flush the CPU cache, if the page is unmapped as cached, and later is mapped as uncached. This may cause strange conditions.
> I suspect MmAllocatePagesForMdl have the same dangers as arbitrary physical pages mapping.
No. MmAllocatePagesForMdl is a supported API, so “may cause undefined behavior” can’t be part of its contract.
I’m not sure Windows will flush the CPU cache, if the page is unmapped as cached, and later is mapped as uncached. This may cause strange conditions.
Yes, Windows will flush the processor cache in this scenario. Without the flush, stale data from the cache could get dropped on top of more recent updates made through the non-cached mapping. And that would definitely cause some pretty strange conditions.
Thanks,
Pavel
Ah, that makes sense, thanks for the information. I guess my PAGE_GUARD
trick will not work because the memory is mapped to multiple locations. I
think I am going to try to debug the problem by using memory-access
breakpoints. This will take a little more work, but hopefully it will point
me to the source of the corruption.
Thanks again!
~Leif
On Sun, May 22, 2011 at 11:03 PM, Pavel Lebedynskiy wrote:
> > I suspect MmAllocatePagesForMdl have the same dangers as arbitrary
> physical pages mapping.
>
> No. MmAllocatePagesForMdl is a supported API, so “may cause undefined
> behavior” can’t be part of its contract.
>
> > I’m not sure Windows will flush the CPU cache, if the page is unmapped as
> cached, and later is mapped as uncached. This may cause strange conditions.
>
> Yes, Windows will flush the processor cache in this scenario. Without the
> flush, stale data from the cache could get dropped on top of more recent
> updates made through the non-cached mapping. And that would definitely cause
> some pretty strange conditions.
>
> Thanks,
> Pavel
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
If it is required that the driver allocates the memory and not the
application, what you can do here is create a system thread. In there
allocate the memory with PAGE_GUARD protection with ZwAllocateVirtualMemory
from within the system process context. Then create an MDL, probe and lock
the pages and map them into kernel space with MmMapLockedPagesSpecifyCache.
Then you have to map them another time to get a user VA for your
application. One advantage of charging the memory against the system
process is that it will stay around, and does not die your application can
terminate at any point and your driver will remain responsible for freeing
and cleaning everything up.
//Daniel
“Leif Ames” wrote in message news:xxxxx@ntdev…
> Hi,
>
> I am trying to debug a memory corruption issue, but I’m running into a
> problem:
>
> I have a block of memory that I am allocating in a driver with
> MmAllocatePagesForMdl, then mapping into userspace with
> MmMapLockedPagesSpecifyCache. I pass the user-mode address to an
> application via an IoCtl and use the shared memory to pass messages
> between
> the application and driver.
>
> Everything has been working great, but now my memory is getting corrupted
> occasionally. I think a piece of user-mode code is misbehaving and
> writing
> into my shared memory, but I am trying to track down exactly what is doing
> it. I tried using IBM Rational Purify, but it did not red-flag anything,
> so
> I’m trying another method.
>
> My plan is to set the PAGE_GUARD flag for my shared memory pages and latch
> a
> structured exception handler to check for bad data and trigger a
> breakpoint
> so I can see the call stack of the thread that is corrupting the memory.
>
> It seems like this method should do the trick, but I am unable to set the
> PAGE_GUARD flag for the shared memory. I have tried using VirtualProtect
> in
> the user-mode code, but it returns ERROR_INVALID_ADDRESS, even though a
> call
> to VirtualQuery with the same address succeeds and returns valid
> information
> about the memory. I tried setting the flag in the driver using
> MmProtectMdlSystemAddress, but it returns STATUS_INVALID_PAGE_PROTECTION
> when I add the PAGE_GUARD flag. I also tried using MmSecureVirtualMemory,
> but it just returns a NULL handle when I add the PAGE_GUARD flag.
>
> Is there any way to set this flag from the driver? Or, why can’t I set
> the
> flag from the user-mode application? Does it matter which memory caching
> mode I specify in MmMapLockedPagesSpecifyCache?
>
> I will also need to be able to re-set the PAGE_GUARD flag from within the
> exception handler, because it is automatically disabled every time the
> exception is raised.
>
> Any ideas?
>
> Thanks in advance,
> ~Leif
>
In addtion, I have been using this technique with success for sharing
exectuable memory between processes. Of course this technique works fine for
allocating memory but if page protection is to be modified on the fly
(memory is full, now don’t touch it anymore) which I suspect to be the case,
this is a bit more complicated because your driver needs to call
ZwProtectVirtualMemory which is not documnted from within a system thread. A
more easy scenario would still be that your application allocates the memory
and not the driver, then the application can change the page protection on
the fly. And of course every mapping. into a process needs to be unmapped on
application termination or you will get that infamous driver left locked
pages bugcheck.
//Daniel
wrote in message news:xxxxx@ntdev…
> If it is required that the driver allocates the memory and not the
> application, what you can do here is create a system thread. In there
> allocate the memory with PAGE_GUARD protection with
> ZwAllocateVirtualMemory from within the system process context. Then
> create an MDL, probe and lock the pages and map them into kernel space
> with MmMapLockedPagesSpecifyCache. Then you have to map them another time
> to get a user VA for your application. One advantage of charging the
> memory against the system process is that it will stay around, and does
> not die your application can terminate at any point and your driver will
> remain responsible for freeing and cleaning everything up.
>
> //Daniel
>
>
>
> “Leif Ames” wrote in message news:xxxxx@ntdev…
>> Hi,
>>
>> I am trying to debug a memory corruption issue, but I’m running into a
>> problem:
>>
>> I have a block of memory that I am allocating in a driver with
>> MmAllocatePagesForMdl, then mapping into userspace with
>> MmMapLockedPagesSpecifyCache. I pass the user-mode address to an
>> application via an IoCtl and use the shared memory to pass messages
>> between
>> the application and driver.
>>
>> Everything has been working great, but now my memory is getting corrupted
>> occasionally. I think a piece of user-mode code is misbehaving and
>> writing
>> into my shared memory, but I am trying to track down exactly what is
>> doing
>> it. I tried using IBM Rational Purify, but it did not red-flag anything,
>> so
>> I’m trying another method.
>>
>> My plan is to set the PAGE_GUARD flag for my shared memory pages and
>> latch a
>> structured exception handler to check for bad data and trigger a
>> breakpoint
>> so I can see the call stack of the thread that is corrupting the memory.
>>
>> It seems like this method should do the trick, but I am unable to set the
>> PAGE_GUARD flag for the shared memory. I have tried using VirtualProtect
>> in
>> the user-mode code, but it returns ERROR_INVALID_ADDRESS, even though a
>> call
>> to VirtualQuery with the same address succeeds and returns valid
>> information
>> about the memory. I tried setting the flag in the driver using
>> MmProtectMdlSystemAddress, but it returns STATUS_INVALID_PAGE_PROTECTION
>> when I add the PAGE_GUARD flag. I also tried using
>> MmSecureVirtualMemory,
>> but it just returns a NULL handle when I add the PAGE_GUARD flag.
>>
>> Is there any way to set this flag from the driver? Or, why can’t I set
>> the
>> flag from the user-mode application? Does it matter which memory caching
>> mode I specify in MmMapLockedPagesSpecifyCache?
>>
>> I will also need to be able to re-set the PAGE_GUARD flag from within the
>> exception handler, because it is automatically disabled every time the
>> exception is raised.
>>
>> Any ideas?
>>
>> Thanks in advance,
>> ~Leif
>>
>
>
>
Forget my replies. There is no advantage here to allocate by the system
process. I should show some restraint before dumping my irrelevant solutions
here, sorry for that.
//Daniel
>I will also need to be able to re-set the PAGE_GUARD flag from within the exception handler
That’s why sending IOCTLs to the driver is much better.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
>memory is mapped to multiple locations. I think I am going to try to debug the problem by using memory-access
breakpoints.
You can also allocate the memory in user mode and transfer it to the driver by sending it in an IOCTL pended in the driver.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
A-ha, that is a good idea! I’ll see if I can get it working that way.
~Leif
On Tue, May 24, 2011 at 9:29 AM, Maxim S. Shatskih
wrote:
> >memory is mapped to multiple locations. I think I am going to try to
> debug the problem by using memory-access
> >breakpoints.
>
> You can also allocate the memory in user mode and transfer it to the driver
> by sending it in an IOCTL pended in the driver.
>
> –
> Maxim S. Shatskih
> Windows DDK MVP
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>