MiMappedPageWriter on ReadOnly Layered File system

NTFSD
1

My Layered FS driver do not expect any write to reach it, and it is currently using its user mode component to ensure that all it handles is read request. (For same reason this driver does not register for filter callbacks)

It was working fine with several application BUT recently with Word 2003, I saw a BSOD, call stack is generating from MiMappedPageWriter, which trying to write a modified page and for that calling FsRtlAcquireFileForModWriteEx which in turns is calling NTFS::NtfsAcquireFileForModWrite and it is crashing while trying to acquire PagingResource.

Now in my case; I shouldn’t have reached here but I did. Which means some one got write permission on the file; which was not expected by my driver.

So my question is how can I ensure that only Read only access is allowed on files opened by my driver. I am already calling

IoSetShareAccess( GENERIC_READ,
FILE_SHARE_READ,
FileObject,
&Fcb->ShareAccess );

while creating the FCB; but that does not seems to do the trick. So how can I make it a read only driver.

Thanks
Aditya

(There are no other driver installed on the stack, so some other driver grabbing the FO and using it for write is not a possibility, plus I have verified all calls to CreateFile, CreateFileMapping and MapViewOfFile in user mode at NT Layer, and am sure that they have not asked for write access at all)

2

I just took a look on FAT and it looks like that the onus lies with the driver completing create. So that should be my driver. (Correct me if I am wrong)

3

How repeatable is this? I’d start by trapping the offending create in the
CreateSection callback.

Also what OS? I’m always suspicious of the prefetcher (and whatever it was
called prior to Vista) and of course the LUA filter is also on the stack
before you add anything? What about MSAV (or whatever its called, MSE?).
That creates sections all over the place.

What was the file? And what was the stack? Although it seems unlikely
(because it does try hard not to) I am sure that I could imagine a situation
where NTFS borrows a fileObject…

Rod

4

Its 100% repeatable, the file is FName.dll, (at program files\common\MS Shared\Smart Tag)

oleaut32.dll API LoadXXXXAsTypeLib, open the file, mapping and than map it (all read only), (I debuged all the calls and it does not do anything funny at all, I checked the attributes of mapped page and it is indeed read only)

Later I got BSOD on exactly same FO.

Its XP SP3, as I said no other app is installed so MSAV is not in here, (I am not sure about LUA though)

>And what was the stack?
At the time of crash? Its MiMappedPageWriter passing call to -> FsRtlAcquireForModWrite -> NTFS::AcquiredForModWrite(Ex), I could see NTFS trying to acquire paging io resource and crashing (which I have not set; while creating FCB). I could register for the callback and can work around it.

But I shouldn’t be seeing this at first place.

Thanks for your time,

5

There is no LUA on XP.

So when you say “For same reason this driver does not register for filter callbacks” do you mean your driver ? If you have a driver that completes IRP_MJ_CREATE requests then it should definitely register for all filter callbacks so that the FILE_OBJECT doesn’t leak into the file system (like it looks it did).

Is this an image mapping or a file mapping ?

6

As Alex says, no LUA on XP, but there is a prefetcher of some sort. Also SR
has a habit of sending requests to the top of the stack which shouldn’t
matter but might…

should definitely register for all filter callbacks

To emphasize (and it is usually Alex that does that), This *must* include
the name providers…

R

7

>>do you mean your driver

Yes, my driver. I was thinking that as, it will never receive any paging request (on virtual files; which it intercepts); I’ll be safe without handling them. Though yeah I am thinking to add them now;

>Is this an image mapping or a file mapping ?

file mapping. (Oleaut32.dll call is opening this file as a data file, i.e. no sec_image)

Thanks

8

I’m not sure why it happens that you see the FsRtlAcquireForModWrite call. I don’t know if the contract is that you’re not supposed to see it at all or if you’re supposed to see it but ignore the writes if you see it for a read-only file (though it would make sense that the access is reflected in the page protection which would indicate that the access if propagated all the way to the view) …

Did you look at all IRP_MJ_CREATEs for that stream (see if it has hardlinks) ? A FILE_OBJECT can be used for any IO irrespective of the rights that were requested and granted for it. Basically once IRP_MJ_CREATE is completed in the file system, the FO becomes just a way for various OS components (Io, Mm, Cc) to identify the underlying stream so they will use whichever FO they have handy that points to the stream they need. Cc in particular will initialize caching on one FO and will keep a reference to it and operate on it regardless of how many other FOs are created for that stream.

Basically, are you sure that you didn’t miss any section creation that might have requested write for that SCB?

9

So I register for acquireformodwrite callback, got the one which was causing the crash; complete it with FLT_PREOP_COMPLETE and the application ran fine. Though to my surprise; I did not receive any write IRP after that. (I should; right?)

Is there some condition where the MiMappedPageWriter will ask the FS to acquire the lock to write modified mapped pages BUT do not write them?

10

received the write (came a bit late) :slight_smile:

11

Unexpected writes to read-only files can be caused by driver bugs. For example, when processing a paging read, if a driver builds its own MDL instead of using the one provided by the memory manager, it will cause pages to be marked dirty when the MDL is unlocked.

Checked kernels on win7 and later have asserts that should catch most of these problems. You can also set a write breakpoint on the PFN field which contains the “modified” bit and see where it gets set.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, September 29, 2011 10:33 AM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] MiMappedPageWriter on ReadOnly Layered File system

received the write (came a bit late) :slight_smile:

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

12

Awesome, Thank you Mr. Lebedinsky.

>For example, when processing a paging read, if a driver builds its own MDL instead of using
the one provided by the memory manager, it will cause pages to be marked dirty
when the MDL is unlocked.

Could it be possible by any chance in NtCreateSection Processing as well. I mean any code path which does it due to some condition caused by my driver. For instance I noticed that this NtCreateSection call asked file size from FS even though it is given in the FCB->Header.

> You can also set a write breakpoint on the PFN field which contains
the “modified” bit and see where it gets set.

I am sorry but I do not get it. Can you please elaborate on it.

13

I don’t immediately see how something could go wrong inside NtCreateSection such that file pages (which may not even be resident at this point) would later get modified.

Here’s how you can set the breakpoint I mentioned below. Note that this assumes that your pages are always getting modified, or at least you have some way to tell at the time you get a paging read request whether they will eventually be modified.

First, set a breakpoint (or just put a hardcoded DbgBreakPoint call) in your driver that fires when you get a paging read request (IRP_PAGING_IO). Then dump the MDL from the IRP:

21: kd> dt nt!_IRP 0xfffffa8aaeff6010 MdlAddress +0x008 MdlAddress : 0xfffffa8b12e797d0 _MDL

21: kd> dt /v nt!_MDL (tells you the size of the structure)
…

21: kd> dq 0xfffffa8b12e797d0 fffffa8b12e797d0 0000000000000000 0000000040420038
fffffa8b12e797e0 0000000000000000 fffff88000b53000 fffffa8b12e797f0 0000000000000000 0000000000001000 <- end of the MDL structure
fffffa8b12e79800 0000000008790b92

After the end of the MDL is the array of pointer-sized physical page frame numbers, or PFNs. In this case there is only one PFN because the MDL’s byte count is one page (0x1000). Pick one of the PFNs and use !pfn to find the address of the corresponding MMPFN structure:

21: kd> !pfn 00000000`08790b92
PFN 08790B92 at address FFFFFA8196B22B60
flink FFFFFA8B12E79730 blink / share count 00000000 pteaddress FFFFF8A002C5B8E0
reference count 0001 used entry count 0000 Cached color 1 Priority 4
restore pte FA8AD5C7552804A0 containing page 86112EF Standby PR
Shared ReadInProgress

Dump the MMPFN structure to find the offset of the Modified bit:

21: kd> dt nt!_MMPFN FFFFFA8196B22B60 -b
…
+0x018 u3 :
+0x002 e1 : _MMPFNENTRY
+0x000 Modified : 0y0

In this case, the offset is 0x1a (0x018+0x002). Set a write breakpoint on the byte that contains the modified bit for your PFN:

21: kd> ba w1 FFFFFA8196B22B60+0x018+0x002

21: kd> g

Now you should hit several breakpoints as adjacent PFN flags (occupying the same byte as the Modified bit) are changing as a result of page read processing. After every breakpoint, re-run the !pfn command above. If you see that the modified bit has been set, do a stack trace and it will tell you why the page became dirty.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, September 29, 2011 11:32 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] MiMappedPageWriter on ReadOnly Layered File system

Awesome, Thank you Mr. Lebedinsky.

>>For example, when processing a paging read, if a driver builds its own
>>MDL instead of using
the one provided by the memory manager, it will cause pages to be marked dirty when the MDL is unlocked.

Could it be possible by any chance in NtCreateSection Processing as well. I mean any code path which does it due to some condition caused by my driver. For instance I noticed that this NtCreateSection call asked file size from FS even though it is given in the FCB->Header.

>> You can also set a write breakpoint on the PFN field which contains
the “modified” bit and see where it gets set.

I am sorry but I do not get it. Can you please elaborate on it.

—
NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

14

>>After the end of the MDL is the array of pointer-sized physical page frame numbers, or PFNs

Alright, this is new (This is not documented anywhere, does it). I reached till MDL but didn’t know; how to reach PFN with that. This will solved that.

I will try this.

Thanks a lot for your time,

15

>>After the end of the MDL is the array of pointer-sized physical page frame

>numbers, or PFNs
Alright, this is new (This is not documented anywhere, does it).

It’s an implementation detail, so subject to change and all. However, it’s
been “documented” for a long time via the MmGetMdlPfnArray macro in the
wdm.h:

//++
// PPFN_NUMBER
// MmGetMdlPfnArray (
// __in PMDL Mdl
// )
//
// Routine Description:
//
// The MmGetMdlPfnArray routine returns the virtual address of the
// first element of the array of physical page numbers associated with
// the MDL.
//
// Arguments:
//
// Mdl - Pointer to an MDL.
//
// Return Value:
//
// Returns the virtual address of the first element of the array of
// physical page numbers associated with the MDL.
//
//–

#define MmGetMdlPfnArray(Mdl) ((PPFN_NUMBER)(Mdl + 1))

-scott

–
Scott Noone
Consulting Associate and Chief System Problem Analyst
OSR Open Systems Resources, Inc.
http://www.osronline.com

16

Actually the pfn arrray ptr are documented. It is right after the MDL structure. Well, at least it is in the File system internals book, iirc.

But the technique is brilliant to get that :slight_smile:

-pro

On Sep 30, 2011, at 3:07 AM, xxxxx@gmail.com wrote:

>> After the end of the MDL is the array of pointer-sized physical page frame numbers, or PFNs

Alright, this is new (This is not documented anywhere, does it). I reached till MDL but didn’t know; how to reach PFN with that. This will solved that.

I will try this.

Thanks a lot for your time,

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

17

got it, I always suspected this but was not sure. My driver create virtual local FO and fetch data from a network/local file. Now I never saw this problem when I read data from other local FS. but if the container file is at network.(inside workstation, sending request to VMWare HGFS network redirector) I see this error.

Now my read does something like this,

Read received for virtual FO from offset O of length L.
modify O, Verify L
Fetch the buffer address from MDL
call ZwReadFile on network file and pass, O, L and buffer

I put a breakpoint as mentioned by Pavel and this is how the stack looks; when modified bit is set.

b90615f8 80548c46 nt!MiDeferredUnlockPages+0xb6
b9061628 8054b49a nt!MiFreePoolPages+0xa4
b9061668 b9b42a8e nt!ExFreePoolWithTag+0x1ba
WARNING: Stack unwind information not available. Following frames may be wrong.
b9061678 b9b40b20 vmhgfs+0x9a8e
b9061688 b9b3fd37 vmhgfs+0x7b20
b9061694 b9b3e18b vmhgfs+0x6d37
b90616a4 b9b3e216 vmhgfs+0x518b
b90616bc b9b3e251 vmhgfs+0x5216
b90616c8 b9b39f7f vmhgfs+0x5251
b90616f4 b9b3a03f vmhgfs+0xf7f
b906173c 804ef18f vmhgfs+0x103f
b9061788 8057f982 nt!IopfCallDriver+0x31
b906179c 8057c9e7 nt!IopSynchronousServiceTail+0x70
b9061834 8054161c nt!NtReadFile+0x55d
b9061834 80500b89 nt!KiFastCallEntry+0xfc
b90618d0 b9375202 nt!ZwReadFile+0x11
b9061b60 ba6d4888 My!MyPreRead+0x4e2 [This is my driver]
b9061bc0 ba6d62a0 fltmgr!FltpPerformPreCallbacks+0x2d4
b9061bd4 ba6d6c48 fltmgr!FltpPassThroughInternal+0x32
b9061bf0 ba6d7059 fltmgr!FltpPassThrough+0x1c2

currently the pfn status is

!pfn 00020983
PFN 00020983 at address 81416A54
flink 89A601A8 blink / share count 00000000 pteaddress E1750B60
reference count 0002 Cached color 0
restore pte 89919590000004C0 containing page 00B5A3 Zeroed MPR
Modified Shared ReadInProgress

when I had received this read IRP in my driver

kd> !pfn 00020983
PFN 00020983 at address 81416A54
flink 89A601A8 blink / share count 00000000 pteaddress E1750B60
reference count 0001 Cached color 0
restore pte 89919590000004C0 containing page 00B5A3 Zeroed PR
Shared ReadInProgress

Now I will create a temp buffer and pass that in my network read file call; instead of passing original buffer and that should solve it hopefully.

But is it possible to shed some light on possibly what just happened here.

18

And to Mr. Pavel Lebedinsky,

This was a real nice thing to learn. how easy it made to investigate this.

Thanks a lot.

19

Yes, a temp buffer solved it. So it was an issue by the lower layer, perhaps it is doing some manipulation of MDL as stated above. (quite strange though;)

20

No, that was a bug in your driver. Using a temp buffer will avoid it, but the proper solution is to reuse the original MDL instead of calling ZwReadFile/FltReadFile (which will internally build a new MDL, causing the pages to get marked dirty on unlock).

Win8 has a new FltReadFileEx API which takes an additional MDL parameter which can be used for this. On other versions of Windows you can use FltPerformSynchronousIo and pass the address of the original MDL in FLT_CALLBACK_DATA->Iopb->Parameters.Read.MdlAddress when processing a paging read:

PFLT_CALLBACK_DATA cbd;
FltAllocateCallbackData(…, &cbd);

cbd->Iopb->MajorFunction = IRP_MJ_READ;
cbd->Iopb->MinorFunction = 0;

cbd->Iopb->Parameters.Read.MdlAddress = Irp->MdlAddress;

// initialize other Parameters.Read fields…

cbd->Iopb->IrpFlags |=
IRP_SYNCHRONOUS_API |
IRP_NOCACHE |
IRP_PAGING_IO |
IRP_SYNCHRONOUS_PAGING_IO;

FltPerformSynchronousIo(cbd);

cbd->Iopb->Parameters.Read.MdlAddress = NULL;

FltFreeCallbackData(cbd);

Setting MdlAddress to NULL before calling FltFreeCallbackData is necessary to prevent the filter manager from freeing the MDL (which would be incorrect in this case).

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Friday, September 30, 2011 10:01 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] MiMappedPageWriter on ReadOnly Layered File system

Yes, a temp buffer solved it. So it was an issue by the lower layer, perhaps it is doing some manipulation of MDL as stated above. (quite strange though;)

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer