Driver Problems? Questions? Issues?
Put OSR's experience to work for you! Contact us for assistance with:
  • Creating the right design for your requirements
  • Reviewing your existing driver code
  • Analyzing driver reliability/performance issues
  • Custom training mixed with consulting and focused directly on your specific areas of interest/concern.
Check us out. OSR, the Windows driver experts.

On-Access, Transparent, Per-File Data Encryption:

OSR's File Encryption Solution Framework (FESF) provides all the infrastructure you need to build a transparent file encryption product REALLY FAST.

Super flexible policy determination and customization, all done in user-mode. Extensive starter/sample code provided.

Proven, robust, flexible. In use in multiple commercial products.

Currently available on Windows. FESF for Linux will ship in 2018.

For more info: https://www.osr.com/fesf

Go Back   OSR Online Lists > ntfsd
Welcome, Guest
You must login to post to this list
  Message 1 of 9  
08 Nov 17 10:03
John
xxxxxx@gmail.com
Join Date: 10 May 2014
Posts To This List: 35
Separate cache for images

Is there a separate cache for loaded images? In my isolation filter, I noticed that non-Windows images when loading into a process will not trigger a IRP_MJ_READ more than once. I still see reads for Windows system files (ntdll, kernel32) but the non Windows don't seem to cause a read even though I can see the image load during the callback. So either my isolation filter is not working properly or there there is a separate cache. And if there is a separate cache, is there a way to force a read to occur? That's why I wrote an isolation filter to begin with.
  Message 2 of 9  
08 Nov 17 10:59
rod widdowson
xxxxxx@steadingsoftware.com
Join Date: 11 Sep 2006
Posts To This List: 831
Separate cache for images

> Is there a separate cache for loaded images? As in executable images? Yes. Check out the FAT sources and look for MmFlushImageSection
  Message 3 of 9  
08 Nov 17 12:43
Slava Imameev
xxxxxx@hotmail.com
Join Date: 13 Sep 2013
Posts To This List: 264
Separate cache for images

<QUOTE> Is there a separate cache for loaded images? </QUOTE> The system keeps physical pages for an executable image section in the active or standby lists even if no process maps the image file. Pages are kept resident with valid data until the number of free pages in the system drops below some threshold and the Memory Manager reclaims pages. When a CPU accesses a virtual address range backed by an executable image section a page fault handler doesn't call a file system to read page from a file if a related prototype page table entry has a valid page reference. The page fault handler establishes a virtual to physical mapping for this physical page and increments a shared count for the physical page. That said, you might not see read requests because pages are in the active or standby lists as a result of access in the past OR because an executable image code has never been executed or accessed. Mapping an executable image doesn't equate to its execution. <QUOTE> And if there is a separate cache, is there a way to force a read to occur? </QUOTE> This requires invalidating all virtual to physical mappings in all processes that map this page and this functionality is not exported. In normal operating mode this happens when the Working Set Manager trims process working set in response to the free pages pool/list depletion. Drivers can use MmFlushImageSection which invalidates prototype page table entries if there is no mapped view for an image section.
  Message 4 of 9  
08 Nov 17 14:14
John
xxxxxx@gmail.com
Join Date: 10 May 2014
Posts To This List: 35
Separate cache for images

Thanks Rod and Slava. I was already using MmFlushImageSection in CREATE and SET_INFO (delete and rename) but wasn't using it in CLEANUP like the docs mention. So I added it there however, whenever I call it from CLEANUP it always returns FALSE for the non system files. Some system files return TRUE while others also always return FALSE. In CLEANUP, should MmFlush be called before or after Cc functions or does it not matter? I get the same results either way but just curious on the proper flow of things. And do all handles have to be closed when calling MmFlush or will the memory manager mark it as flushable and do its thing once the ref count reaches zero? So when MmFlush fails, that means there is still a mapped view of the section. I find it unusual that ntdll will return TRUE but every process running will have a mapped view of that image. Any thoughts on where a reference is being missed (I'm assuming it's a reference thing) for Mm to think the image is still mapped?
  Message 5 of 9  
08 Nov 17 15:39
Slava Imameev
xxxxxx@hotmail.com
Join Date: 13 Sep 2013
Posts To This List: 264
Separate cache for images

<QUOTE> In CLEANUP, should MmFlush be called before or after Cc functions or does it not matter? </QUOTE> Before. It is safer. <QUOTE> And do all handles have to be closed when calling MmFlush or will the memory manager mark it as flushable and do its thing once the ref count reaches zero? </QUOTE> File handles are not related to section control area in this way. MmFlushImageSection checks for section control area references( i.e. checks for NtCreateSection) and mapped views ( i.e. NtMapViewOfSection). <QUOTE> I find it unusual that ntdll will return TRUE but every process running will have a mapped view of that image. </QUOTE> Below is a WinDBG output for ntdll.dll control area for Win10 16299.15.amd64fre.rs3_release.170928-1534 . The number of mapped views and user references are not zero. I don't have a reasonable explanation for MmFlushImageSection returning TRUE for this control area. 3: kd> !ca 0xffffb70b`8eb78790 ControlArea @ ffffb70b8eb78790 Segment ffffd983a2b485a0 Flink ffffb70b8ff86110 Blink ffffb70b8dbfb470 Section Ref 2 Pfn Ref 169 Mapped Views 3d User Ref 3f WaitForDel 0 Flush Count 8a40 File Object ffffb70b8e7ccb70 ModWriteCount 0 System Views 7264 WritableRefs c0001e Flags (a0) Image File \Windows\System32\ntdll.dll Segment @ ffffd983a2b485a0 ControlArea ffffb70b8eb78790 BasedAddress 00007ff98d7c0000 Total Ptes 1e0 Segment Size 1e0000 Committed 0 Image Commit c Image Info ffffd983a2b485e8 ProtoPtes ffffd983a2b79010 Flags (c2822000) DebugSymbolsLoaded ProtectionMask Subsection 1 @ ffffb70b8eb78810 ControlArea ffffb70b8eb78790 Starting Sector 0 Number Of Sectors 2 Base Pte ffffd983a2b79010 Ptes In Subsect 1 Unused Ptes 0 Flags 2 Sector Offset 0 Protection 1 Subsection 2 @ ffffb70b8eb78848 ControlArea ffffb70b8eb78790 Starting Sector 2 Number Of Sectors 887 Base Pte ffffd983a2b79018 Ptes In Subsect 111 Unused Ptes 0 Flags 6 Sector Offset 0 Protection 3 Subsection 3 @ ffffb70b8eb78880 ControlArea ffffb70b8eb78790 Starting Sector 889 Number Of Sectors 1 Base Pte ffffd983a2b798a0 Ptes In Subsect 1 Unused Ptes 0 Flags 6 Sector Offset 0 Protection 3 Subsection 4 @ ffffb70b8eb788b8 ControlArea ffffb70b8eb78790 Starting Sector 88a Number Of Sectors 22c Base Pte ffffd983a2b798a8 Ptes In Subsect 46 Unused Ptes 0 Flags 2 Sector Offset 0 Protection 1 Subsection 5 @ ffffb70b8eb788f0 ControlArea ffffb70b8eb78790 Starting Sector ab6 Number Of Sectors 20 Base Pte ffffd983a2b79ad8 Ptes In Subsect 8 Unused Ptes 0 Flags a Sector Offset 0 Protection 5 Subsection 6 @ ffffb70b8eb78928 ControlArea ffffb70b8eb78790 Starting Sector ad6 Number Of Sectors 70 Base Pte ffffd983a2b79b18 Ptes In Subsect e Unused Ptes 0 Flags 2 Sector Offset 0 Protection 1 Subsection 7 @ ffffb70b8eb78960 ControlArea ffffb70b8eb78790 Starting Sector b46 Number Of Sectors 1b Base Pte ffffd983a2b79b88 Ptes In Subsect 4 Unused Ptes 0 Flags a Sector Offset 0 Protection 5 Subsection 8 @ ffffb70b8eb78998 ControlArea ffffb70b8eb78790 Starting Sector b61 Number Of Sectors 1 Base Pte ffffd983a2b79ba8 Ptes In Subsect 1 Unused Ptes 0 Flags 2 Sector Offset 0 Protection 1 Subsection 9 @ ffffb70b8eb789d0 ControlArea ffffb70b8eb78790 Starting Sector b62 Number Of Sectors 351 Base Pte ffffd983a2b79bb0 Ptes In Subsect 6b Unused Ptes 0 Flags 2 Sector Offset 0 Protection 1 Subsection 10 @ ffffb70b8eb78a08 ControlArea ffffb70b8eb78790 Starting Sector eb3 Number Of Sectors 3 Base Pte ffffd983a2b79f08 Ptes In Subsect 1 Unused Ptes 0 Flags 2 Sector Offset 0 Protection 1
  Message 6 of 9  
15 Nov 17 01:42
John
xxxxxx@gmail.com
Join Date: 10 May 2014
Posts To This List: 35
Separate cache for images

I've tried calling MmFlushImageSection in my driver to no avail. It doesn't seem to have any effect on the image loading behavior. <Quote>Pages are kept resident with valid data until the number of free pages in the system drops below some threshold and the Memory Manager reclaims pages. </Quote> Is there a way to lower this threshold or temporarily cross it so it causes a purge? <Quote>This requires invalidating all virtual to physical mappings in all processes that map this page and this functionality is not exported. In normal operating mode this happens when the Working Set Manager trims process working set in response to the free pages pool/list depletion.</Quote> Again, any way to force this condition through other means? <Quote>That said, you might not see read requests because pages are in the active or standby lists as a result of access in the past OR because an executable image code has never been executed or accessed.</Quote> Shouldn't there be at least one read initially? I wrote a test case where the parent app copies an existing dll to a new directory. In that directory there's another application that will call LoadLibrary on the new dll (different filename than the original). However as I monitor in ProcMon, there are no IRP_MJ_READ requests on the new dll. It gets written, ACQUIRED_SECTION_SYNC is called a few times and then the load image callback is called. The output is below. Looking at FastFat, there are certain conditions where it calls MmFlushImageSection (and I assume it's similar with NTFS) and I tried to recreate those situations in my user mode app (you'll see a SetDispositionInformationFile toggle on the new file in the ProcMon output below). Still doesn't seem to help. I guess I'll start from the beginning so maybe someone smarter than me can suggest something. I have a requirement where a DLL gets injected into certain applications. This DLL has some place holders in a section that needs to get filled in before execution begins. That is something I can't change. How I modify the placeholders is under my control. The easiest would be if I could just modify the data during PsSetLoadImageNotifyRoutine but page protections can't be changed because of the address space lock. Modifying on read seemed like a good option, however with the image caching that doesn't seem to play by normal rules (at least in my Win7 VM) I'm not sure if that's a viable solution either. It doesn't have to be done in the kernel as I can pass the state information to user-mode but there may be an issue with execution (can I modify in user-mode before DllAttach is called?) Any thoughts on the best way to accomplish this would be helpful. Parent.exe 3056 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Generic Write, Read Data/List Directory, Read Attributes, Delete, Disposition: Create, Options: Sequential Access, Synchronous IO Non-Alert, Non-Directory File, Attributes: ANCI, ShareMode: None, AllocationSize: 0, Impersonating: NT AUTHORITY\SYSTEM, OpenResult: Created Parent.exe 3056 IRP_MJ_QUERY_VOLUME_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryAttributeInformationVolume, FileSystemAttributes: Case Preserved, Case Sensitive, Unicode, ACLs, Compression, Named Streams, EFS, Object IDs, Reparse Points, Sparse Files, Quotas, Transactions, 0x3c00000, MaximumComponentNameLength: 255, FileSystemName: NTFS Parent.exe 3056 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryBasicInformationFile, CreationTime: 11/14/2017 8:24:17 PM, LastAccessTime: 11/14/2017 8:24:17 PM, LastWriteTime: 11/14/2017 8:24:17 PM, ChangeTime: 11/14/2017 8:24:17 PM, FileAttributes: A Parent.exe 3056 IRP_MJ_SET_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: SetEndOfFileInformationFile, EndOfFile: 42,760 Parent.exe 3056 IRP_MJ_WRITE C:\Program Files\Test\temp\dll32.dll SUCCESS Offset: 0, Length: 42,760, Priority: Normal Parent.exe 3056 IRP_MJ_SET_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: SetBasicInformationFile, CreationTime: 0, LastAccessTime: 0, LastWriteTime: 11/8/2017 12:38:09 PM, ChangeTime: 11/14/2017 8:23:48 PM, FileAttributes: n/a Parent.exe 3056 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Parent.exe 3056 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Generic Write, Read Attributes, Delete, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: N, ShareMode: None, AllocationSize: n/a, Impersonating: NT AUTHORITY\SYSTEM, OpenResult: Opened Parent.exe 3056 IRP_MJ_SET_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: SetDispositionInformationFile, Delete: True Parent.exe 3056 IRP_MJ_SET_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: SetDispositionInformationFile, Delete: False Parent.exe 3056 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Parent.exe 3056 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS System 4 IRP_MJ_WRITE C:\Program Files\Test\temp\dll32.dll SUCCESS Offset: 0, Length: 45,056, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal System 4 IRP_MJ_SET_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: SetEndOfFileInformationFile, EndOfFile: 42,760 System 4 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS SyncType: SyncTypeOther System 4 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 FASTIO_NETWORK_QUERY_OPEN C:\Program Files\Test\temp\dll32.dll FAST IO DISALLOWED Test.exe 3112 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Read Attributes, Disposition: Open, Options: Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened Test.exe 3112 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryBasicInformationFile, CreationTime: 11/14/2017 8:24:17 PM, LastAccessTime: 11/14/2017 8:24:17 PM, LastWriteTime: 11/8/2017 12:38:09 PM, ChangeTime: 11/14/2017 8:23:48 PM, FileAttributes: A Test.exe 3112 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Read Data/List Directory, Execute/Traverse, Synchronize, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Delete, AllocationSize: n/a, OpenResult: Opened Test.exe 3112 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll FILE LOCKED WITH ONLY READERS SyncType: SyncTypeCreateSection, PageProtection: PAGE_EXECUTE Test.exe 3112 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryStandardInformationFile, AllocationSize: 45,056, EndOfFile: 42,760, NumberOfLinks: 1, DeletePending: False, Directory: False Test.exe 3112 FASTIO_ACQUIRE_FOR_CC_FLUSH C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 FASTIO_RELEASE_FOR_CC_FLUSH C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS SyncType: SyncTypeOther Test.exe 3112 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryNameInformationFile, Name: \Program Files\Test\temp\dll32.dll Test.exe 3112 Load Image C:\Program Files\Test\temp\dll32.dll SUCCESS Image Base: 0x6b6b0000, Image Size: 0xf000 Test.exe 3112 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, Delete, AllocationSize: n/a, OpenResult: Opened Test.exe 3112 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryBasicInformationFile, CreationTime: 11/14/2017 8:24:17 PM, LastAccessTime: 11/14/2017 8:24:17 PM, LastWriteTime: 11/8/2017 12:38:09 PM, ChangeTime: 11/14/2017 8:23:48 PM, FileAttributes: A Test.exe 3112 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, AllocationSize: n/a, OpenResult: Opened Procmon.exe 2744 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryStandardInformationFile, AllocationSize: 45,056, EndOfFile: 42,760, NumberOfLinks: 1, DeletePending: False, Directory: False Procmon.exe 2744 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll FILE LOCKED WITH ONLY READERS SyncType: SyncTypeCreateSection, PageProtection: PAGE_READONLY Procmon.exe 2744 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryStandardInformationFile, AllocationSize: 45,056, EndOfFile: 42,760, NumberOfLinks: 1, DeletePending: False, Directory: False Procmon.exe 2744 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS SyncType: SyncTypeOther Procmon.exe 2744 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a, ShareMode: Read, AllocationSize: n/a, OpenResult: Opened Procmon.exe 2744 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryStandardInformationFile, AllocationSize: 45,056, EndOfFile: 42,760, NumberOfLinks: 1, DeletePending: False, Directory: False Procmon.exe 2744 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll FILE LOCKED WITH ONLY READERS SyncType: SyncTypeCreateSection, PageProtection: PAGE_READONLY Procmon.exe 2744 FASTIO_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryStandardInformationFile, AllocationSize: 45,056, EndOfFile: 42,760, NumberOfLinks: 1, DeletePending: False, Directory: False Procmon.exe 2744 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS SyncType: SyncTypeOther Procmon.exe 2744 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Procmon.exe 2744 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS Test.exe 3112 IRP_MJ_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryNameInformationFile, Name: \Program Files\Test\temp\dll32.dll Parent.exe 3056 IRP_MJ_CREATE C:\Program Files\Test\temp\dll32.dll SUCCESS Desired Access: Read Attributes, Delete, Disposition: Open, Options: Non-Directory File, Open Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened Parent.exe 3056 IRP_MJ_QUERY_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: QueryAttributeTagFile, Attributes: A, ReparseTag: 0x0 Parent.exe 3056 IRP_MJ_SET_INFORMATION C:\Program Files\Test\temp\dll32.dll SUCCESS Type: SetDispositionInformationFile, Delete: True Parent.exe 3056 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS Parent.exe 3056 IRP_MJ_CLEANUP C:\Program Files\Test\temp\dll32.dll SUCCESS Parent.exe 3056 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS System 4 FASTIO_ACQUIRE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS SyncType: SyncTypeOther System 4 FASTIO_RELEASE_FOR_SECTION_SYNCHRONIZATION C:\Program Files\Test\temp\dll32.dll SUCCESS System 4 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS System 4 IRP_MJ_CLOSE C:\Program Files\Test\temp\dll32.dll SUCCESS
  Message 7 of 9  
15 Nov 17 04:10
Slava Imameev
xxxxxx@hotmail.com
Join Date: 13 Sep 2013
Posts To This List: 264
Separate cache for images

<QUOTE> Modifying on read seemed like a good option, however with the image caching that doesn't seem to play by normal rules (at least in my Win7 VM) </QUOTE> A filter driver should see a paging read request when a CPU tries to fetch data from a virtual address for a PTE marked as invalid and when a page fault can't be resolved with a page in the active or standby list ( this is what you call "image caching" ). The one thing you should understand is that this page is shared by all processes that map this dll/exe and your filter doesn't make it private for a process as happens for copy on write (COW) pages on first modification outside FSD stack. Your filter should also be prepared to modify a page multiple times as it gets evicted and reread again by the Memory Manager. <QUOTE> (can I modify in user-mode before DllAttach is called?) </QUOTE> Yes. And there is more - you can make modified pages private for a process with NtProtectVirtualMemory( PAGE_WRITECOPY ) before modification like a debugger does before it sets a breakpoint. COW pages are backed by a page file so you don't need to modify them again for the same process after they are reclaimed and read again from a page file. <QUOTE> Is there a way to lower this threshold or temporarily cross it so it causes a purge? </QUOTE> There is some values in the registry. They are described in the "Windows Internals" book series. I believe they are read once at boot so this might be not what you want. Alternatively, it is possible to run a memory hog process with elevated privileges that allocates and commits a huge amount of memory in its virtual address space, but this might result in DoS for other subsystems including the kernel mode one. <QUOTE> Shouldn't there be at least one read initially? </QUOTE> "access" means "read" in this context. So, yes there should be a paging read to bring data from a file system to a page allocated by the Memory Manager to map an executable image. <QUOTE> LoadLibrary on the new dll (different filename than the original). However as I monitor in ProcMon, there are no IRP_MJ_READ requests on the new dll. It gets written, ACQUIRED_SECTION_SYNC is called a few times and then the load image callback is called. The output is below. </QUOTE> LoadLibrary creates a file mapping structures to support future page faults but doesn't allocate pages for an executable portion of a file. These pages are read when a page fault is resolved on first access or by a prefetcher thread. In your case the file was written before being mapped as an image so it also had pages cached through a data section, but data section and image section don't share pages. So this should not be an issue here.
  Message 8 of 9  
15 Nov 17 07:28
Scott Noone
xxxxxx@osr.com
Join Date: 10 Jul 2002
Posts To This List: 942
List Moderator
Separate cache for images

<QUOTE> In your case the file was written before being mapped as an image so it also had pages cached through a data section, but data section and image section don't share pages </QUOTE> You're right they don't share, but Mm will try to copy the data from the data section to the image section instead of going to disk (MiCopyDataPageToImagePage). So, it would be correct to not see a fault on the image section if you didn't successfully flush/purge the data section after write. -scott OSR @OSRDrivers
  Message 9 of 9  
15 Nov 17 15:14
John
xxxxxx@gmail.com
Join Date: 10 May 2014
Posts To This List: 35
Separate cache for images

Thank you Slava and Scott. I added a call to MmForceSectionClosed(SOP, TRUE) in PRE_CLEANUP and I'm now seeing the reads I would expect. I'm sure it's not great for performance but at least for now it works. If only the address space lock wasn't held in the image callback, it'd make life so much simpler.
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the ntfsd list to be able to post.

All times are GMT -5. The time now is 00:35.


Copyright ©2015, OSR Open Systems Resources, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license