Change filesize problem

NTFSD
1

Hi everybody,
I’m developing a compression minifilter which, at this point, compresses and decompresses fine directly from the disk and directly to the disk. For example, when I use notepad to create and open a file or when I use explorer to copy a file into the filtered device and to copy it back to a normal device, everything’s fine.

When I’m finishing writing the file, I change the file size to only the size of actual data written on disk, and the file size returned from IRP_MJ_QUERY_INFORMATION, IRP_MJ_NETWORK_QUERY_OPEN and IRP_MJ_DIRECTORY_CONTROL completion rutines is the decompressed file size.

The problem I’m having is when I open a compressed file from, for example, wordpad:
If I have a file originally of 1000 bytes and I compress it and save it in 400 bytes, wordpad would only read the first 400 bytes. I’ve been using FileSpy and I noticed that every file size is correctly changed (also allocationsize), but when wordpad reaches 400 bytes it gets STATUS_END_OF_FILE as return from the FASTIO_READ.
The first solution I tried was to change filesize and alloctionsize in FCB when processing the create completion routine because those sizes where 400 and not 1000 bytes, and it worked, but I guess that’s a bit unsafe or I was doing it wrong because when I processed a write IRP and later wanted to read or write directly to the file I got STATUS_FILE_CORRUPT_ERROR. Then I tried using CcSetFileSizes but I got back to the same problem.
So I left FCB alone and tried to disable fastio, but I still got the same STATUS_END_OF_FILE to wordpad, but this time from non cached IRP_MJ_READ IRP, so, just for testing, I hard coded the values of IoStatus in the read completion routine to be successful in the cached case, but now wordpad gets stuck doing a IRP_MJ_READ from Offset 500, ToRead FFE, Read FFE again and again.

I’ve been looking at the FAQ and a lot of posts for a while, but I still don’t know what I’m doing wrong… or just what I’m not doing.
I hope someone can help me,

Thanks in advance,
Andr?s

2

Sounds like you are using a layered file system to achieve the compression. Since the data cached in your layer is the uncompressed data, yes, the FCB of your file objects should contain the uncompressed file sizes (file size and valid data length).

Who returns STATUS_FILE_CORRUPT_ERROR when you set the FCB to have the correct sizes? The file system below you? Or your layer?

Lijun

From: “xxxxx@hotmail.com”
To: Windows File Systems Devs Interest List
Sent: Fri, March 12, 2010 2:49:15 PM
Subject: [ntfsd] Change filesize problem

Hi everybody,
I’m developing a compression minifilter which, at this point, compresses and decompresses fine directly from the disk and directly to the disk. For example, when I use notepad to create and open a file or when I use explorer to copy a file into the filtered device and to copy it back to a normal device, everything’s fine.

When I’m finishing writing the file, I change the file size to only the size of actual data written on disk, and the file size returned from IRP_MJ_QUERY_INFORMATION, IRP_MJ_NETWORK_QUERY_OPEN and IRP_MJ_DIRECTORY_CONTROL completion rutines is the decompressed file size.

The problem I’m having is when I open a compressed file from, for example, wordpad:
If I have a file originally of 1000 bytes and I compress it and save it in 400 bytes, wordpad would only read the first 400 bytes. I’ve been using FileSpy and I noticed that every file size is correctly changed (also allocationsize), but when wordpad reaches 400 bytes it gets STATUS_END_OF_FILE as return from the FASTIO_READ.
The first solution I tried was to change filesize and alloctionsize in FCB when processing the create completion routine because those sizes where 400 and not 1000 bytes, and it worked, but I guess that’s a bit unsafe or I was doing it wrong because when I processed a write IRP and later wanted to read or write directly to the file I got STATUS_FILE_CORRUPT_ERROR. Then I tried using CcSetFileSizes but I got back to the same problem.
So I left FCB alone and tried to disable fastio, but I still got the same STATUS_END_OF_FILE to wordpad, but this time from non cached IRP_MJ_READ IRP, so, just for testing, I hard coded the values of IoStatus in the read completion routine to be successful in the cached case, but now wordpad gets stuck doing a IRP_MJ_READ from Offset 500, ToRead FFE, Read FFE again and again.

I’ve been looking at the FAQ and a lot of posts for a while, but I still don’t know what I’m doing wrong… or just what I’m not doing.
I hope someone can help me,

Thanks in advance,
Andr?s

—
NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

Hi Lijun, thankyou for your replay,
I think who’s returning STATUS_FILE_CORRUPT_ERROR is the file system below me… because it happens when I call FltReadFile and FltWriteFile, but I notice that it only happens when receive and process a non cached IRP_MJ_WRITE before I call either of these two functions.

I don’t know if it has something to do with the problem… but I’m using the same FileObject that I receive from the FltObjects argument in the completion and dispatch routines (which already has de FCB sizes modified), to call the FltReadFile and FltWriteFile functions…

Thanks,
Andr?s

4

Hi again,
I placed a couple of breakpoints, I noticed that I’m getting the STATUS_FILE_CORRUPT_ERROR value in the completion routine of IRP_MJ_WRITE, in the Data->IoStatus.Status parameter. And when I call FltReadFile or FltWriteFile, the system tries to complete the same writing that failed last time with STATUS_FILE_CORRUPT_ERROR.

I looked at the code of fastfat of Windows XP that comes with the WDK, and it think that, for the writing, STATUS_FILE_CORRUPT_ERROR is returned when FileSize > FcbOrDcb->Header.AllocationSize.LowPart… but when I change FileSize, AllocationSize and ValidDataLength in the FCB, I’m pretty sure that AllocationSize >= FileSize.

I was thinking of changing the three values of the filesize in the FCB before the PreWrite routine is about to return, and change them back again when PostWrite is called… but I think that’s a little crazy and unsafe.

Thanks in advance,
Andr?s

5

It seems to me you are not using the layered file system approach after all after reading your two new emails. Before changing the file sizes in with the FCB, I?recommend?you?reconsider your design and make sure the FCB belows to your file system. In my opinion, only the file system owning that FCB should?be responsible for?changing it and interact with Cache manager.?If?not, you are actually tampering the cache for the lower file system and it will doom to fail.?

If you want to allow caching at all, you need to consider using the layered approach.

Lijun?

From: “xxxxx@hotmail.com”
To: Windows File Systems Devs Interest List
Sent: Tue, March 16, 2010 1:54:01 AM
Subject: RE:[ntfsd] Change filesize problem

Hi again,
I placed a couple of breakpoints, I noticed that I’m getting the STATUS_FILE_CORRUPT_ERROR value in the completion routine of IRP_MJ_WRITE, in the Data->IoStatus.Status parameter. And when I call FltReadFile or FltWriteFile, the system tries to complete the same writing that failed last time with STATUS_FILE_CORRUPT_ERROR.

I looked at the code of fastfat of Windows XP that comes with the WDK, and it think that, for the writing, STATUS_FILE_CORRUPT_ERROR is returned when FileSize > FcbOrDcb->Header.AllocationSize.LowPart… but when I change FileSize, AllocationSize and ValidDataLength in the FCB, I’m pretty sure that AllocationSize >= FileSize.

I was thinking of changing the three values of the filesize in the FCB before the PreWrite routine is about to return, and change them back again when PostWrite is called… but I think that’s a little crazy and unsafe.

Thanks in advance,
Andr?s

—
NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

6

Hi, sorry for my very late reply.
Lijun, thanks for your suggestion, I’ve been reading the posts about the topic and you’re absolutly right about using the layered approach.
I was trying to do a pass-through layered minifilter based on the FastFat code but I ran into some troubles, and I hope someone can tell me what I’m doing wrong:

In the dispatch routine of IRP_MJ_CREATE I’m creating a new FILE_OBJECT with IoCreateStreamFileObjectLite(FltObjects->FileObject,NULL), I initialize the FileName field and send down the same IRP but this time associated with the new FILE_OBJECT. I create an FCB and SOP instance for the file (if they didn’t exist already), allocating and initializing the resources and fastmutex, and set them in the corresponding fields of the original FILE_OBJECT; also, in the original FILE_OBJECT, I set LockOperation, DeletePending, ReadAccess, WriteAccess, DeleteAccess, SharedRead, SharedWrite and SharedDelete with the values that returned in the new FILE_OBJECT, and PrivateCacheMap to NULL. Finally I set the operation as completed.
In the dispatch routine of IRP_MJ_CLEANUP, I just call CcUninitializeCacheMap for the FILE_OBJECT that I receive (which I guess is the original FILE_OBJECT in the create routine). And I also set the operation as completed.
In the dispatch routine of IRP_MJ_CLOSE, I send down the IRP I receive, associated to the FILE_OBJECT that was created in the create routine, and then I call ObDereferenceObject with that same object.
When the FCB of the real FILE_OBJECT is destroyed, I also destroy my FCB and SOP, calling FsRtlTeardownPerStreamContexts (because I called FsRtlSetupAdvancedHeader), and freeing all the resources, fastmutex and the structures themselves.
I got another PreOperation function that receives all the other IRP_MJ calls, and when it is called and the FltObjects->FileObject is one of those objects I filtered, I send down the IRP with the FILE_OBJECT that I created in the create routine.
Now the problem that I have is when I try to copy a file in explorer to the filtered device; I receive a IRP_MJ_CREATE, IRP_MJ_CLEANUP and IRP_MJ_CLOSE (among other calls), and everything seems fine, but sometime after that, I get a bugcheck in FatAcquireFcbForLazyWrite (I?m pretty sure it has something to do with the cache) with stack trace

f79c0cf0 804e3b1f e18b9c18 00000001 80550190 Fastfat!FatAcquireFcbForLazyWrite+0xc
f79c0d2c 804e61fe 831fcd90 8055b140 831c7b30 nt!CcWriteBehind+0x27
f79c0d74 80534c12 831fcd90 00000000 831c7b30 nt!CcWorkerThread+0x126
f79c0dac 805c61ee 831fcd90 00000000 00000000 nt!ExpWorkerThread+0x100
f79c0ddc 80541de2 80534b12 00000000 00000000 nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

And sometimes I get a bugcheck in FatDecodeFileObject but I don?t know why because all the IRP that I send down are with the FILE_OBJECT and FCB created by FastFat…
I don?t know if any FCB is being torndown in the middle of the a request.

Any help will be highly appreciated,
Andr?s

7

On 4/5/2010 3:23 PM, xxxxx@hotmail.com wrote:

f79c0cf0 804e3b1f e18b9c18 00000001 80550190 Fastfat!FatAcquireFcbForLazyWrite+0xc
f79c0d2c 804e61fe 831fcd90 8055b140 831c7b30 nt!CcWriteBehind+0x27
f79c0d74 80534c12 831fcd90 00000000 831c7b30 nt!CcWorkerThread+0x126
f79c0dac 805c61ee 831fcd90 00000000 00000000 nt!ExpWorkerThread+0x100
f79c0ddc 80541de2 80534b12 00000000 00000000 nt!PspSystemThreadStartup+0x34
00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16

And sometimes I get a bugcheck in FatDecodeFileObject but I don?t know why because all the IRP that I send down are with the FILE_OBJECT and FCB created by FastFat…
I don?t know if any FCB is being torndown in the middle of the a request.

The above crash is due to you not mapping over the file object in the
callback routines for file acquisition. Are you setting up routines for
fast_io entry points in your registration routine? If you need
additional functionality, you can register for the same fast_io acquire
routines by calling the FsRtlRegisterFileSystemFilterCallbacks().

The DecodeFileObject crash is because one of ‘your’ file objects is
making it to the underlying file system. When it tries to decode the
FsContext, it realizes it’s not one of its file objects and bug checks.

In general …

If you are going to take ownership of the file object passed into the
IRP_MJ_CREATE handler then under no conditions can this file object make
it to the underlying file system for any request.

As a suggestion, instead of calling IoCreateStreamFO(), just call
FltCreateFileEx() or a variant to get a handle and file object (close
the handle when you get a cleanup on your file object). The file object
you get from that call will be what is passed down to the underlying
file system for all requests. When any request is handled by your
filter, swap out the file object passed in with your file object and
pass down the request. You can begin the implementation by simply moving
over the file object in a default dispatch routine but you must ALWAYS
perform this mapping for file objects which you own.

Pete

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

8

> The DecodeFileObject crash is because one of ‘your’ file objects is making

it to the underlying file system. When it tries to decode the FsContext,
it realizes it’s not one of its file objects and bug checks.


It would really be nice if the in box FSDs would just fail these I/Os
instead of crashing (maybe just ASSERT in the checked build?). I realize
that it shouldn’t happen and the file object is potentially garbage in these
cases, but it would be better citizen behavior in my opinion since it does
occasionally happen with these types of filters.

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

“Peter Scott” wrote in message
news:xxxxx@ntfsd…
> On 4/5/2010 3:23 PM, xxxxx@hotmail.com wrote:
>>
>> f79c0cf0 804e3b1f e18b9c18 00000001 80550190
>> Fastfat!FatAcquireFcbForLazyWrite+0xc
>> f79c0d2c 804e61fe 831fcd90 8055b140 831c7b30 nt!CcWriteBehind+0x27
>> f79c0d74 80534c12 831fcd90 00000000 831c7b30 nt!CcWorkerThread+0x126
>> f79c0dac 805c61ee 831fcd90 00000000 00000000 nt!ExpWorkerThread+0x100
>> f79c0ddc 80541de2 80534b12 00000000 00000000
>> nt!PspSystemThreadStartup+0x34
>> 00000000 00000000 00000000 00000000 00000000 nt!KiThreadStartup+0x16
>>
>> And sometimes I get a bugcheck in FatDecodeFileObject but I don?t know
>> why because all the IRP that I send down are with the FILE_OBJECT and FCB
>> created by FastFat…
>> I don?t know if any FCB is being torndown in the middle of the a request.
>>
>
> The above crash is due to you not mapping over the file object in the
> callback routines for file acquisition. Are you setting up routines for
> fast_io entry points in your registration routine? If you need additional
> functionality, you can register for the same fast_io acquire routines by
> calling the FsRtlRegisterFileSystemFilterCallbacks().
>
> The DecodeFileObject crash is because one of ‘your’ file objects is making
> it to the underlying file system. When it tries to decode the FsContext,
> it realizes it’s not one of its file objects and bug checks.
>
> In general …
>
> If you are going to take ownership of the file object passed into the
> IRP_MJ_CREATE handler then under no conditions can this file object make
> it to the underlying file system for any request.
>
> As a suggestion, instead of calling IoCreateStreamFO(), just call
> FltCreateFileEx() or a variant to get a handle and file object (close the
> handle when you get a cleanup on your file object). The file object you
> get from that call will be what is passed down to the underlying file
> system for all requests. When any request is handled by your filter, swap
> out the file object passed in with your file object and pass down the
> request. You can begin the implementation by simply moving over the file
> object in a default dispatch routine but you must ALWAYS perform this
> mapping for file objects which you own.
>
> Pete
>
> –
> Kernel Drivers
> Windows File System and Device Driver Consulting
> www.KernelDrivers.com
> 866.263.9295
>

9

On 4/6/2010 8:08 AM, Scott Noone wrote:

> The DecodeFileObject crash is because one of ‘your’ file objects is
> making it to the underlying file system. When it tries to decode the
> FsContext, it realizes it’s not one of its file objects and bug checks.


> It would really be nice if the in box FSDs would just fail these I/Os
> instead of crashing (maybe just ASSERT in the checked build?). I realize
> that it shouldn’t happen and the file object is potentially garbage in
> these cases, but it would be better citizen behavior in my opinion since
> it does occasionally happen with these types of filters.
>

I completely agree.

Pete

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

10

Hi everybody,

Pete, thank you very much for your suggestions, I applied them and now everything seems to be working just fine.
I got a couple of doubts and I was wondering if you could help me with them:
I used FltCreate and ObReferenceObjectByName to get the file object in the PreCreate routine (I didn?t use FltCreateEx because, for some reason, it is not defined with my winxp checked configuration), in the PreCleanup routine I call FltClose with the file handle, and in the PreClose routine I call ObDereferenceObject with the file object? but the thing is that I call it twice because when I was rolling down the close IRP, I was used to receive a context cleanup related to a stream context of the real file object, but when I call only once ObDereferenceObject I never get that call, making a second call does the trick… I?ve read that it isn?t possible to know when a stream context cleanup happens, because it is related to stream data itself and not to the file object (I suppose it?s related to the FCB), but I thought there was only one instance of the file object (the one I created), so I don?t know how bad is what I did.

The second question that I had is related to the generic pass-through function. When I use FltSetCallbackDataDirty and set the TargetFileObject, everthing?s fine; but when I use FltReissueSynchronousIo also changing TargetFileObject and completing the operation, somtimes I get an AccessViolation exception with the following stack trace:

b6709aa4 00000000 830d1d7c fltMgr!FltpPassThroughInternal+0x19
830d1bc0 00000000 00000004 fltMgr!FltReissueSynchronousIo+0x13f
82e11f5c b6709af8 b6709b28 MiniFilterTest!PreGeneric+0x5b
00709b70 b6709bb8 b6709be0 fltMgr!FltpPerformPreCallbacks+0x2d4
00709b70 8301a190 00000000 fltMgr!FltpPassThroughFastIo+0x3b
82d8d4b0 00000001 b6709bb8 fltMgr!FltpFastIoQueryStandardInfo+0x101
82d8d4b0 b6709c0c 82dd34a8 nt!FsRtlGetFileSize+0x35
82d8d4b0 b6709cac b6709d0c nt!MiCreateDataFileMap+0x1c
b6709d1c 000f0005 00000000 nt!MmCreateSection+0x549
028ee280 000f0005 00000000 nt!NtCreateSection+0x12f
028ee280 000f0005 00000000 nt!KiFastCallEntry+0xf8
7c8094e5 028ee280 000f0005 ntdll!KiFastSystemCallRet
000f0005 c0000034 028ee2b0 ntdll!ZwCreateSection+0xc
000009d4 00000000 00000002 ntdll!RtlNtStatusToDosError+0x38
WARNING: Frame IP not in any known module. Following frames may be wrong.
7c809a7f 00009b3f 028ee2d0 0x7c80955f
02573d60 000009d4 0259d25c ntdll!RtlAllocateHeap+0xeac
02548770 028ee4dc 7c920041 0x76bf3595
00091378 7c92005d 00000080 ntdll!RtlpFreeToHeapLookaside+0x22
77dbfc8a 0004012e 028ee54c ntdll!RtlFreeHeap+0x1e9
00000000 00000000 00000000 0x28ee560
00000000 00000000 00000000 0x28ee704

I thoght that those two mechanisms were equivalent, I mean, I can use the FltSetCallbackDataDirty way, but I keep asking my self if I did something wrong.

Thanks in advance,
Andr?s

11

On 4/6/2010 1:21 PM, xxxxx@hotmail.com wrote:

Hi everybody,

Pete, thank you very much for your suggestions, I applied them and now everything seems to be working just fine.
I got a couple of doubts and I was wondering if you could help me with them:
I used FltCreate and ObReferenceObjectByName to get the file object in the PreCreate routine (I didn?t use FltCreateEx because, for some reason, it is not defined with my winxp checked configuration), in the PreCleanup routine I call FltClose with the file handle, and in the PreClose routine I call ObDereferenceObject with the file object? but the thing is that I call it twice because when I was rolling down the close IRP, I was used to receive a context cleanup related to a stream context of the real file object, but when I call only once ObDereferenceObject I never get that call, making a second call does the trick… I?ve read that it isn?t possible to know when a stream context cleanup happens, because it is related to stream data itself and not to the file object (I suppose it?s related to the FCB), but I thought there was only one instance of the file object (the one I created), so I don?t know how bad is what I did.

Instead of the Ob call you are making above, you should just call
ObReferenceObjectByHandle() to get a file object for the handle you get
back. This way when you get the cleanup, call FltClose on your handle
and when you get the close, call ObDereferenceObject() on the file
object you got from the ObReferenceObjectByHandle() routine.

The second question that I had is related to the generic pass-through function. When I use FltSetCallbackDataDirty and set the TargetFileObject, everthing?s fine; but when I use FltReissueSynchronousIo also changing TargetFileObject and completing the operation, somtimes I get an AccessViolation exception with the following stack trace:

Do the former, simply update the file object and mark the block as
dirty. If you need to perform some processing on completion, request a
completion callback.

Also remember that some requests have embedded handles and file objects.
For instance in your pre-create, be sure that the related file object is
not one of yours. If it is you need to update it. As well there are some
set file info requests like rename that have file objects within the
data structure. I seem to remember that also being one request in the
set file info that has a handle within the data block.

You’ll also need to handle the name provider callbacks since these pass
file objects around. If you are not handling these, you’ll get the
DecodeFileObject() exception at some point.

Check out the SimRep sample filter, it does most of this stuff but there
are a few issues with it so use it for reference only.

Pete

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

12

>I seem to remember that also being one request in the set file info that

has a handle within the data block.

FSCTL_MOVE_FILE has a handle embedded in the structure, which is a PITA.

-scott

–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com

“Peter Scott” wrote in message
news:xxxxx@ntfsd…
> On 4/6/2010 1:21 PM, xxxxx@hotmail.com wrote:
>> Hi everybody,
>>
>> Pete, thank you very much for your suggestions, I applied them and now
>> everything seems to be working just fine.
>> I got a couple of doubts and I was wondering if you could help me with
>> them:
>> I used FltCreate and ObReferenceObjectByName to get the file object in
>> the PreCreate routine (I didn?t use FltCreateEx because, for some reason,
>> it is not defined with my winxp checked configuration), in the PreCleanup
>> routine I call FltClose with the file handle, and in the PreClose routine
>> I call ObDereferenceObject with the file object? but the thing is that I
>> call it twice because when I was rolling down the close IRP, I was used
>> to receive a context cleanup related to a stream context of the real file
>> object, but when I call only once ObDereferenceObject I never get that
>> call, making a second call does the trick… I?ve read that it isn?t
>> possible to know when a stream context cleanup happens, because it is
>> related to stream data itself and not to the file object (I suppose it?s
>> related to the FCB), but I thought there was only one instance of the
>> file object (the one I created), so I don?t know how bad is what I did.
>>
>
> Instead of the Ob call you are making above, you should just call
> ObReferenceObjectByHandle() to get a file object for the handle you get
> back. This way when you get the cleanup, call FltClose on your handle and
> when you get the close, call ObDereferenceObject() on the file object you
> got from the ObReferenceObjectByHandle() routine.
>
>> The second question that I had is related to the generic pass-through
>> function. When I use FltSetCallbackDataDirty and set the
>> TargetFileObject, everthing?s fine; but when I use
>> FltReissueSynchronousIo also changing TargetFileObject and completing the
>> operation, somtimes I get an AccessViolation exception with the following
>> stack trace:
>>
>
> Do the former, simply update the file object and mark the block as dirty.
> If you need to perform some processing on completion, request a completion
> callback.
>
> Also remember that some requests have embedded handles and file objects.
> For instance in your pre-create, be sure that the related file object is
> not one of yours. If it is you need to update it. As well there are some
> set file info requests like rename that have file objects within the data
> structure. I seem to remember that also being one request in the set file
> info that has a handle within the data block.
>
> You’ll also need to handle the name provider callbacks since these pass
> file objects around. If you are not handling these, you’ll get the
> DecodeFileObject() exception at some point.
>
> Check out the SimRep sample filter, it does most of this stuff but there
> are a few issues with it so use it for reference only.
>
> Pete
>
> –
> Kernel Drivers
> Windows File System and Device Driver Consulting
> www.KernelDrivers.com
> 866.263.9295
>

13

Hi,

Pete and Scott, thank you very much for your help.
I’m swapping not only the principal file object of the IRP, but the file objects that are present inside the parameters of other IRP calls. And I haven’t seen a bugcheck for a while? but I’m getting some weird results sometimes.
I would appreciate very much if someone can help me.

I’m using ObReferenceObjectByHandle to get the fileobject to pass down to all the IRPs that I receive, and I’m also checking if RelatedFileObject is one of file objects created by me, and if so, I replace it? but I don’t know how deep should I go with this? I mean, should I check the RelatedFileObject of the RelatedFileObject until someone is NULL?

Now, when I modify a file in notepad, the file size becomes zero, even though, in the post operation of the IRP_MJ_SET_INFORMATION call, I’m copying the values from the FCB of the file object that went down, to the FCB of the upper file object. I think this could be related to the fact that the real FCB is never being released, I conclude this because I have a stream context related to the real file object, and I never get the context cleanup call; but I have a similar situation in the upper file object, and I do receive the context cleanup call.

Thanks in advance,
Andr?s

14

On 4/10/2010 9:35 PM, xxxxx@hotmail.com wrote:

Hi,

Pete and Scott, thank you very much for your help.
I’m swapping not only the principal file object of the IRP, but the file objects that are present inside the parameters of other IRP calls. And I haven’t seen a bugcheck for a while? but I’m getting some weird results sometimes.
I would appreciate very much if someone can help me.

I’m using ObReferenceObjectByHandle to get the fileobject to pass down to all the IRPs that I receive, and I’m also checking if RelatedFileObject is one of file objects created by me, and if so, I replace it? but I don’t know how deep should I go with this? I mean, should I check the RelatedFileObject of the RelatedFileObject until someone is NULL?

Just check the RFO in the current FO since anything previous in the
chain would have already been handled by your filter.

Now, when I modify a file in notepad, the file size becomes zero, even though, in the post operation of the IRP_MJ_SET_INFORMATION call, I’m copying the values from the FCB of the file object that went down, to the FCB of the upper file object. I think this could be related to the fact that the real FCB is never being released, I conclude this because I have a stream context related to the real file object, and I never get the context cleanup call; but I have a similar situation in the upper file object, and I do receive the context cleanup call.

While SimRep does not fake file sizes, it does handle all the file
object ‘exchanging’ required in the filter. Have you looked through
there and made sure there is not something you are missing? Are you
handling all the FastIO entry points? This is a rather complex problem
and there are dozens of reasons why things could be failing.

I would guess you are currently doing this but try to get the filter
working without any modification to any data, just simply pass-through.
You are still swapping out file objects but that’s it.

Pete

Thanks in advance,
Andrés

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

15

Pete, thanks again for all your help.
Ok, so I’m just taking care of the RelatedFileObject in the current FileObject as you said.

I’m not quite sure I’m looking at the right simrep example… because the one that I’ve got in the WDK doesn’t seem to manage file objects very much: in the pre create routine it swaps the filename field, returns STATUS_REPARSE and IO_STATUS_REPARSE and completes the operation. I don’t know if I should use this technique in my pass-through layered minifilter. If there’s an example that does swap file objects, it would be very helpful.

About the FastIO… well, I think so, I mean, I’m filtering almost every IRP with no flags in the FLT_OPERATION_REGISTRATION structure, and also I’m never using FLT_IS_FASTIO_OPERATION macro to treat differently the calls, and I’m not using FsRtlRegisterFileSystemFilterCallbacks.

Thanks,
Andr?s

16

On 4/11/2010 5:21 PM, xxxxx@hotmail.com wrote:

Pete, thanks again for all your help.
Ok, so I’m just taking care of the RelatedFileObject in the current FileObject as you said.

I’m not quite sure I’m looking at the right simrep example… because the one that I’ve got in the WDK doesn’t seem to manage file objects very much: in the pre create routine it swaps the filename field, returns STATUS_REPARSE and IO_STATUS_REPARSE and completes the operation. I don’t know if I should use this technique in my pass-through layered minifilter. If there’s an example that does swap file objects, it would be very helpful.

My mistake on the simrep sample, I recalled the implementation incorrectly.

What are you running into now? Are you still hitting a crash?

About the FastIO… well, I think so, I mean, I’m filtering almost every IRP with no flags in the FLT_OPERATION_REGISTRATION structure, and also I’m never using FLT_IS_FASTIO_OPERATION macro to treat differently the calls, and I’m not using FsRtlRegisterFileSystemFilterCallbacks.

That should do it or you could just disable FastIO until you have things
running.

Pete

Thanks,
Andrés

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

17

Hi,
Pete, thank you very much again for your response and your help.
I was able to resolve the problem about the zero file size when writing in notepad, and other problems that I wasn’t aware of.
When I tried to read the file from wordpad, the system hanged, so I checked some IRPs with FileSpy and I realized that I didn’t update the CurrentByteOffset field in the upper file object… so in the completion routine of IRP_MJ_READ, IRP_MJ_WRITE and IRP_MJ_SET_INFORMATION, I set the values that were returned in the lower file object, as well as the values of ValidDataLength, FileSize and AllocationSize.

I have a couple of doubts and I hope someone can help me.

I seem to be able to perform some normal operations successfully, for example, I can use any file in the device in wordpad and it works fine. But when I use notepad (again), if I add a some characters at the end of the file, and save it, the next time I open the file, with notepad or wordpad, the last written characters are now blanks, but when I do the same, adding the characters with wordpad, everything’s fine. Perhaps it has something to do with flushing the cache?.. I mean, in the precleanup operation of the upper file object I’m only doing a fltclose of the lower file object, and probably I should do something about the upper file object SOP but I’m pretty lost here…

Something else weird is happening; when I permanently delete a file in explorer everything’s fine. But when I hit just delete to send the file to the Recycle Bin, I can’t do any other action on the device, and I checked the !locks extension of WinDbg and there’s just one lock held, but I think is something else because it’s also held sometimes when the minifilter is not even attached. Looking at the events on FileSpy, there only IRP_MJ_CREATE, IRP_MJ_QUERY_VOLUME_INFORMATION, IRP_MJ_CLEANUP and IRP_MJ_CLOSE operations sent to the volume root folder (in this case D:) again and again. Also, this only happens when there’s not the Recycle Bin folder, I think it has something to do with the creation of the file D:\Recycled\desktop.ini, because when this file and folder exist, I can move to the Recycle Bin normally.

Thanks in advance,
Andr?s

18

On 4/12/2010 11:32 PM, xxxxx@hotmail.com wrote:

Hi,
Pete, thank you very much again for your response and your help.
I was able to resolve the problem about the zero file size when writing in notepad, and other problems that I wasn’t aware of.
When I tried to read the file from wordpad, the system hanged, so I checked some IRPs with FileSpy and I realized that I didn’t update the CurrentByteOffset field in the upper file object… so in the completion routine of IRP_MJ_READ, IRP_MJ_WRITE and IRP_MJ_SET_INFORMATION, I set the values that were returned in the lower file object, as well as the values of ValidDataLength, FileSize and AllocationSize.

I have a couple of doubts and I hope someone can help me.

I seem to be able to perform some normal operations successfully, for example, I can use any file in the device in wordpad and it works fine. But when I use notepad (again), if I add a some characters at the end of the file, and save it, the next time I open the file, with notepad or wordpad, the last written characters are now blanks, but when I do the same, adding the characters with wordpad, everything’s fine. Perhaps it has something to do with flushing the cache?.. I mean, in the precleanup operation of the upper file object I’m only doing a fltclose of the lower file object, and probably I should do something about the upper file object SOP but I’m pretty lost here…

Are you correctly handling all the paging requests being sent down? It
sounds like you are missing a paging write possibly, either dropping it
or not correctly handling it. For instance, you would need to propagate
the setfileinfo calls that extend the file as well as ensuring that you
extend the file during a write that goes beyond the EOF, this is for
non-paging writes only.

I would suggest tracing all of the file sizes for each request, on
dispatch and completion. Then make sure you are sending down the paging
requests correctly.

How are you initializing caching on the upper file object? The one which
you are taking responsibility for needs to have caching initialized,
file sizes updated correctly, etc.

Remember now that you are taking ownership of those upper file objects,
you need to do everything to those file objects that the underlying file
system does to the lower file objects.

Something else weird is happening; when I permanently delete a file in explorer everything’s fine. But when I hit just delete to send the file to the Recycle Bin, I can’t do any other action on the device, and I checked the !locks extension of WinDbg and there’s just one lock held, but I think is something else because it’s also held sometimes when the minifilter is not even attached. Looking at the events on FileSpy, there only IRP_MJ_CREATE, IRP_MJ_QUERY_VOLUME_INFORMATION, IRP_MJ_CLEANUP and IRP_MJ_CLOSE operations sent to the volume root folder (in this case D:) again and again. Also, this only happens when there’s not the Recycle Bin folder, I think it has something to do with the creation of the file D:\Recycled\desktop.ini, because when this file and folder exist, I can move to the Recycle Bin normally.

What happens with further actions, say WordPad a file? Do you see the
open come in, etc.?

Pete

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

19

Hi, sorry again for my late reply.

Pete, thanks again for your reply, I really appreciate all your help.
I was able to fix the problem about the writes that were beyond the EOF in notepad and wordpad. I think it was something related with a delayed write, because when I waited a little while, all the written data suddenly appear; so, when I call FltCreateFile, I just add FILE_WRITE_THROUGH in the CreateOptions parameter and now things are just fine… but I don’t know if, instead of this, I should wait for Resource or PagingResource of the upper or lower FCB somewhere in the cleanup or close operations.

About the D:\Recycled\desktop.ini file, the last request I receive is a FAST_LOCK with status FAILURE, after an IRP_MJ_CREATE. Although I’m still able to “use” the device normally, I mean, through cmd.exe I can access the folders and files, and, also, I can use notepad or wordpad with any file. After that, when I foce explorer to terminate, I receive a IRP_MJ_LOCK_CONTROL/IRP_MN_LOCK with status STATUS_CANCELLED, and I can use explorer agian, but when I try to just open the D: drive, it hangs again with the same FAST_LOCK request.
Do I need to manage IRP_MJ_LOCK_CONTROL in a different way rather than just swapping the FileObject of the request?

Any help will be highly appreciated,
Andr?s