redirection with STATUS_REPARSE and FltReissueSynchronousIo

Jason_Zhang · November 14, 2012, 11:42am

I have a minifilter driver which handles stub (reparse point) file HSM restoration. When the user double click the stub file, the driver reads the contents in the reparse point file and passes these information to the the application program. The application program uses this information to retrieve the real file from the archive device and the filter driver renames the file with the stub file. So when the user opens the file, it is a restored file.

Most of the functions are implemented in the PostCreate() callback function. I need to implement a redirection to the retrieved temp file if the temp file can’t be renamed to the stub file.

I want to use the following logic, but not sure about the usage of the FltReissueSynchronousIo()
PostCreate()
{
if(pData->IoStatus.Status == STATUS_REPARSE)
{
// calls user mode application to retrieve the file and rename the retrieved temp file if the retrieved file can be renamed to the stub file, return SUCCESS. Otherwise, return ERROR_RENAME_KEY_TO_STUB and change the pData file
// name to the retrieved temp file for further redirection
status = HSMGetFileInUserMode();

if(NT_SUCCESS(status))
{
return FLT_POSTOP_FINISHED_PROCESSING;
}
// if the retrieved file can’t be renamed, begin redirection. pData is the callback data
else if(status == ERROR_RENAME_KEY_TO_STUB)
{
pData->IoStatus.Status = STATUS_REPARSE;
pData->IoStatus.Information = IO_REPARSE;
if(FltIsOperationSynchronous(pData))
{
FltSetCallbackDataDirty(Data);
FltReissueSynchronousIo(pFltObjects->Instance, pData);
}
}
}
return FLT_POSTOP_FINISHED_PROCESSING;
}

As to the FltIsOperationSynchronousIo(), with the MS document, the minifilter driver will initiate a new request to the minifilter driver lower than it and then to the file system driver. when it is completed by the file system driver, it will go through the stack upper than this minifilter driver. So the postcreate routine of the filter driver reissuing the request will not be called when the request back, right ? if so, the Data->IoStatus.Status will be set to STATUS_SUCCESS by the FS when the request back?
If we can use the FltReissueSynchronousIo() to implement the redirection, do we still need to use the pData->IoStatus.Status = STATUS_REPARSE ? With my understanding, the I/O manager will start a new IRP_MJ_CREATE once it finds the status is set to STATUS_REPARSE. Will the driver postCreate() function be
called once again with the new IRP_MJ_CREATE ?
Currently, when the rename fails, and during redirection, what confuses me is that the HSMGetFileInUserMode() is called several times. This causes problem because each time the HSMGetFileInUserMode() is called, there is a retrieval action triggered. There are lots of temp files retrieved to local before the temp files is redirected and opened. With my understanding, when the I/O manager sends another IRP_MJ_CREATE with STATUS_REPARSE, the file name in the data is pointing to the retrieved temp file. The FS will set the Data->IoStatus.Status to STATUS_SUCCESS and when it goes through the minifilter postCreate routine, it will not call the HSMGetFileInUserMode() again.
If FltReissueSynchronousIo(pFltObjects->Instance, pData) is used for redirection, do I still need to return FLT_PREOP_SYNCHRONIZE in the precreate callback routine?

Alex_Carp · November 14, 2012, 12:09pm

Hi Jason,

Well, Data->IoStatus is only meaningful on the way up (once the operation has completed). So setting it before sending the request down doesn’t actually do anything for you (since the filters below yours will be in preCreate and so they won’t look at it and then whoever completes the create operation will overwrite it with their status).

So if you want to use STATUS_REPARSE then you need to set the name in the FileObject->FileName to the name of the temporary file that you recalled the data and then set the IoStatus to STATUS_REPARSE and then complete the operation (and not send it down).

Thanks,
Alex.

Jason_Zhang · November 14, 2012, 12:31pm

Thanks, Alex.
Clarify two things:

The calling of the FltReissueSynchronousIo() in postCreate() will cause the request goes down again immediately (The request is back from FS and through the postCreate() routine)
The FltReissueSynchronousIo() is not used in redicretion. Set the status to STATUS_REPARSE SO that the I/O manager can find the status and start another IRP_MJ_CREATE for the redirection.

PostCreate()
{
if(pData->IoStatus.Status == STATUS_REPARSE)
{
// if the retrieved file can be renamed to the stub file, return SUCCESS. Otherwise, return ERROR_RENAME_KEY_TO_STUB and change the pData file
// name to the retrieved temp file for further redirection
status = HSMGetFileInUserMode();

if(NT_SUCCESS(status))
{
return FLT_POSTOP_FINISHED_PROCESSING;;
}
// if the retrieved file can’t be renamed, begin redirection. pData is the callback data
else if(status == ERROR_RENAME_KEY_TO_STUB)
{
pData->IoStatus.Status = STATUS_REPARSE;
pData->IoStatus.Information = IO_REPARSE;

}
}
return FLT_POSTOP_FINISHED_PROCESSING;;
}

My old implementation is like this, but I don’t understand the HSMGetFileInUserMode() is called several times. This causes problem because each time the HSMGetFileInUserMode() is called, there is a retrieval action triggered. There are lots of temp files retrieved to local before the temp files is redirected and opened. With my understanding, when the I/O manager sends another IRP_MJ_CREATE with STATUS_REPARSE, the file name in the data is pointing to the retrieved temp file. The FS will set the Data->IoStatus.Status to STATUS_SUCCESS and when it goes through the minifilter postCreate routine, it will not call the HSMGetFileInUserMode() again.

Alex_Carp · November 14, 2012, 12:39pm

yup, the request is immediately sent down
no. the status you set is pretty much ignored, like I explained above. it will not be seen by the IO manager because it is overwritten by the FS (incidentally, it will be overwritten with STATUS_REPARSE unless you change the file to not be a reparse point anymore).
since you didn’t change the file name the STATUS_REPARSE will most likely retry to open the same file (the stub file) and so you’ll go through the whole thing again. setting FileObject->FileName is mandatory.

Thanks,
Alex.

Jason_Zhang · November 14, 2012, 12:54pm

Thanks a lot, Alex.

I forget to mention the FileObject->FileName is set in the function HSMInUserMode(), that function calls application for file retrieval, try to rename the stub file with the retrieved temp file first, if not work, it will set the FileObject->FileName to the temp file. That is why I am confused so many callings of the HSMGetFileInUserMode() for the retrieval
The old implementation is like this
if(pData->IoStatus.Status == STATUS_REPARSE)
{
status = HSMGetFileInUserMode();
if(status == ERROR_RENAME_KEY_TO_STUB)
{
pData->IoStatus.Status = STATUS_REPARSE;
pData->IoStatus.Information = IO_REPARSE;
}
}

Alex_Carp · November 14, 2012, 1:29pm

Well, I didn’t see any parameters being passed to HSMInUserMode(). Anyway, why not put a breakpoint in HSMInUserMode() and see where it’s called from and why ?

Thanks,
Alex.

Jason_Zhang · November 14, 2012, 5:13pm

Thanks a lot, Alex.

I am quite new to filter driver. Use Windbg for the debug ?

From current test results, the pdf file doesn’t have such problem, it can
be redirected immediately. However, MS office files (docx, xlsx, pptx), the
redirection will be tried at least 3 times before the temp file is opened.
These are tested on win2008R2 without any anti-virus software installed.

Also find the symantec family anti-virus will affect and more redirections
will be called if the machine has anti-virus software installed.

2012/11/14

> Well, I didn’t see any parameters being passed to HSMInUserMode(). Anyway,
> why not put a breakpoint in HSMInUserMode() and see where it’s called from
> and why ?
>
> Thanks,
> Alex.
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Alex_Carp · November 15, 2012, 10:34am

Windbg seems to be the choice for many on this list.

It looks like the number of retries is something controlled by the application. However, I wouldn’t focus too much on the number of retries. I’m thinking that if the files would be restored correctly then the application shouldn’t see any failure and presumably it wouldn’t retry a successful create.

Thanks,
Alex.

Jason_Zhang · November 15, 2012, 1:31pm

Appreciate your help, Alex.

From the output of DebugView, when the PostCreate is called first time,
before leaving the PostCreate, I can see the
pData->Iopb->TargetFileObject->FileName has been changed to the temp file
name, the pData->IoStatus.Status is STATUS_REPARSE and
pData->IoStatus.Information is IO_REPARSE,
But these changes seem not to take effect, the PostCreate is called again
and again, pData->Iopb->TargetFileObject->FileName is the stub file name
when it is called.

2012/11/15

> Windbg seems to be the choice for many on this list.
>
> It looks like the number of retries is something controlled by the
> application. However, I wouldn’t focus too much on the number of retries.
> I’m thinking that if the files would be restored correctly then the
> application shouldn’t see any failure and presumably it wouldn’t retry a
> successful create.
>
> Thanks,
> Alex.
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Jason_Zhang · November 16, 2012, 3:24pm

I think the restoration is good. I can directly open each of the restored
temp files.

For the same ZIP format stubbed file, if I use WinRAR to open it, it will
trigger 2 times of the retry before the redirection successful. However, if
I use 7-zip for the same file, the redirection is successful without any
additional retries. (whatever numbers of the files in the ZIP package)

Does it related to the application? The MS office files are composed of 3
folders and 1 xml inside. Is there any reason that the redirection can’t be
finished at one time?

Thanks!

2012/11/15 Song Zhang

> Appreciate your help, Alex.
>
> From the output of DebugView, when the PostCreate is called first time,
> before leaving the PostCreate, I can see the
> pData->Iopb->TargetFileObject->FileName has been changed to the temp file
> name, the pData->IoStatus.Status is STATUS_REPARSE and
> pData->IoStatus.Information is IO_REPARSE,
> But these changes seem not to take effect, the PostCreate is called again
> and again, pData->Iopb->TargetFileObject->FileName is the stub file name
> when it is called.
>
> 2012/11/15
>
> Windbg seems to be the choice for many on this list.
>>
>> It looks like the number of retries is something controlled by the
>> application. However, I wouldn’t focus too much on the number of retries.
>> I’m thinking that if the files would be restored correctly then the
>> application shouldn’t see any failure and presumably it wouldn’t retry a
>> successful create.
>>
>> Thanks,
>> Alex.
>>
>> —
>> NTFSD is sponsored by OSR
>>
>> For our schedule of debugging and file system seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
>