IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it.
I’m using IoCreateStreamFileObjectLite to create that object, initialize
it with my data, roll a create IRP and call file system with it. First
of all, I hope that this doesn’t violate any rules and I can do it.
Second question is how many times do I have to call ObDereferenceObject
on this FO (assuming that create has succeeded)? Currently I do it twice
(one for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir

OK, I should call ObDeref just once :slight_smile: Everything else seem to work
fine (so far)

Vladimir

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 1:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it.
I’m using IoCreateStreamFileObjectLite to create that object, initialize
it with my data, roll a create IRP and call file system with it. First
of all, I hope that this doesn’t violate any rules and I can do it.
Second question is how many times do I have to call ObDereferenceObject
on this FO (assuming that create has succeeded)? Currently I do it twice
(one for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You only need to call ObDereferenceObject once. However, I do send a CLEANUP
IRP to the FSD if the CREATE succeeded before calling ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m using
IoCreateStreamFileObjectLite to create that object, initialize it with my
data, roll a create IRP and call file system with it. First of all, I hope
that this doesn’t violate any rules and I can do it. Second question is how
many times do I have to call ObDereferenceObject on this FO (assuming that
create has succeeded)? Currently I do it twice (one for Create IRP and
another for IoCreateStream) and system doesn’t complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP
IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using
IoCreateStreamFileObjectLite to create that object, initialize it with
my
data, roll a create IRP and call file system with it. First of all, I
hope
that this doesn’t violate any rules and I can do it. Second question is
how
many times do I have to call ObDereferenceObject on this FO (assuming
that
create has succeeded)? Currently I do it twice (one for Create IRP and
another for IoCreateStream) and system doesn’t complain (I do use
Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object table
(this is a standard sequense). Upon successful insertion the routin will get
a valid handle for the object. Immediately dfter that the
IoCreateStreamFileObject routine closes the handle. As this is the only
handle for the file object, IRP_MJ_CLEANUP request is sent down to file
system driver stack. But the underlying file system has not completed
IRP_MJ_CREATE operation for this file object yet, so the cleanup request is
ignored (file systems in checked build bark on such requests). After create
request has successfuly completed, you MUST send clenup request for this
file object.

The IoCreateStreamFileObjectLite routine does not insert created file object
into handle table. So, handle is never created for such an object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems with
this or that file system (FastFat, NTFS, DFS, SR, redirectors, etc.).
Besides, some variables and routines neccessary for rolling a create request
are not exported at all. You’d better go with shadow devices which are much
easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in message
news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP
IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using
IoCreateStreamFileObjectLite to create that object, initialize it with
my
data, roll a create IRP and call file system with it. First of all, I
hope
that this doesn’t violate any rules and I can do it. Second question is
how
many times do I have to call ObDereferenceObject on this FO (assuming
that
create has succeeded)? Currently I do it twice (one for Create IRP and
another for IoCreateStream) and system doesn’t complain (I do use
Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table
(this is a standard sequense). Upon successful insertion the routin will
get
a valid handle for the object. Immediately dfter that the
IoCreateStreamFileObject routine closes the handle. As this is the only
handle for the file object, IRP_MJ_CLEANUP request is sent down to file
system driver stack. But the underlying file system has not completed
IRP_MJ_CREATE operation for this file object yet, so the cleanup request
is
ignored (file systems in checked build bark on such requests). After
create
request has successfuly completed, you MUST send clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object
into handle table. So, handle is never created for such an object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with
this or that file system (FastFat, NTFS, DFS, SR, redirectors, etc.).
Besides, some variables and routines neccessary for rolling a create
request
are not exported at all. You’d better go with shadow devices which are
much
easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message
news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP
IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using
IoCreateStreamFileObjectLite to create that object, initialize it with
my
data, roll a create IRP and call file system with it. First of all, I
hope
that this doesn’t violate any rules and I can do it. Second question is
how
many times do I have to call ObDereferenceObject on this FO (assuming
that
create has succeeded)? Currently I do it twice (one for Create IRP and
another for IoCreateStream) and system doesn’t complain (I do use
Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

… and how in the world can I open file by its ID from the User Mode,
so I can write tests? SDK is silent on that. At least, it’s not
something that can be easily found :slight_smile:

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 7:11 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You can’t do it using Win32 CreateFile, you need to use NtCreateFile.

Alexei.

“Vladimir Chtchetkine” wrote in message
news:xxxxx@ntfsd…
… and how in the world can I open file by its ID from the User Mode,
so I can write tests? SDK is silent on that. At least, it’s not
something that can be easily found :slight_smile:

> I already figured out that I need to deref it just once. Besides, I

switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

You can also use ObCreateObject for creating a brand
new file object, then fill the members.
File name must be allocated using ExAllocatePool.
Then use ObInsertObject to insert it into the handle table.

The file object created that way should have exactly
two references, one for the handle and one for the file object itself),
except for the case when ObInsertObject fails
(it will dereference the file object then).

If you dereference the last FO’s reference, the I/O manager
will roll the cleanup IRP.

If you want to know how much references your file object has
(e.g. your stream file object), look at the address [FileObject - 0x18].
This is the ULONG which contains the number of references
(Of course for debug purposes only).

L.

----- Original Message -----
From: “Vladimir Chtchetkine”
To: “Windows File Systems Devs Interest List”
Sent: Thursday, February 26, 2004 12:02 AM
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP
IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using
IoCreateStreamFileObjectLite to create that object, initialize it with
my
data, roll a create IRP and call file system with it. First of all, I
hope
that this doesn’t violate any rules and I can do it. Second question is
how
many times do I have to call ObDereferenceObject on this FO (assuming
that
create has succeeded)? Currently I do it twice (one for Create IRP and
another for IoCreateStream) and system doesn’t complain (I do use
Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@volny.cz
To unsubscribe send a blank email to xxxxx@lists.osr.com

Vladimir,

The approach you have suggested is the simplest (using another IRP as a
“template”) because once you try to build the entire thing yourself it
is really rather rigid. It sounds much like you’ve been lucky.

It has been many, many years since we wrote code to do this, but I
distinctly remember the reaction when the process was done was amazement
at how rigid the whole process was (there was not much lattitude in what
you could do if you wanted it to work in all situations.) The more
restricted the use you make of your code, the less likely you are to see
problems (my recollection is that the LAST problem we saw was related to
using the same create code over one of the redirectors. Something
subtle, but vital to correct behavior.)

As far as I can tell, IRP_MJ_CREATE comes from one place in the OS,
which also leads to the observation that nobody else is going to expect
anything EXCEPT what that one code path generates.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 10:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

Why would you need this? You could cancel the IRP after calling the FSD if you
decide you need to.

Shadow device technique is no help here. What I’m trying to do is to get a path to
a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD.


Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32 developers.
Alfa File Monitor - File monitoring library for Win32 developers.

Dejan:

I need to know file path before passing create IRP down to the FSD
because my system can have this file opened at that time (internally, to
do some “dirty work” on it). So, before I pass that IRP down I need to
cancel whatever I do with that file and close it so there will be no
sharing/access violations when FSD handles this create.

Regards,

Vladimir

-----Original Message-----
From: Dejan Maksimovic [mailto:xxxxx@alfasp.com]
Sent: Thursday, February 26, 2004 4:43 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Why would you need this? You could cancel the IRP after calling the
FSD if you
decide you need to.

Shadow device technique is no help here. What I’m trying to do is to
get a path to
a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD.


Kind regards, Dejan M. MVP for DDK
http://www.alfasp.com E-mail: xxxxx@alfasp.com
Alfa Transparent File Encryptor - Transparent file encryption services.
Alfa File Protector - File protection and hiding library for Win32
developers.
Alfa File Monitor - File monitoring library for Win32 developers.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Well, I finally made “shadow” approach working for open by ID. It turned
out that (at least on my XP-SP1) you don’t really need related FO. It’s
enough to format FileName.Buffer in “<id>” format (just as Tony
suggested). Without leading ‘' it will fail if there is no related FO.
So, the only question remains starting with which version/SP you don’t
need related FO to open a file by ID? I don’t have W2K box available at
the moment so I can’t check it right now…

As far as rolling create IRPs go… Unfortunately, I can’t avoid it
completely. The problem is that I have to “redirect” create IRPs that
are coming to my device to completely different device stack. So, if
that “another” stack requires more stack locations that are left in the
IRP that I need to redirect I have no other choice but to create a
“duplicate” IRP with enough stack locations, send it there and complete
the original IRP with whatever was the result of rolled IRP. In
production system I immediately adjust number of stack locations needed
for my device’s stack so, most likely I will not get into “not enough
stack” condition anymore. But in testing I always keep my stack as low
as possible so I roll create IRPs for each and every create that my
device receives. I tested this internally and at all plugfests and
everything was working just fine. So, may be in my particular case all
those confusions with create IRPs eliminated each other (“even number of
bugs” theory).
And yes, I know about STATUS_REPARSE :slight_smile: But unfortunately, if I return
it I’m loosing track of the FileObject that I need to monitor after
create. There is simply no reliable way how you can sync back FO for
which you initially returned STATUS_REPARSE. In 99% you will see the
same FO coming to the stack to which you “reparsed” it. But in 1%
another FO will be allocated.

Regards,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Thursday, February 26, 2004 4:21 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

The approach you have suggested is the simplest (using another IRP as a
“template”) because once you try to build the entire thing yourself it
is really rather rigid. It sounds much like you’ve been lucky.

It has been many, many years since we wrote code to do this, but I
distinctly remember the reaction when the process was done was amazement
at how rigid the whole process was (there was not much lattitude in what
you could do if you wanted it to work in all situations.) The more
restricted the use you make of your code, the less likely you are to see
problems (my recollection is that the LAST problem we saw was related to
using the same create code over one of the redirectors. Something
subtle, but vital to correct behavior.)

As far as I can tell, IRP_MJ_CREATE comes from one place in the OS,
which also leads to the observation that nobody else is going to expect
anything EXCEPT what that one code path generates.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 10:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You may try to pass related file objects pointer in the path for the shadow
device. For example,
??\MyShadowDevice\RelatedOpen<relatedfileobjectpointer><id>. In your
shadow device’s IRP_MJ_CREATE dispatch routine you will have to set related
file object pointer in file object before passing the request to the file
system. This way you may be sure that file system will get a file object
setup as expected. As Tony mentioned, absolute opens by ID may not be
supported on older OSes, and if some of some of FSDs do not check related
file object, it does not mean that all FSDs will not try to do that.

–htfv

“Vladimir Chtchetkine” wrote in message
news:xxxxx@ntfsd…
Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table
(this is a standard sequense). Upon successful insertion the routin will
get
a valid handle for the object. Immediately dfter that the
IoCreateStreamFileObject routine closes the handle. As this is the only
handle for the file object, IRP_MJ_CLEANUP request is sent down to file
system driver stack. But the underlying file system has not completed
IRP_MJ_CREATE operation for this file object yet, so the cleanup request
is
ignored (file systems in checked build bark on such requests). After
create
request has successfuly completed, you MUST send clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object
into handle table. So, handle is never created for such an object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with
this or that file system (FastFat, NTFS, DFS, SR, redirectors, etc.).
Besides, some variables and routines neccessary for rolling a create
request
are not exported at all. You’d better go with shadow devices which are
much
easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message
news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP
IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using
IoCreateStreamFileObjectLite to create that object, initialize it with
my
data, roll a create IRP and call file system with it. First of all, I
hope
that this doesn’t violate any rules and I can do it. Second question is
how
many times do I have to call ObDereferenceObject on this FO (assuming
that
create has succeeded)? Currently I do it twice (one for Create IRP and
another for IoCreateStream) and system doesn’t complain (I do use
Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Generally, I would trust a driver, that rolls an IRP_MJ_CREATE IRP from
scratch, no more than to one that hooks system services or routines. As to
sending requests to other device stacks… Shadow devices gonna help you.

–htfv

“Vladimir Chtchetkine” wrote in message
news:xxxxx@ntfsd…
Well, I finally made “shadow” approach working for open by ID. It turned
out that (at least on my XP-SP1) you don’t really need related FO. It’s
enough to format FileName.Buffer in “<id>” format (just as Tony
suggested). Without leading ‘' it will fail if there is no related FO.
So, the only question remains starting with which version/SP you don’t
need related FO to open a file by ID? I don’t have W2K box available at
the moment so I can’t check it right now…

As far as rolling create IRPs go… Unfortunately, I can’t avoid it
completely. The problem is that I have to “redirect” create IRPs that
are coming to my device to completely different device stack. So, if
that “another” stack requires more stack locations that are left in the
IRP that I need to redirect I have no other choice but to create a
“duplicate” IRP with enough stack locations, send it there and complete
the original IRP with whatever was the result of rolled IRP. In
production system I immediately adjust number of stack locations needed
for my device’s stack so, most likely I will not get into “not enough
stack” condition anymore. But in testing I always keep my stack as low
as possible so I roll create IRPs for each and every create that my
device receives. I tested this internally and at all plugfests and
everything was working just fine. So, may be in my particular case all
those confusions with create IRPs eliminated each other (“even number of
bugs” theory).
And yes, I know about STATUS_REPARSE :slight_smile: But unfortunately, if I return
it I’m loosing track of the FileObject that I need to monitor after
create. There is simply no reliable way how you can sync back FO for
which you initially returned STATUS_REPARSE. In 99% you will see the
same FO coming to the stack to which you “reparsed” it. But in 1%
another FO will be allocated.

Regards,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Thursday, February 26, 2004 4:21 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

The approach you have suggested is the simplest (using another IRP as a
“template”) because once you try to build the entire thing yourself it
is really rather rigid. It sounds much like you’ve been lucky.

It has been many, many years since we wrote code to do this, but I
distinctly remember the reaction when the process was done was amazement
at how rigid the whole process was (there was not much lattitude in what
you could do if you wanted it to work in all situations.) The more
restricted the use you make of your code, the less likely you are to see
problems (my recollection is that the LAST problem we saw was related to
using the same create code over one of the redirectors. Something
subtle, but vital to correct behavior.)

As far as I can tell, IRP_MJ_CREATE comes from one place in the OS,
which also leads to the observation that nobody else is going to expect
anything EXCEPT what that one code path generates.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 10:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

No, shadow devices are not going to help me in this particular case (I’m
not talking about ID now, I’m talking about stack locations). Or
solution with shadow devs will be even more complex than rolling your
own create…

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Thursday, February 26, 2004 10:23 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Generally, I would trust a driver, that rolls an IRP_MJ_CREATE IRP from
scratch, no more than to one that hooks system services or routines. As
to
sending requests to other device stacks… Shadow devices gonna help
you.

–htfv

“Vladimir Chtchetkine” wrote in
message
news:xxxxx@ntfsd…
Well, I finally made “shadow” approach working for open by ID. It turned
out that (at least on my XP-SP1) you don’t really need related FO. It’s
enough to format FileName.Buffer in “<id>” format (just as Tony
suggested). Without leading ‘' it will fail if there is no related FO.
So, the only question remains starting with which version/SP you don’t
need related FO to open a file by ID? I don’t have W2K box available at
the moment so I can’t check it right now…

As far as rolling create IRPs go… Unfortunately, I can’t avoid it
completely. The problem is that I have to “redirect” create IRPs that
are coming to my device to completely different device stack. So, if
that “another” stack requires more stack locations that are left in the
IRP that I need to redirect I have no other choice but to create a
“duplicate” IRP with enough stack locations, send it there and complete
the original IRP with whatever was the result of rolled IRP. In
production system I immediately adjust number of stack locations needed
for my device’s stack so, most likely I will not get into “not enough
stack” condition anymore. But in testing I always keep my stack as low
as possible so I roll create IRPs for each and every create that my
device receives. I tested this internally and at all plugfests and
everything was working just fine. So, may be in my particular case all
those confusions with create IRPs eliminated each other (“even number of
bugs” theory).
And yes, I know about STATUS_REPARSE :slight_smile: But unfortunately, if I return
it I’m loosing track of the FileObject that I need to monitor after
create. There is simply no reliable way how you can sync back FO for
which you initially returned STATUS_REPARSE. In 99% you will see the
same FO coming to the stack to which you “reparsed” it. But in 1%
another FO will be allocated.

Regards,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Thursday, February 26, 2004 4:21 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

The approach you have suggested is the simplest (using another IRP as a
“template”) because once you try to build the entire thing yourself it
is really rather rigid. It sounds much like you’ve been lucky.

It has been many, many years since we wrote code to do this, but I
distinctly remember the reaction when the process was done was amazement
at how rigid the whole process was (there was not much lattitude in what
you could do if you wanted it to work in all situations.) The more
restricted the use you make of your code, the less likely you are to see
problems (my recollection is that the LAST problem we saw was related to
using the same create code over one of the redirectors. Something
subtle, but vital to correct behavior.)

As far as I can tell, IRP_MJ_CREATE comes from one place in the OS,
which also leads to the observation that nobody else is going to expect
anything EXCEPT what that one code path generates.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 10:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

If setting stack locations counter during creation of a shadow device object
is more complex than rolling own create, then I think you have something
misunderstood in the conception of shadow devices or may be device stacks.
What is it so specific in your particular case? Perhaps, you have another
poorly designed driver?

–htfv

“Vladimir Chtchetkine” wrote in message
news:xxxxx@ntfsd…
No, shadow devices are not going to help me in this particular case (I’m
not talking about ID now, I’m talking about stack locations). Or
solution with shadow devs will be even more complex than rolling your
own create…

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Thursday, February 26, 2004 10:23 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Generally, I would trust a driver, that rolls an IRP_MJ_CREATE IRP from
scratch, no more than to one that hooks system services or routines. As
to
sending requests to other device stacks… Shadow devices gonna help
you.

–htfv

“Vladimir Chtchetkine” wrote in
message
news:xxxxx@ntfsd…
Well, I finally made “shadow” approach working for open by ID. It turned
out that (at least on my XP-SP1) you don’t really need related FO. It’s
enough to format FileName.Buffer in “<id>” format (just as Tony
suggested). Without leading ‘' it will fail if there is no related FO.
So, the only question remains starting with which version/SP you don’t
need related FO to open a file by ID? I don’t have W2K box available at
the moment so I can’t check it right now…

As far as rolling create IRPs go… Unfortunately, I can’t avoid it
completely. The problem is that I have to “redirect” create IRPs that
are coming to my device to completely different device stack. So, if
that “another” stack requires more stack locations that are left in the
IRP that I need to redirect I have no other choice but to create a
“duplicate” IRP with enough stack locations, send it there and complete
the original IRP with whatever was the result of rolled IRP. In
production system I immediately adjust number of stack locations needed
for my device’s stack so, most likely I will not get into “not enough
stack” condition anymore. But in testing I always keep my stack as low
as possible so I roll create IRPs for each and every create that my
device receives. I tested this internally and at all plugfests and
everything was working just fine. So, may be in my particular case all
those confusions with create IRPs eliminated each other (“even number of
bugs” theory).
And yes, I know about STATUS_REPARSE :slight_smile: But unfortunately, if I return
it I’m loosing track of the FileObject that I need to monitor after
create. There is simply no reliable way how you can sync back FO for
which you initially returned STATUS_REPARSE. In 99% you will see the
same FO coming to the stack to which you “reparsed” it. But in 1%
another FO will be allocated.

Regards,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Thursday, February 26, 2004 4:21 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

The approach you have suggested is the simplest (using another IRP as a
“template”) because once you try to build the entire thing yourself it
is really rather rigid. It sounds much like you’ve been lucky.

It has been many, many years since we wrote code to do this, but I
distinctly remember the reaction when the process was done was amazement
at how rigid the whole process was (there was not much lattitude in what
you could do if you wanted it to work in all situations.) The more
restricted the use you make of your code, the less likely you are to see
problems (my recollection is that the LAST problem we saw was related to
using the same create code over one of the redirectors. Something
subtle, but vital to correct behavior.)

As far as I can tell, IRP_MJ_CREATE comes from one place in the OS,
which also leads to the observation that nobody else is going to expect
anything EXCEPT what that one code path generates.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 10:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Alexey:

Well, it’s all about “probabilities” isn’t it? My (as you put it)
“poorly designed driver” has been extensively tested for last 5 years,
it’s a shipping product for last 4 years and there were no bugs detected
(neither by testing nor by customers) related to rolling my own create
IRP. There were (and still are) others, but not this one. Which leads me
to the conclusion that even if there is a bug in that piece of code,
probability that it’s going to be hit is substantially lower than what
is needed to motivate me to redesign this code branch. Especially, in
the shipping product. May be, even as it is, it’s not a bad design after
all? :wink:
I’m not saying that shadow device approach is not an option at all in
this case. But certainly it’s not that simple as just setting proper
stack location counter.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Thursday, February 26, 2004 1:36 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

If setting stack locations counter during creation of a shadow device
object
is more complex than rolling own create, then I think you have something

misunderstood in the conception of shadow devices or may be device
stacks.
What is it so specific in your particular case? Perhaps, you have
another
poorly designed driver?

–htfv

“Vladimir Chtchetkine” wrote in
message
news:xxxxx@ntfsd…
No, shadow devices are not going to help me in this particular case (I’m
not talking about ID now, I’m talking about stack locations). Or
solution with shadow devs will be even more complex than rolling your
own create…

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Thursday, February 26, 2004 10:23 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Generally, I would trust a driver, that rolls an IRP_MJ_CREATE IRP from
scratch, no more than to one that hooks system services or routines. As
to
sending requests to other device stacks… Shadow devices gonna help
you.

–htfv

“Vladimir Chtchetkine” wrote in
message
news:xxxxx@ntfsd…
Well, I finally made “shadow” approach working for open by ID. It turned
out that (at least on my XP-SP1) you don’t really need related FO. It’s
enough to format FileName.Buffer in “<id>” format (just as Tony
suggested). Without leading ‘' it will fail if there is no related FO.
So, the only question remains starting with which version/SP you don’t
need related FO to open a file by ID? I don’t have W2K box available at
the moment so I can’t check it right now…

As far as rolling create IRPs go… Unfortunately, I can’t avoid it
completely. The problem is that I have to “redirect” create IRPs that
are coming to my device to completely different device stack. So, if
that “another” stack requires more stack locations that are left in the
IRP that I need to redirect I have no other choice but to create a
“duplicate” IRP with enough stack locations, send it there and complete
the original IRP with whatever was the result of rolled IRP. In
production system I immediately adjust number of stack locations needed
for my device’s stack so, most likely I will not get into “not enough
stack” condition anymore. But in testing I always keep my stack as low
as possible so I roll create IRPs for each and every create that my
device receives. I tested this internally and at all plugfests and
everything was working just fine. So, may be in my particular case all
those confusions with create IRPs eliminated each other (“even number of
bugs” theory).
And yes, I know about STATUS_REPARSE :slight_smile: But unfortunately, if I return
it I’m loosing track of the FileObject that I need to monitor after
create. There is simply no reliable way how you can sync back FO for
which you initially returned STATUS_REPARSE. In 99% you will see the
same FO coming to the stack to which you “reparsed” it. But in 1%
another FO will be allocated.

Regards,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Thursday, February 26, 2004 4:21 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

The approach you have suggested is the simplest (using another IRP as a
“template”) because once you try to build the entire thing yourself it
is really rather rigid. It sounds much like you’ve been lucky.

It has been many, many years since we wrote code to do this, but I
distinctly remember the reaction when the process was done was amazement
at how rigid the whole process was (there was not much lattitude in what
you could do if you wanted it to work in all situations.) The more
restricted the use you make of your code, the less likely you are to see
problems (my recollection is that the LAST problem we saw was related to
using the same create code over one of the redirectors. Something
subtle, but vital to correct behavior.)

As far as I can tell, IRP_MJ_CREATE comes from one place in the OS,
which also leads to the observation that nobody else is going to expect
anything EXCEPT what that one code path generates.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 10:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Tony!

Actually, I need to do some more research on “if related FO is really
required for open by ID on my XP-SP1” (I just noticed that some
experiments that I did were not so clean). But, most likely, I will
follow your advice and use one of the FOs that are “around”. Is any FO
good for that? I mean does it matter if this “related FO” is a directory
or file? Has its handle been closed (i.e. cleanup IRP has already been
sent for this file)? Well, after all, I guess, I can use related FO that
I got in the IRP that I’m processing! :slight_smile:

If you don’t mind, can you give some 5 top reasons why rolling create
IRP is “that hard”? I’ve been doing this a lot and have never had any
problems. May be it was just a “luck”? Also my create IRPs are not 100%
pure create IRPs. I do reuse some params (like SecurityContext) that I
receive with the original IRP. Also I never go through the whole
completion process as I free that IRP in the completion routine and
return STATUS_MORE_PROCESSING_REQUIRED.

TIA,

Vladimir

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Wednesday, February 25, 2004 6:47 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Vladimir,

Rolling create IRPs is, in my observation, the most difficult IRP to
construct and send. It can be done, but it is a tremendous amount of
work…

Open by ID does require a related file object, but this can be a file
object for any directory (possibly any file, I’d have to verify that) on
the volume. So, if you happen to keep a file object around for the root
directory (for example) then you can just use that. Indeed, more recent
versions of Windows appear to allow absolute file ID paths in NTFS (so
the first ‘character’ of the name is L’\’ and then the next 8 or 16
bytes are the file ID or object ID) so you don’t even need related file
objects in those cases. Older versions of Windows DID require relative
opens, however. And looking at the 3790 CDFS (which supports open by
file ID) it requires the relative file object mechanism, although my
quick glance suggests that it doesn’t look at the related file object -
at least not in CDFS. Did you actually try an IRP_MJ_CREATE with a null
related file object and a valid file ID and have it fail?

The purpose of IoCreateStreamFileObject* is to create a file object so
the file system can use it; if you want to use it for a create IRP you
can do so. See FastFat’s use to represent the mapped directory
(cachesup.c) for an example. There is also a comparable example in CDFS
(cachesup.c again). They really do just get back a pristine, clean file
object and you can use it in the manner you have described.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Chtchetkine
Sent: Wednesday, February 25, 2004 9:11 PM
To: ntfsd redirect
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Thanks, Alexey!

So, I will switch back to “lite” version and roll cleanup IRP.

Shadow device technique is no help here. What I’m trying to do is to get
a path to a file for which I see “open by ID” create IRP. And I need
this path before I call underlying FSD. I tried to “shadow” it but it
doesn’t work for “open by ID”. At least it’s not that easy to make it
work. It looks like “open by ID” requires related FO + ID in
FileObject.FileName. At least that’s the picture I see in my filter for
“open by ID” IRPs. Maintaining related FOs on shadow device seem to be
bigger headache than just rolling cleanup IRP :slight_smile: As far as rolling
create IRP… I never had a single problem with that (well, when once
I’ve got it working). What confuses me, though, is using FO obtained via
IoCreateStreamFileObject in that IRP. Although FO seem to be not
initialized when it comes from IoCreateStream (no FsContexts, no Vpb, no
nothing, basically) but somehow I’m not certain if I can use it in
create IRP.

Regards,

Vladimir

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Wednesday, February 25, 2004 4:02 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IoCreateStreamFileObjectLite and create IRP

Hate to tell this, but you might be wrong. The IoCreateStreamFileObject
routine creates a file object and then inserts it into process object
table (this is a standard sequense). Upon successful insertion the
routin will get a valid handle for the object. Immediately dfter that
the IoCreateStreamFileObject routine closes the handle. As this is the
only handle for the file object, IRP_MJ_CLEANUP request is sent down to
file system driver stack. But the underlying file system has not
completed IRP_MJ_CREATE operation for this file object yet, so the
cleanup request is ignored (file systems in checked build bark on such
requests). After create request has successfuly completed, you MUST send
clenup request for this

file object.

The IoCreateStreamFileObjectLite routine does not insert created file
object into handle table. So, handle is never created for such an
object.

I worked a lot on rolling IRP_MJ_CREATE requests some time ago. While I
cannot say that it is impossible, it often causes all kinds of problems
with this or that file system (FastFat, NTFS, DFS, SR, redirectors,
etc.).
Besides, some variables and routines neccessary for rolling a create
request are not exported at all. You’d better go with shadow devices
which are much easier and (what’s more important) stable solution.

–htfv

“Vladimir Chtchetkine” wrote in
message news:xxxxx@ntfsd…
Thanks, Ted!

I already figured out that I need to deref it just once. Besides, I
switched to IoCreateStreamFileObject instead of “lite” one so (according
to the doc) cleanup IRP will be sent automatically.

Regards,

Vladimir

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, February 25, 2004 2:39 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IoCreateStreamFileObjectLite and create IRP

You only need to call ObDereferenceObject once. However, I do send a
CLEANUP IRP to the FSD if the CREATE succeeded before calling
ObDereferenceObject.
You need to do this to reset sharing info from the successful CREATE.

/ted

-----Original Message-----
From: Vladimir Chtchetkine [mailto:xxxxx@borland.com]
Sent: Wednesday, February 25, 2004 4:40 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IoCreateStreamFileObjectLite and create IRP

Hi everyone!

I need to roll my own create IRP and I need a FileObject for it. I’m
using IoCreateStreamFileObjectLite to create that object, initialize it
with my data, roll a create IRP and call file system with it. First of
all, I hope that this doesn’t violate any rules and I can do it. Second
question is how many times do I have to call ObDereferenceObject on this
FO (assuming that create has succeeded)? Currently I do it twice (one
for Create IRP and another for IoCreateStream) and system doesn’t
complain (I do use Verifier).
So, am I OK with what I’m doing?

TIA,

Vladimir


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as:
xxxxx@borland.com
To unsubscribe send a blank email to xxxxx@lists.osr.com