… is it real? I mean I really want to use it, but it seems that only filter manager provides an API for it and linking to fltmgr library just to use push locks seems to be excessive. So, I wander, is it a WDK bug (not exposing ExXxx push lock API), or it is yet another undocumented feature that just leaked through filter manager, because advanced FCB header has a push lock in it?
Why not use executive resources? These are only for FltManager, and you
shouldn’t be using them elsewhere.
–
Don Burn (MVP, Windows DDK)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
wrote in message news:xxxxx@ntdev…
> … is it real? I mean I really want to use it, but it seems that only
> filter manager provides an API for it and linking to fltmgr library just
> to use push locks seems to be excessive. So, I wander, is it a WDK bug
> (not exposing ExXxx push lock API), or it is yet another undocumented
> feature that just leaked through filter manager, because advanced FCB
> header has a push lock in it?
>
>
> Information from ESET NOD32 Antivirus, version of virus
> signature database 4051 (20090504)
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
Information from ESET NOD32 Antivirus, version of virus signature database 4051 (20090504)
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
> or it is yet another undocumented feature that just
leaked through filter manager
Push locks is a undocumented synchronization objects. It is object for synchronization only (no object-three manager support). Push locks are capable of being acquired in both shared and exclusive mode. Properties include:
* cannot be acquired recursively
* small size (sizeof(PVOID))
* …
Why do you think that you need to use this?
As a general rule of thumb, you should not optimize by changing which locking primatives you use until you have optimized everything else (including restricting the code running under the lock to the minimum possible subset) and profiling shows that the lock is a clear bottleneck.
The majority of all performance problems with locks that you are likely to see are a result of misusing locks in general (i.e. running very lengthy code under the lock), and not a problem with the specific lock implementation in question.
- S
-----Original Message-----
From: xxxxx@gmail.com
Sent: Monday, May 04, 2009 07:07
To: Windows System Software Devs Interest List
Subject: [ntdev] EX_PUSH_LOCK
… is it real? I mean I really want to use it, but it seems that only filter manager provides an API for it and linking to fltmgr library just to use push locks seems to be excessive. So, I wander, is it a WDK bug (not exposing ExXxx push lock API), or it is yet another undocumented feature that just leaked through filter manager, because advanced FCB header has a push lock in it?
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Well, the doc claims that push locks provide much better than exec. resource performance on shared locking. In my driver I have multiple lists and AVL tables, that are concurrently accessed and are on the performance sensitive path. The problem is that I can’t reliably predict the size of those lists and tables to make an informed decision on which locking primitive to use: exec. resource, or fast mutex. But what I know is that majority of the time I need a shared lock to protect those lists / tables (I mean, 99% of operations are lookups). So, push lock seems to fit what I need better than fast mutex and exec. resource. Currently I use exec. resource (just for the sake of scalability), so switching to push locks seems to be very natural.
Personally I wouldn’t go about changing the primitives I use unless I had some clear performance data that showed the cost of acquiring & releasing a lock was my bottleneck. I believe that’s been the suggestion by others as well.
What data leads you to believe that ERESOURCE is your bottleneck?
-p
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Monday, May 04, 2009 9:01 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] EX_PUSH_LOCK
Well, the doc claims that push locks provide much better than exec. resource performance on shared locking. In my driver I have multiple lists and AVL tables, that are concurrently accessed and are on the performance sensitive path. The problem is that I can’t reliably predict the size of those lists and tables to make an informed decision on which locking primitive to use: exec. resource, or fast mutex. But what I know is that majority of the time I need a shared lock to protect those lists / tables (I mean, 99% of operations are lookups). So, push lock seems to fit what I need better than fast mutex and exec. resource. Currently I use exec. resource (just for the sake of scalability), so switching to push locks seems to be very natural.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
When I switch to fast mutex from eresource I see about 5-10% perf improvements on low and medium loads with small lists and tables. On heavy loads perf drops (of course) because of concurrency issues. Majority of the time (in the real life) loads are low and tables are small, but that’s the corner cases of high loads and big tables that my driver must handle, that make me using eresources. Again, there is nothing fancy going on under the lock: just AVL table lookups, where keys are strings. So, the only thing to optimize here is to optimize RtlCompareUnicodeString, which doesn’t give me any leverage.
So just include ntifs.h and be done with it. The worst case for this
is that you have one compilation module that includes ntifs.h to get
the push locks and to export locking operations to your other modules.
Build it, test and measure it, and see if it helps.
Mark Roddy
On Mon, May 4, 2009 at 1:32 PM, wrote:
> When I switch to fast mutex from eresource I see about 5-10% perf improvements on low and medium loads with small lists and tables. On heavy loads perf drops (of course) because of concurrency issues. Majority of the time (in the real life) loads are low and tables are small, but that’s the corner cases of high loads and big tables that my driver must handle, that make me using eresources. Again, there is nothing fancy going on under the lock: just AVL table lookups, where keys are strings. So, the only thing to optimize here is to optimize RtlCompareUnicodeString, which doesn’t give me any leverage.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
Mark: that’s the problem!
Push lock exists in FltMgr domain only, and I’m not willing to link to FltMgr library just for the sake of push locks.
*
Well, I guess, I’ve got an answer: “push locks are not documented, deal with it”. Fair enough for me 
It is not just the ntifs.h it is the flt stuff and having the fltmgr.sys
running, personally I someone required this and it was not a filesystem
filter I would wonder what was going on. Also, until someone from
Microsoft says otherwise, how do we know the locking does not have a
dependancy (now or in the future) that expects this to be a file system
filter. I know it is improbable, but drivers need to be developed
defensively.
–
Don Burn (MVP, Windows DDK)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
“Mark Roddy” wrote in message news:xxxxx@ntdev…
So just include ntifs.h and be done with it. The worst case for this
is that you have one compilation module that includes ntifs.h to get
the push locks and to export locking operations to your other modules.
Build it, test and measure it, and see if it helps.
Mark Roddy
On Mon, May 4, 2009 at 1:32 PM, wrote:
> When I switch to fast mutex from eresource I see about 5-10% perf
> improvements on low and medium loads with small lists and tables. On heavy
> loads perf drops (of course) because of concurrency issues. Majority of
> the time (in the real life) loads are low and tables are small, but that’s
> the corner cases of high loads and big tables that my driver must handle,
> that make me using eresources. Again, there is nothing fancy going on
> under the lock: just AVL table lookups, where keys are strings. So, the
> only thing to optimize here is to optimize RtlCompareUnicodeString, which
> doesn’t give me any leverage.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
Information from ESET NOD32 Antivirus, version of virus signature
database 4052 (20090504)
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
Information from ESET NOD32 Antivirus, version of virus signature database 4052 (20090504)
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
I would reverse-engineer the ExXxxPushLock routines and write my own with the same logic.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
> … is it real? I mean I really want to use it, but it seems that only filter manager provides an API for it and linking to fltmgr library just to use push locks seems to be excessive. So, I wander, is it a WDK bug (not exposing ExXxx push lock API), or it is yet another undocumented feature that just leaked through filter manager, because advanced FCB header has a push lock in it?
>
While I COMPLETELY AGREE that optimizing ones sync primitives is pretty far down on the list of “things that cause performance problems” it seems that most of you guys are being terribly DOUR about this.
Push Locks are excellent synchronization primitives. While ERESOURCES are nice, they are not exactly lightweight. In fact, I went looking to see if push locks were exported recently myself… I was sad to only find them in the Filter Manager.
Would somebody over at Microsoft PLEASE consider exporting push locks for driver dev use in the near future? They’ve been around since XP, are obviously stable, and provide VERY good performance when non-contended.
Peter
OSR
Push locks, at least the ones in the kernel, have had their perf characteristics tweaked over the past 2 releases. Tweaks which would have not been compatible with the previous behavior. Now, there is no reason we cannot have both a public and private version of the lock where one is more stable than then other…
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Monday, May 04, 2009 1:40 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] EX_PUSH_LOCK
While I COMPLETELY AGREE that optimizing ones sync primitives is pretty far down on the list of “things that cause performance problems” it seems that most of you guys are being terribly DOUR about this.
Push Locks are excellent synchronization primitives. While ERESOURCES are nice, they are not exactly lightweight. In fact, I went looking to see if push locks were exported recently myself… I was sad to only find them in the Filter Manager.
Would somebody over at Microsoft PLEASE consider exporting push locks for driver dev use in the near future? They’ve been around since XP, are obviously stable, and provide VERY good performance when non-contended.
Peter
OSR
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Live and learn.
Still, please consider the request? We don’t really have an optimized “lock on a local flag, devolve to a wait if there’s contention” kind of standard lock exported for driver use… not counting spinlocks, of course, which devolve to a busy wait.
I think this would be a very useful lock for driver devs, if somebody could just find the time to get it into the kit,
Peter
OSR
I completely agree that it would be useful, sorry if that was what came across. I was addressing the “they are stable” statement. I will spin up a conversation here to see what the push lock story is
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@osr.com
Sent: Monday, May 04, 2009 1:52 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] EX_PUSH_LOCK
Live and learn.
Still, please consider the request? We don’t really have an optimized “lock on a local flag, devolve to a wait if there’s contention” kind of standard lock exported for driver use… not counting spinlocks, of course, which devolve to a busy wait.
I think this would be a very useful lock for driver devs, if somebody could just find the time to get it into the kit,
Peter
OSR
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
On Mon, May 4, 2009 at 1:53 PM, Don Burn wrote:
> It is not just the ntifs.h it is the flt stuff and having the fltmgr.sys
> running,
It is a test to see if this affects performance (in a positive
manner). The answer may be ‘no’ in which case the hack can be
discarded. If on the other hand the answer is ‘yes’ then the OP can
figure out if the risk is acceptable. And then also the subsequent
issue of why push locks are boxed into fltmgr comes into play.
Mark Roddy