Driver Problems? Questions? Issues?
Put OSR's experience to work for you! Contact us for assistance with:
  • Creating the right design for your requirements
  • Reviewing your existing driver code
  • Analyzing driver reliability/performance issues
  • Custom training mixed with consulting and focused directly on your specific areas of interest/concern.
Check us out. OSR, the Windows driver experts.

Monthly Seminars at OSR Headquarters

East Coast USA
Windows Internals and SW Drivers, Dulles (Sterling) VA, 13 November 2017

Kernel Debugging & Crash Analysis for Windows, Nashua (Amherst) NH, 4 December 2017

Writing WDF Drivers I: Core Concepts, Nashua (Amherst) NH, 8 January 2018

WDF Drivers II: Advanced Implementation Techniques, Nashua (Amherst) NH, 15 January 2018


Go Back   OSR Online Lists > ntdev
Welcome, Guest
You must login to post to this list
  Message 1 of 17  
04 May 09 10:07
Vladimir Chtchetkine
xxxxxx@gmail.com
Join Date: 27 Jan 2006
Posts To This List: 41
EX_PUSH_LOCK

... is it real? I mean I really want to use it, but it seems that only filter manager provides an API for it and linking to fltmgr library just to use push locks seems to be excessive. So, I wander, is it a WDK bug (not exposing ExXxx push lock API), or it is yet another undocumented feature that just leaked through filter manager, because advanced FCB header has a push lock in it?
  Message 2 of 17  
04 May 09 10:27
Don Burn
xxxxxx@acm.org
Join Date:
Posts To This List: 3179
EX_PUSH_LOCK

Why not use executive resources? These are only for FltManager, and you shouldn't be using them elsewhere. -- Don Burn (MVP, Windows DDK) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr <xxxxx@gmail.com> wrote in message news:123513@ntdev... > .. is it real? I mean I really want to use it, but it seems that only > filter manager provides an API for it and linking to fltmgr library just > to use push locks seems to be excessive. So, I wander, is it a WDK bug > (not exposing ExXxx push lock API), or it is yet another undocumented > feature that just leaked through filter manager, because advanced FCB > header has a push lock in it? > > > __________ Information from ESET NOD32 Antivirus, version of virus > signature database 4051 (20090504) __________ <...excess quoted lines suppressed...> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4051 (20090504) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
  Message 3 of 17  
04 May 09 11:12
ntdev member 41880
xxxxxx@rambler.ru
Join Date:
Posts To This List: 22
EX_PUSH_LOCK

> or it is yet another undocumented feature that just > leaked through filter manager Push locks is a undocumented synchronization objects. It is object for synchronization only (no object-three manager support). Push locks are capable of being acquired in both shared and exclusive mode. Properties include: * cannot be acquired recursively * small size (sizeof(PVOID)) * ....
  Message 4 of 17  
04 May 09 11:32
Ken Johnson
xxxxxx@valhallalegends.com
Join Date: 24 Jul 2008
Posts To This List: 1022
EX_PUSH_LOCK

Why do you think that you need to use this? As a general rule of thumb, you should not optimize by changing which locki= ng primatives you use until you have optimized everything else (including r= estricting the code running under the lock to the minimum possible subset) = and profiling shows that the lock is a clear bottleneck. The majority of all performance problems with locks that you are likely to = see are a result of misusing locks in general (i.e. running very lengthy co= de under the lock), and not a problem with the specific lock implementation= in question. - S -----Original Message----- From: xxxxx@gmail.com <xxxxx@gmail.com> Sent: Monday, May 04, 2009 07:07 To: Windows System Software Devs Interest List <xxxxx@lists.osr.com> Subject: [ntdev] EX_PUSH_LOCK ... is it real? I mean I really want to use it, but it seems that only filt= er manager provides an API for it and linking to fltmgr library just to use= push locks seems to be excessive. So, I wander, is it a WDK bug (not expos= ing ExXxx push lock API), or it is yet another undocumented feature that ju= st leaked through filter manager, because advanced FCB header has a push lo= ck in it? --- NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.o= sronline.com/page.cfm?name=3DListServer
  Message 5 of 17  
04 May 09 12:02
Vladimir Chtchetkine
xxxxxx@gmail.com
Join Date: 27 Jan 2006
Posts To This List: 41
EX_PUSH_LOCK

Well, the doc claims that push locks provide much better than exec. resource performance on shared locking. In my driver I have multiple lists and AVL tables, that are concurrently accessed and are on the performance sensitive path. The problem is that I can't reliably predict the size of those lists and tables to make an informed decision on which locking primitive to use: exec. resource, or fast mutex. But what I know is that majority of the time I need a shared lock to protect those lists / tables (I mean, 99% of operations are lookups). So, push lock seems to fit what I need better than fast mutex and exec. resource. Currently I use exec. resource (just for the sake of scalability), so switching to push locks seems to be very natural.
  Message 6 of 17  
04 May 09 13:08
ntdev member 8437
xxxxxx@windows.microsoft.com
Join Date:
Posts To This List: 1404
EX_PUSH_LOCK

Personally I wouldn't go about changing the primitives I use unless I had s= ome clear performance data that showed the cost of acquiring & releasing a = lock was my bottleneck. I believe that's been the suggestion by others as = well. What data leads you to believe that ERESOURCE is your bottleneck? -p -----Original Message----- From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr= .com] On Behalf Of xxxxx@gmail.com Sent: Monday, May 04, 2009 9:01 AM To: Windows System Software Devs Interest List Subject: RE:[ntdev] EX_PUSH_LOCK Well, the doc claims that push locks provide much better than exec. resourc= e performance on shared locking. In my driver I have multiple lists and AVL= tables, that are concurrently accessed and are on the performance sensitiv= e path. The problem is that I can't reliably predict the size of those list= s and tables to make an informed decision on which locking primitive to use= : exec. resource, or fast mutex. But what I know is that majority of the ti= me I need a shared lock to protect those lists / tables (I mean, 99% of ope= rations are lookups). So, push lock seems to fit what I need better than fa= st mutex and exec. resource. Currently I use exec. resource (just for the s= ake of scalability), so switching to push locks seems to be very natural.=20 --- NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit:=20 http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.o= sronline.com/page.cfm?name=3DListServer
  Message 7 of 17  
04 May 09 13:33
Vladimir Chtchetkine
xxxxxx@gmail.com
Join Date: 27 Jan 2006
Posts To This List: 41
EX_PUSH_LOCK

When I switch to fast mutex from eresource I see about 5-10% perf improvements on low and medium loads with small lists and tables. On heavy loads perf drops (of course) because of concurrency issues. Majority of the time (in the real life) loads are low and tables are small, but that's the corner cases of high loads and big tables that my driver must handle, that make me using eresources. Again, there is nothing fancy going on under the lock: just AVL table lookups, where keys are strings. So, the only thing to optimize here is to optimize RtlCompareUnicodeString, which doesn't give me any leverage.
  Message 8 of 17  
04 May 09 13:43
Mark Roddy
xxxxxx@gmail.com
Join Date: 25 Feb 2000
Posts To This List: 4014
EX_PUSH_LOCK

So just include ntifs.h and be done with it. The worst case for this is that you have one compilation module that includes ntifs.h to get the push locks and to export locking operations to your other modules. Build it, test and measure it, and see if it helps. Mark Roddy On Mon, May 4, 2009 at 1:32 PM, <xxxxx@gmail.com> wrote: > When I switch to fast mutex from eresource I see about 5-10% perf improve= ments on low and medium loads with small lists and tables. On heavy loads p= erf drops (of course) because of concurrency issues. Majority of the time (= in the real life) loads are low and tables are small, but that's the corner= cases of high loads and big tables that my driver must handle, that make m= e using eresources. Again, there is nothing fancy going on under the lock: = just AVL table lookups, where keys are strings. So, the only thing to optim= ize here is to optimize RtlCompareUnicodeString, which doesn't give me any = leverage. > > --- > NTDEV is sponsored by OSR > > For our schedule of WDF, WDM, debugging and other seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at http://www= .osronline.com/page.cfm?name=3DListServer >
  Message 9 of 17  
04 May 09 13:52
Vladimir Chtchetkine
xxxxxx@gmail.com
Join Date: 27 Jan 2006
Posts To This List: 41
EX_PUSH_LOCK

Mark: that's the problem! Push lock exists in FltMgr domain only, and I'm not willing to link to FltMgr library just for the sake of push locks. * Well, I guess, I've got an answer: "push locks are not documented, deal with it". Fair enough for me :-)
  Message 10 of 17  
04 May 09 13:54
Don Burn
xxxxxx@acm.org
Join Date:
Posts To This List: 3179
EX_PUSH_LOCK

It is not just the ntifs.h it is the flt stuff and having the fltmgr.sys running, personally I someone required this and it was not a filesystem filter I would wonder what was going on. Also, until someone from Microsoft says otherwise, how do we know the locking does not have a dependancy (now or in the future) that expects this to be a file system filter. I know it is improbable, but drivers need to be developed defensively. -- Don Burn (MVP, Windows DDK) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr "Mark Roddy" <xxxxx@hollistech.com> wrote in message news:123529@ntdev... So just include ntifs.h and be done with it. The worst case for this is that you have one compilation module that includes ntifs.h to get the push locks and to export locking operations to your other modules. Build it, test and measure it, and see if it helps. Mark Roddy On Mon, May 4, 2009 at 1:32 PM, <xxxxx@gmail.com> wrote: > When I switch to fast mutex from eresource I see about 5-10% perf > improvements on low and medium loads with small lists and tables. On heavy > loads perf drops (of course) because of concurrency issues. Majority of > the time (in the real life) loads are low and tables are small, but that's > the corner cases of high loads and big tables that my driver must handle, > that make me using eresources. Again, there is nothing fancy going on > under the lock: just AVL table lookups, where keys are strings. So, the > only thing to optimize here is to optimize RtlCompareUnicodeString, which > doesn't give me any leverage. > <...excess quoted lines suppressed...> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4052 (20090504) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 4052 (20090504) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com
  Message 11 of 17  
04 May 09 15:36
Maxim S. Shatskih
xxxxxx@storagecraft.com
Join Date: 20 Feb 2003
Posts To This List: 10395
EX_PUSH_LOCK

I would reverse-engineer the ExXxxPushLock routines and write my own = with the same logic. --=20 Maxim S. Shatskih Windows DDK MVP xxxxx@storagecraft.com http://www.storagecraft.com <xxxxx@gmail.com> wrote in message news:123513@ntdev... > .. is it real? I mean I really want to use it, but it seems that only = filter manager provides an API for it and linking to fltmgr library just = to use push locks seems to be excessive. So, I wander, is it a WDK bug = (not exposing ExXxx push lock API), or it is yet another undocumented = feature that just leaked through filter manager, because advanced FCB = header has a push lock in it?=20 >
  Message 12 of 17  
04 May 09 16:41
Peter Viscarola (OSR)
xxxxxx@osr.com
Join Date:
Posts To This List: 5949
List Moderator
EX_PUSH_LOCK

While I COMPLETELY AGREE that optimizing ones sync primitives is pretty far down on the list of "things that cause performance problems" it seems that most of you guys are being terribly DOUR about this. Push Locks are excellent synchronization primitives. While ERESOURCES are nice, they are not exactly lightweight. In fact, I went looking to see if push locks were exported recently myself... I was sad to only find them in the Filter Manager. Would somebody over at Microsoft PLEASE consider exporting push locks for driver dev use in the near future? They've been around since XP, are obviously stable, and provide VERY good performance when non-contended. Peter OSR
  Message 13 of 17  
04 May 09 16:48
Doron Holan
xxxxxx@microsoft.com
Join Date: 08 Sep 2005
Posts To This List: 10090
EX_PUSH_LOCK

Push locks, at least the ones in the kernel, have had their perf characteri= stics tweaked over the past 2 releases. Tweaks which would have not been c= ompatible with the previous behavior. Now, there is no reason we cannot ha= ve both a public and private version of the lock where one is more stable t= han then other... d -----Original Message----- From: xxxxx@lists.osr.com [mailto:bounce-365060-26293@lists.o= sr.com] On Behalf Of xxxxx@osr.com Sent: Monday, May 04, 2009 1:40 PM To: Windows System Software Devs Interest List Subject: RE:[ntdev] EX_PUSH_LOCK While I COMPLETELY AGREE that optimizing ones sync primitives is pretty far= down on the list of "things that cause performance problems" it seems that= most of you guys are being terribly DOUR about this. Push Locks are excellent synchronization primitives. While ERESOURCES are = nice, they are not exactly lightweight. In fact, I went looking to see if = push locks were exported recently myself... I was sad to only find them in = the Filter Manager. Would somebody over at Microsoft PLEASE consider exporting push locks for d= river dev use in the near future? They've been around since XP, are obviou= sly stable, and provide VERY good performance when non-contended. Peter OSR --- NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit:=20 http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.o= sronline.com/page.cfm?name=3DListServer
  Message 14 of 17  
04 May 09 16:53
Peter Viscarola (OSR)
xxxxxx@osr.com
Join Date:
Posts To This List: 5949
List Moderator
EX_PUSH_LOCK

<QUOTE> Push locks, at least the ones in the kernel, have had their perf characteristics tweaked over the past 2 releases. Tweaks which would have not been compatible with the previous behavior. </QUOTE> Live and learn. Still, please consider the request? We don't really have an optimized "lock on a local flag, devolve to a wait if there's contention" kind of standard lock exported for driver use... not counting spinlocks, of course, which devolve to a busy wait. :-) I think this would be a very useful lock for driver devs, if somebody could just find the time to get it into the kit, Peter OSR
  Message 15 of 17  
04 May 09 17:00
Doron Holan
xxxxxx@microsoft.com
Join Date: 08 Sep 2005
Posts To This List: 10090
EX_PUSH_LOCK

I completely agree that it would be useful, sorry if that was what came acr= oss. I was addressing the "they are stable" statement. I will spin up a co= nversation here to see what the push lock story is d -----Original Message----- From: xxxxx@lists.osr.com [mailto:bounce-365066-26293@lists.o= sr.com] On Behalf Of xxxxx@osr.com Sent: Monday, May 04, 2009 1:52 PM To: Windows System Software Devs Interest List Subject: RE:[ntdev] EX_PUSH_LOCK <QUOTE> Push locks, at least the ones in the kernel, have had their perf characteri= stics=20 tweaked over the past 2 releases. Tweaks which would have not been compati= ble=20 with the previous behavior. </QUOTE> Live and learn. Still, please consider the request? We don't really have an optimized "loc= k on a local flag, devolve to a wait if there's contention" kind of standar= d lock exported for driver use... not counting spinlocks, of course, which = devolve to a busy wait. :-) I think this would be a very useful lock for driver devs, if somebody could= just find the time to get it into the kit, Peter OSR =20 --- NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit:=20 http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.o= sronline.com/page.cfm?name=3DListServer
  Message 16 of 17  
04 May 09 18:20
Mark Roddy
xxxxxx@gmail.com
Join Date: 25 Feb 2000
Posts To This List: 4014
EX_PUSH_LOCK

On Mon, May 4, 2009 at 1:53 PM, Don Burn <xxxxx@acm.org> wrote: > It is not just the ntifs.h it is the flt stuff and having the fltmgr.sys > running, It is a test to see if this affects performance (in a positive manner). The answer may be 'no' in which case the hack can be discarded. If on the other hand the answer is 'yes' then the OP can figure out if the risk is acceptable. And then also the subsequent issue of why push locks are boxed into fltmgr comes into play. Mark Roddy
  Message 17 of 17  
04 May 09 23:10
john bloe
xxxxxx@yahoo.com
Join Date: 16 Apr 2004
Posts To This List: 30
EX_PUSH_LOCK

http://winprogger.com/blog/?p=131
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the ntdev list to be able to post.

All times are GMT -5. The time now is 07:04.


Copyright ©2015, OSR Open Systems Resources, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license