Driver Problems? Questions? Issues?
Put OSR's experience to work for you! Contact us for assistance with:
  • Creating the right design for your requirements
  • Reviewing your existing driver code
  • Analyzing driver reliability/performance issues
  • Custom training mixed with consulting and focused directly on your specific areas of interest/concern.
Check us out. OSR, the Windows driver experts.

OSR Seminars


Go Back   OSR Online Lists > ntdev
Welcome, Guest
You must login to post to this list
  Message 1 of 20  
02 Apr 18 14:01
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

Hello friends, My question involves the discrepancies in time I see when copying data from bulk storage (HDD, SSD, CD/DVD). If I do a transfer at a level lower than the filesystem (e.g. read \\.\CDROM0) I seem to get much faster speeds than by copying the files themselves, to the tune of a 5x increase in speed. It seems to me the creation and management of the files is what adds the overhead. Can anyone verify this, or propose an alternative? I have noticed something similar with FLOSS projects I use; their directory structures cause lots of lag due to IO and the IO they perform takes a long time. There are typically many files. My apologies for not having good timing data to discuss. I will see what I can come up with. If anyone can suggest a methodology I would appreciate it, it is not immediately obvious to me how I would compare same data transfer sizes. I realize this question may not be entirely appropriate, but I hope to get a good answer here. Cheers, R0b0t1
  Message 2 of 20  
02 Apr 18 14:11
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

On Mon, Apr 2, 2018 at 1:00 PM, R0b0t1 <xxxxx@gmail.com> wrote: > Hello friends, > > If I do a transfer at a level > lower than the filesystem (e.g. read \\.\CDROM0) I seem to get much > faster speeds than by copying the files themselves, to the tune of a > 5x increase in speed. > Sorry, the increase is more like 10-20x. 20 minutes vs. 3+ hours. Cheers, R0b0t1
  Message 3 of 20  
02 Apr 18 18:20
Alex Grig
xxxxxx@broadcom.com
Join Date: 14 Apr 2008
Posts To This List: 3235
Interaction of File Systems with Bulk Disk Access

I find it highly unlikely. Only if your file access is highly inefficient because you're using wrong access flags and patterns
  Message 4 of 20  
02 Apr 18 20:00
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

On Mon, Apr 2, 2018 at 5:19 PM, xxxxx@broadcom.com <xxxxx@lists.osr.com> wrote: > I find it highly unlikely. Only if your file access is highly inefficient because you're using wrong access flags and patterns > Are the right ones documented anywhere? If anyone wants to reproduce, populate a directory with lots of files with small names. I'll need to check if it is just very "wide" directories with this problem, or any fixed amount under a directory tree. > --- > NTDEV is sponsored by OSR > > Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev> > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! > Details at <http://www.osr.com/seminars> > > To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer>
  Message 5 of 20  
03 Apr 18 08:03
Peter Viscarola
xxxxxx@osr.com
Join Date:
Posts To This List: 6243
List Moderator
Interaction of File Systems with Bulk Disk Access

I?m not sure I understand your question. Does it not reduce to: If I copy a raw disk, directly, in big chunks... should this be faster than copying the same data from that disk first located and maintained via the file system and then read by the chance manager? Is it not natural and obvious that the file system and rest of the system will add overhead as they add additional ?value?? This is the reason that many backup products do direct disk reads (first reading the storage bitmap, then reading those sectors in large chunks), I stated of doing file by file copy. If you bypass the file system and VM subsystem, you naturally have less overhead this if you use them, right? Peter OSR @OSRDrivers
  Message 6 of 20  
03 Apr 18 14:23
Tim Roberts
xxxxxx@probo.com
Join Date: 28 Jan 2005
Posts To This List: 12008
Interaction of File Systems with Bulk Disk Access

xxxxx@osr.com wrote: > I?m not sure I understand your question. > > Does it not reduce to: > ... > If you bypass the file system and VM subsystem, you naturally have less overhead this if you use them, right? More than that, his original question mentioned a CD-ROM device.  CD-ROMs have absolutely dreadful random seek times.  I'm not at all to surprised to find a 10x or 20x performance improvement by streaming to/from a CD. -- Tim Roberts, xxxxx@probo.com Providenza & Boekelheide, Inc.
  Message 7 of 20  
07 Apr 18 20:55
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

On Tue, Apr 3, 2018 at 1:22 PM, xxxxx@probo.com <xxxxx@lists.osr.com> wrote: > xxxxx@osr.com wrote: >> I?m not sure I understand your question. >> >> Does it not reduce to: >> ... >> If you bypass the file system and VM subsystem, you naturally have less overhead this if you use them, right? > > More than that, his original question mentioned a CD-ROM device. > CD-ROMs have absolutely dreadful random seek times. I'm not at all to > surprised to find a 10x or 20x performance improvement by streaming <...excess quoted lines suppressed...> Well, I think the explanations given are sufficient. I think my confusion was from the lack of IOP shaping in the driver layers.
  Message 8 of 20  
08 Apr 18 15:59
Peter Viscarola
xxxxxx@osr.com
Join Date:
Posts To This List: 6243
List Moderator
Interaction of File Systems with Bulk Disk Access

<quote> IOP shaping </quote> I have no idea what that even is. Peter OSR @OSRDrivers
  Message 9 of 20  
08 Apr 18 18:24
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

On Sun, Apr 8, 2018 at 2:59 PM, xxxxx@osr.com <xxxxx@lists.osr.com> wrote: > <quote> > IOP shaping > </quote> > > I have no idea what that even is. > IO requests are rescheduled to improve performance. Newer drive control firmware also helps, by rescheduling writes so that e.g. a read of A, C, B does not move the head A -> C -> B but instead moves the head A -> B -> C (assuming A, B, and C are linear in space). My understanding is that the OS has to hold back requests instead of firing them off immediately for this to work well. It can also be used in a way similar to packet shaping, but for QoS management of disk bandwidth. Cheers, R0b0t1
  Message 10 of 20  
08 Apr 18 19:28
Peter Viscarola
xxxxxx@osr.com
Join Date:
Posts To This List: 6243
List Moderator
Interaction of File Systems with Bulk Disk Access

<quote> IO requests are rescheduled to improve performance </quote> We call that ?seek optimization? ? I?ve written a whole lot of that code in my career... but none for hard drives in the last decade plus. In my experience, this type of optimization is seriously out of date, because we can no longer guess at the true geometry of the drive. So, ?nearest sector first? and ?elevator service / next sector first? based on Starting LBA are no longer very effective. Windows has never had any such code in the OS (though it is entirely possible that one or two of the original disk drivers ? maybe the floppy driver ? MAY have had such code in them at some point. I don?t remember). But I can authoritatively tell you that no such code exists in Windows today. Peter OSR @OSRDrivers
  Message 11 of 20  
08 Apr 18 19:31
Bob Ammerman
xxxxxx@ramsystems.biz
Join Date: 05 Jun 2016
Posts To This List: 56
Interaction of File Systems with Bulk Disk Access

Yep, modern drives do all this internally. Hence command queuing. * Bob ? Bob Ammerman ? xxxxx@ramsystems.biz ? 716.864.8337 138 Liston St Buffalo, NY 14223 www.ramsystems.biz -----Original Message----- From: xxxxx@lists.osr.com <xxxxx@lists.osr.com> On Behalf Of xxxxx@osr.com Sent: Sunday, April 8, 2018 7:28 PM To: Windows System Software Devs Interest List <xxxxx@lists.osr.com> Subject: RE:[ntdev] Interaction of File Systems with Bulk Disk Access <quote> IO requests are rescheduled to improve performance </quote> We call that ?seek optimization? ? I?ve written a whole lot of that code in my career... but none for hard drives in the last decade plus. In my experience, this type of optimization is seriously out of date, because we can no longer guess at the true geometry of the drive. So, ?nearest sector first? and ?elevator service / next sector first? based on Starting LBA are no longer very effective. Windows has never had any such code in the OS (though it is entirely possible that one or two of the original disk drivers ? maybe the floppy driver ? MAY have had such code in them at some point. I don?t remember). But I can authoritatively tell you that no such code exists in Windows today. Peter OSR @OSRDrivers --- NTDEV is sponsored by OSR Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at <http://www.osr.com/seminars> To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer>
  Message 12 of 20  
08 Apr 18 19:51
Peter Viscarola
xxxxxx@osr.com
Join Date:
Posts To This List: 6243
List Moderator
Interaction of File Systems with Bulk Disk Access

>modern drives Thanks for that addition, Mr. Ammerman. I had meant to mention that as well. True for hard drives, and particularly true for SSDs... which have ?a whole lot of stuff going on? :-) Peter OSR @OSRDrivers
  Message 13 of 20  
09 Apr 18 09:30
Mark Roddy
xxxxxx@gmail.com
Join Date: 25 Feb 2000
Posts To This List: 4100
Interaction of File Systems with Bulk Disk Access

Where "modern" is like at least the last 20 years or so. Mark Roddy On Sun, Apr 8, 2018 at 7:31 PM, xxxxx@ramsystems.biz <xxxxx@lists.osr.com> wrote: > Yep, modern drives do all this internally. Hence command queuing. > > * Bob > > > Bob Ammerman > xxxxx@ramsystems.biz > 716.864.8337 > > 138 Liston St <...excess quoted lines suppressed...> --
  Message 14 of 20  
09 Apr 18 09:40
Don Burn
xxxxxx@windrvr.com
Join Date: 23 Feb 2011
Posts To This List: 1406
Interaction of File Systems with Bulk Disk Access

20 years or so for drives, some disk controllers were doing this over 35 years ago. Don Burn Windows Driver Consulting Website: <http://www.windrvr.com> http://www.windrvr.com From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com Sent: Monday, April 09, 2018 9:31 AM To: Windows System Software Devs Interest List <xxxxx@lists.osr.com> Subject: Re: RE:[ntdev] Interaction of File Systems with Bulk Disk Access Where "modern" is like at least the last 20 years or so. Mark Roddy On Sun, Apr 8, 2018 at 7:31 PM, xxxxx@ramsystems.biz <mailto:xxxxx@ramsystems.biz> <xxxxx@lists.osr.com <mailto:xxxxx@lists.osr.com> > wrote: Yep, modern drives do all this internally. Hence command queuing. * Bob Bob Ammerman xxxxx@ramsystems.biz <mailto:xxxxx@ramsystems.biz> 716.864.8337 138 Liston St Buffalo, NY 14223 www.ramsystems.biz <http://www.ramsystems.biz> -----Original Message----- From: xxxxx@lists.osr.com <mailto:xxxxx@lists.osr.com> <xxxxx@lists.osr.com <mailto:xxxxx@lists.osr.com> > On Behalf Of xxxxx@osr.com <mailto:xxxxx@osr.com> Sent: Sunday, April 8, 2018 7:28 PM To: Windows System Software Devs Interest List <xxxxx@lists.osr.com <mailto:xxxxx@lists.osr.com> > Subject: RE:[ntdev] Interaction of File Systems with Bulk Disk Access <quote> IO requests are rescheduled to improve performance </quote> We call that ?seek optimization? ? I?ve written a whole lot of that code in my career... but none for hard drives in the last decade plus. In my experience, this type of optimization is seriously out of date, because we can no longer guess at the true geometry of the drive. So, ?nearest sector first? and ?elevator service / next sector first? based on Starting LBA are no longer very effective. Windows has never had any such code in the OS (though it is entirely possible that one or two of the original disk drivers ? maybe the floppy driver ? MAY have had such code in them at some point. I don?t remember). But I can authoritatively tell you that no such code exists in Windows today. Peter OSR @OSRDrivers --- NTDEV is sponsored by OSR Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at <http://www.osr.com/seminars> To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer> --- NTDEV is sponsored by OSR Visit the list online at: <http://www.osronline.com/showlists.cfm?list=ntdev> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at <http://www.osr.com/seminars> To unsubscribe, visit the List Server section of OSR Online at <http://www.osronline.com/page.cfm?name=ListServer> --- NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at --
  Message 15 of 20  
09 Apr 18 10:11
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

There is quite a bit of interest in kernelspace reordering on Linux, which is what I was trying to refer to. For various reasons the firmware based optimizations are insufficient. So, if Windows does no optimizations, I think that answers my question. On Mon, Apr 9, 2018 at 8:40 AM, xxxxx@windrvr.com <xxxxx@lists.osr.com> wrote: > 20 years or so for drives, some disk controllers were doing this over 35 > years ago. > > > > > > Don Burn > > Windows Driver Consulting <...excess quoted lines suppressed...> in > my career... but none for hard drives in the last decade plus. In my > experience, this type of optimization is seriously out of date, because we > can no longer guess at the true geometry of the drive. So, ?nearest sector > first? and ?elevator service / next sector first? based on Starting LBA are > no longer very effective. Windows has never had any such code in the OS > (though it is entirely possible that one or two of the original disk drivers > ? maybe the floppy driver ? MAY have had such code in them at some point. I > don?t remember). But I can authoritatively tell you that no such code > exists in Windows today. > > Peter > OSR > @OSRDrivers > > > > software > drivers! > Details at <http://www.osr.com/seminars> > > To unsubscribe, visit the List Server section of OSR Online at > <http://www.osronline.com/page.cfm?name=ListServer> > > --- > NTDEV is sponsored by OSR > > Visit the list online at: software > drivers! > Details at <http://www.osr.com/seminars> > > To unsubscribe, visit the List Server section of OSR Online at > <http://www.osronline.com/page.cfm?name=ListServer> > > > > --- NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on > crash dump analysis, WDF, Windows internals and software drivers! Details at > To unsubscribe, visit the List Server section of OSR Online at > > > --- > NTDEV is sponsored by OSR > > Visit the list online at: > <http://www.osronline.com/showlists.cfm?list=ntdev> > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and software > drivers! > Details at <http://www.osr.com/seminars> > > To unsubscribe, visit the List Server section of OSR Online at > <http://www.osronline.com/page.cfm?name=ListServer> --
  Message 16 of 20  
09 Apr 18 10:36
Mark Roddy
xxxxxx@gmail.com
Join Date: 25 Feb 2000
Posts To This List: 4100
Interaction of File Systems with Bulk Disk Access

Actually if I recall correctly the adapter port drivers in windows had or have some sort of scheduling mechanism, not sure if they still do. It remains lame. The agents that understand how a specific disk hardware operates are the agents closest to the disk hardware, generally the firmware on the disk itself or in cases where the disks are tightly coupled to the adapter(s), the adapter firmware. Mark Roddy On Mon, Apr 9, 2018 at 10:11 AM, xxxxx@gmail.com <xxxxx@lists.osr.com> wrote: > There is quite a bit of interest in kernelspace reordering on Linux, which > is what I was trying to refer to. For various reasons the firmware based > optimizations are insufficient. So, if Windows does no optimizations, I > think that answers my question. > > > On Mon, Apr 9, 2018 at 8:40 AM, xxxxx@windrvr.com <xxxxx@lists.osr.com> > wrote: > > 20 years or so for drives, some disk controllers were doing this over 35 > > years ago. <...excess quoted lines suppressed...> --
  Message 17 of 20  
09 Apr 18 11:33
R0b0t1
xxxxxx@gmail.com
Join Date: 24 Mar 2017
Posts To This List: 123
Interaction of File Systems with Bulk Disk Access

Some databases (Postgres, MySQL) have gotten good performance increases by treating their backends more as storage controllers than as simple file access. This leads to some strange performance degradation on filesystems like ZFS, which also work similarly to a database, but can not optimize access based on data content. So it is not always knowledge of the physical media that counts. On Mon, Apr 9, 2018 at 9:36 AM, xxxxx@gmail.com <xxxxx@lists.osr.com> wrote: > Actually if I recall correctly the adapter port drivers in windows had or > have some sort of scheduling mechanism, not sure if they still do. It > remains lame. The agents that understand how a specific disk hardware > operates are the agents closest to the disk hardware, generally the firmware > on the disk itself or in cases where the disks are tightly coupled to the > adapter(s), the adapter firmware. > > Mark Roddy > > On Mon, Apr 9, 2018 at 10:11 AM, xxxxx@gmail.com <xxxxx@lists.osr.com> <...excess quoted lines suppressed...>
  Message 18 of 20  
09 Apr 18 14:36
Peter Viscarola
xxxxxx@osr.com
Join Date:
Posts To This List: 6243
List Moderator
Interaction of File Systems with Bulk Disk Access

<quote> Some databases (Postgres, MySQL) have gotten good performance increases by treating their backends more as storage controllers than as simple file access </quote> Hmmm... Most big and sophisticated database systems want the entire OS out of their way as much as possible. Basically, they REALLY want to be their own operating system. When it comes to disk storage, they definitely have no use for the file system provided by the OS. They can *always* do better... because they have something purpose-built for their needs. <quote> So it is not always knowledge of the physical media that counts. </quote> Well, yes and no. Knowledge of the physical media ALWAYS counts... and the physical media is almost always being optimized by the internal drive logic. In general, the more reads/writes you give the drive in parallel, the better the optimization will be. But... you're right... knowledge of the physical media is not the ONLY thing that counts. Having an on disk structure that's optimized for a given use can provide *tremendous* performance gains over a general purpose file system. For example, years ago there was a system that needed to acquire satellite telemetry that was streaming non-stop at a fairly high rate. The data needed to be recorded without loss, and over a significant period of time. The simple solution to this problem was to do raw (direct) I/O to the drives... and not even attempt to go through NTFS. I mean, if you're going to dump an "enormous" chunk of data onto disk, and you're always going to be the app that reads it back, you don't really need a file system to keep track of your data blocks, your VDL, and when the last time was that you modified the data. You just wanna barf the data onto the disk, perhaps add a timestamp or two and some judicious checksumming, and you're good to go. Bingo! You've created your own on disk structure. No file system required. Peter OSR @OSRDrivers
  Message 19 of 20  
10 Apr 18 20:01
Peter Wieland
xxxxxx@microsoft.com
Join Date: 16 Jul 2009
Posts To This List: 307
Interaction of File Systems with Bulk Disk Access

Windows had IO request reordering code based on LBA back in the 2000 era, and it was beneficial to performance back then. Of course that was back when dinosaurs roamed the earth, so ... It was in scsiport, so not immediately obvious to the outside world. It was in the per-LUN queue and was c-scan like, but IIRC performed when pulling the next request from the queue (rather than when submitting). It's been phased out in scsiport's successors. -p -----Original Message----- From: xxxxx@lists.osr.com <xxxxx@lists.osr.com> On Behalf Of xxxxx@osr.com Sent: Sunday, April 8, 2018 4:28 PM To: Windows System Software Devs Interest List <xxxxx@lists.osr.com> Subject: RE:[ntdev] Interaction of File Systems with Bulk Disk Access <quote> IO requests are rescheduled to improve performance </quote> We call that ?seek optimization? ? I?ve written a whole lot of that code in my career... but none for hard drives in the last decade plus. In my experience, this type of optimization is seriously out of date, because we can no longer guess at the true geometry of the drive. So, ?nearest sector first? and ?elevator service / next sector first? based on Starting LBA are no longer very effective. Windows has never had any such code in the OS (though it is entirely possible that one or two of the original disk drivers ? maybe the floppy driver ? MAY have had such code in them at some point. I don?t remember). But I can authoritatively tell you that no such code exists in Windows today. Peter OSR @OSRDrivers --- NTDEV is sponsored by OSR Visit the list online at: <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.osronline.co m%2Fshowlists.cfm%3Flist%3Dntdev&data=02%7C01%7Cpeterwie%40microsoft.com%7C6ea84b b93672452568bc08d59da8733c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636588269 171296687&sdata=SHLu9zm535zhivIsdpd%2FfMBYbUSkYkkF9tl%2FYQaR1UU%3D&reserved=0> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.osr.com%2Fse minars&data=02%7C01%7Cpeterwie%40microsoft.com%7C6ea84bb93672452568bc08d59da8733c %7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636588269171296687&sdata=iXcD2jryEF PA%2FNrGvkNT6emxMgd8zByfYotsV14sUP4%3D&reserved=0> To unsubscribe, visit the List Server section of OSR Online at <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.osronline.co m%2Fpage.cfm%3Fname%3DListServer&data=02%7C01%7Cpeterwie%40microsoft.com%7C6ea84b b93672452568bc08d59da8733c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636588269 171296687&sdata=FwlQ4ACrBUTtyUUQsC6lrLw%2Fq7OaeS9mVjkiH9YEHqU%3D&reserved=0>
  Message 20 of 20  
11 Apr 18 11:43
Peter Viscarola
xxxxxx@osr.com
Join Date:
Posts To This List: 6243
List Moderator
Interaction of File Systems with Bulk Disk Access

>Windows had IO request reordering code based on LBA back in the 2000 era Thanks, Peter. I appreciate the authoritative answer on this. Peter OSR @OSRDrivers
Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You must login to OSR Online AND be a member of the ntdev list to be able to post.

All times are GMT -5. The time now is 09:50.


Copyright ©2015, OSR Open Systems Resources, Inc.
Based on vBulletin Copyright ©2000 - 2005, Jelsoft Enterprises Ltd.
Modified under license