Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV

Before Posting...

Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


8K sector size

OSR_Community_UserOSR_Community_User Member Posts: 110,217
Hi,

Does any version of Windows supports 8k sector size(physical sector) based storage devices.

I found a link which says that windows doesn't support. "http://support.microsoft.com/kb/2510009"

Just wanted to confirm. Is there a way to support it ?

Thanks,
Amogha

Comments

  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 9,162
    I can confirm for you that Windows does not support sector sizes greater than the Memory Manager's page size. Yuck! Imagine the issues involved.

    Now... you asked "is there a way to support it?" Well, you know, there's ALWAYS a way. I'd have to think about this some more, but I suspect you could write a filter that would handle this.

    Peter
    OSR

    Peter Viscarola
    OSR
    @OSRDrivers

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Thanks a lot Peter

    With 8K sector size, we were able to do raw io on the disk. But NTFS file system format was failing.
  • Peter_Viscarola_(OSR)Peter_Viscarola_(OSR) Administrator Posts: 9,162
    Just a passing note: IIRC, 8K sector size doesn't work on Linux either. I seem to remember an extensive rant by Linus Penguin Pants on this topic, and how 8K sectors were "stupid" or evil or something.

    Peter
    OSR

    Peter Viscarola
    OSR
    @OSRDrivers

  • Alex_GrigAlex_Grig Member Posts: 3,238
    Why would you want to support 8K? Any increase in storage density and in throughput (by skipping overhead) is very marginal, and is not worth trouble.

    I can understand wanting 4K sectors instead of 512, but 8K doesn't make any further advantage.
  • anton_bassovanton_bassov Member MODERATED Posts: 5,281
    > Just a passing note: IIRC, 8K sector size doesn't work on Linux either.

    Please note that Linux block layer does not have hardwired sector sizes. Although sector counts and offsets are indeed managed in terms of 512-byte units , the block layer performs the actual I/O in units of the the size that you have specified - sector size is a parameter to the disk's request queue, rather than the disk itself. Therefore, sectors of the sizes other than 512 bytes will work just fine as long as you are careful enough to remember to make a proper translation - as I have said already, you will be getting the requests that are presented in terms of 512 byte units. However, in practical terms, it would be quite unwise for you to try "non-standard" sector sizes. More on it below

    > I seem to remember an extensive rant by Linus Penguin Pants on this topic, and how 8K
    > sectors were "stupid" or evil or something.

    Don't forget that kernel is not the first one to access the disk - it has to be loaded by someone. Even if your driver works fine with "non-standard" sector size , the rest of the world, including both BIOSes and bootloaders, is normally not prepared to deal with anything other than 512-byte sectors.


    To make things even worse, in order to make it work the kernel has to align all transfers on 8K boundaries from the beginning of the partition. However, if the partition itself is not properly aligned, your 8K block will overlap two physical sectors. Taking into consideration that, for the historical reasons, misaligned partition is a norm, rather than exception, it does not really seem to be an optimal outcome, don't you think. Certainly,
    you can format the disk the way you wish, but, by doing so, you will make it Linux-specific -AFAIK, Windows is just not going to work with partitions that are "properly" aligned for 8K sectors, so that you can forget about dual-booting....



    In other words, it looks like he had made a good point - I don't know why you call it "rant". I hope that my post also explains to the OP why the whole idea of 8K sectors is particularly stupid in the Windows
    world ....


    Anton Bassov
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Does sector size really matter? My memory is that all disks supported
    512-byte sectors through emulation, and the large on-drive caches were
    used to accumulate data for the 8K sector writes. Does anyone have any
    data to support this?
    joe


    > Why would you want to support 8K? Any increase in storage density and in
    > throughput (by skipping overhead) is very marginal, and is not worth
    > trouble.
    >
    > I can understand wanting 4K sectors instead of 512, but 8K doesn't make
    > any further advantage.
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    >I can understand wanting 4K sectors instead of 512, but 8K doesn't make any further advantage.

    My understanding is a major reason for the shift to 4K disk sectors is the ECC code wastes a lot less space than on smaller sizes. As disks have become larger, the ECC codes of the past have become inadequate for decent data integrity. By spreading the ECC code over a 4K chunk instead of a 512B chunk, they can use a longer ECC code, which improves data integrity.

    I'm not really a ECC scientists, so don't know the tradeoffs in block length vs ECC code length, but could imagine the overhead for ECC codes goes down even more for block sizes larger than 4K, so if you went to 8K sectors instead of 4K, you could deliver the same size disk with greater data integrity. On a large storage subsystem, undetected bit rot is a real concern. I believe things like flash also don't really naturally fit 4K as a chunk size, as you need to erase in much larger chunks like 128K. I could see value in having the storage subsystem be flexible enough to accommodate evolving storage technologies.

    It's like if everybody is going through the pain of fixing the 512B assumption, you might was well just make it highly configurable.

    Jan
  • Alex_GrigAlex_Grig Member Posts: 3,238
    A single block is a smallest piece of data the block device can read and write.
    You want to be able to at least read/write a single page. This is why you don't want a block larger than a page.
  • anton_bassovanton_bassov Member MODERATED Posts: 5,281
    > On a large storage subsystem, undetected bit rot is a real concern.

    This problem has been recognized by Sun more than a decade ago, so that they started working on ZFS
    that was, unlike other file systems at the time, from the very beginning designed with data integrity in mind. Linux and DragonflyBSD subsequently followed the move with respectively Btrfs and HAMMER, although, AFAIK, at the moment of this writing none of them yet matches ZFS. AFAIK, MSFT also followed the same path with their ReFS, although very little about its internals seems to be known outside of Redmond.

    Anton Bassov
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Hi all,

    Thanks for the information.

    I have few more doubts.

    1. If the PAGE_SIZE is increased to 8K, then does the OS supports? Is there is way to increase PAGE_SIZE?

    2. Does it has anything to do with 32 bit or 64 bit version of Windows?

    3. Will it be supported in Windows 8 ?

    Thanks
  • anton_bassovanton_bassov Member MODERATED Posts: 5,281
    <quote>

    I have few more doubts.

    1. If the PAGE_SIZE is increased to 8K, then does the OS supports? Is there is way to increase PAGE_SIZE?

    2. Does it has anything to do with 32 bit or 64 bit version of Windows?

    3. Will it be supported in Windows 8 ?

    </quote>


    Well, if you have some doubts about the above issues, I am afraid it is too early for you to do any practical kernel-level programming - at this moment you seem to be unaware even about the details of physical-to-virtual translation. Not that you need to do it yourself, but you really have to understand it - otherwise
    you will be making one mistake after another. Go and get yourself a copy of "Windows internals", plus read
    Intel/AMD manuals....


    Anton Bassov
  • Alex_GrigAlex_Grig Member Posts: 3,238
    >1. If the PAGE_SIZE is increased to 8K, then does the OS supports? Is there is
    way to increase PAGE_SIZE?

    It's hardcoded into all components that depend on it.

    >2. Does it has anything to do with 32 bit or 64 bit version of Windows?

    Only Itanium (RIP!) version has PAGE_SIZE of 8K. There is NO Itanium Windows since Windows 2008 R2.

    >3. Will it be supported in Windows 8 ?

    A business case for that is exactly ZERO.
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > Hi all,
    >
    > Thanks for the information.
    >
    > I have few more doubts.
    >
    > 1. If the PAGE_SIZE is increased to 8K, then does the OS supports? Is
    > there is way to increase PAGE_SIZE?

    PAGE_SIZE is a propery of the hardware platform. To change it, you have
    to (a) be a hardware manufacturer designing your own CPU chip and (b)
    convince Microsoft you will sell millions of units, thus justifying te
    considerable effort required to build a special version of Windows to
    support your hardware.

    [Hint: We saw how well that worked for MIPS, the DEC Alpha, the PowerPC
    and the Itanic. I suspect that the combined sales of all these chips is
    less than a week's output of Intel chips. Maybe even a day's output.]
    >
    > 2. Does it has anything to do with 32 bit or 64 bit version of Windows?

    no. See above answer. Both of these hardware platforms use 4K pages

    >
    > 3. Will it be supported in Windows 8 ?

    Sure. See the answer to the first question. If you are a chip vendor,
    and have made a business case that you will sell millions (or tens of
    millions) of copies of Windows, no question about it. Lacking that, the
    answer would be a very solid "no".
    >
    > Thanks
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 14,852
    [email protected] wrote:
    > 1. If the PAGE_SIZE is increased to 8K, then does the OS supports? Is there is way to increase PAGE_SIZE?
    >
    > 2. Does it has anything to do with 32 bit or 64 bit version of Windows?

    Expanding on what others have said, you need to understand that the
    choice of a page size is not dictated by the operating system. It is
    dictated by the memory-management unit in the CPU. Intel's x86
    processors have 4k pages and 4M pages. That's it.

    --
    Tim Roberts, [email protected]
    Providenza & Boekelheide, Inc.

    Tim Roberts, [email protected]
    Software Wizard Emeritus

  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    On 27/11/2012 18:35, Tim Roberts wrote:
    > Intel's x86
    > processors have 4k pages and 4M pages. That's it.

    Recent 64-bit processors also support 1GB pages.

    --
    Bruce Cran
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    Thanks
  • OSR_Community_UserOSR_Community_User Member Posts: 110,217
    > On 27/11/2012 18:35, Tim Roberts wrote:
    >> Intel's x86
    >> processors have 4k pages and 4M pages. That's it.
    >
    > Recent 64-bit processors also support 1GB pages.
    >
    I knew about the 4Mb pages, but since the likelihood of getting disks with
    4MB sector size is vanishingly small, it didn't seem worthwhile to raise
    te idea. I had not known about the 1GB pages in Win64.
    > --
    > Bruce Cran
    >
    > ---
    > NTDEV is sponsored by OSR
    >
    > For our schedule of WDF, WDM, debugging and other seminars visit:
    > http://www.osr.com/seminars
    >
    > To unsubscribe, visit the List Server section of OSR Online at
    > http://www.osronline.com/page.cfm?name=ListServer
    >
  • anton_bassovanton_bassov Member MODERATED Posts: 5,281
    >> Recent 64-bit processors also support 1GB pages.

    > I knew about the 4Mb pages,

    Actually, 4M pages are on 32-bit processors and not 64-bit ones - the latter ones support 2M pages (and 1G ones) because 4K page table may hold only 512 entires on 64-bit system and not 1024 ones. Just a note....


    Anton Bassov
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started.

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 9-13 Sept 2024 Live, Online
Developing Minifilters 15-19 July 2024 Live, Online
Internals & Software Drivers 11-15 Mar 2024 Live, Online
Writing WDF Drivers 20-24 May 2024 Live, Online