Previous Next

Physical Address Extension (PAE)

PAE Memory Support

Microsoft will continue to support Physical Address Extension (PAE) memory in Microsoft® Windows® Server 2003, Enterprise Edition and Datacenter Edition.

To ensure the highest possible performance and quality for customers purchasing or upgrading systems capable of supporting more than 4 GB of RAM, special Large (PAE) Memory Support hardware compatibility list (HCL) categories have been established, which will provide a listing of the systems, devices, and drivers that have passed testing of this particular capability. Only devices that are listed on the Large Memory Support HCL will be supported by Microsoft when more than 4 GB of RAM is installed in the system.

HCL Testing

Vendors who want to have their systems and devices tested, listed, and supported must send their equipment to the Windows Hardware Quality Lab (WHQL), following these basic guidelines:

Note  No further submissions to WHQL will be accepted for Large Memory testing and listing of 32-bit adapters. 64-bit component adapters and device drivers may be submitted now and in the future for Large Memory testing and listing.

Operating System Support

The PAE mode kernel will not be enabled by default for systems that can support more than 4 GB of RAM. The Windows Server 2003 Enterprise Edition and Datacenter Edition will boot by default in normal mode (support for only 4 GB of RAM).

To boot the system and utilize PAE memory, it will be necessary add the “/PAE” switch to the corresponding entry in the BOOT.INI file. If a problem should arise, Safe Mode may be used, and that will cause the system to boot using normal kernels (support for only 4 GB of RAM) even if the /PAE switch is part of the BOOT.INI file. The PAE mode kernel requires an Intel Architecture processor, Pentium Pro or later, more than 4 GB of RAM, and either Windows 2000 Advanced Server or Windows 2000 Datacenter Server.

Adapter and Driver Issues: LME and DAC Capable

All physical memory is treated as general-purpose memory, so no new APIs are needed to access I/O above the 4 GB physical memory address. Also, direct I/O can be done for greater than 4 GB physical addresses (this requires Dual Address Cycle (DAC) capable or 64-bit PCI devices). Devices and drivers that can perform direct I/O beyond 4 GB are considered Large Memory Enabled (LME).

Because Windows Server 2003 does not have a kernel PAE or LME API or interface, the PAE-X86 kernel ensures that many items are identical to the standard kernel, including:

However, careful device driver development is still required. Hardware devices should be DAC or 64-bit capable with LME drivers; otherwise, the device will function as “legacy” 32 bit and will be double buffered, with lower relative performance.

Although double-buffering can have a relatively small impact (single percentage points) on 8 GB systems, this is enough to impact I/O intensive tasks such as database activity. This is also dependent on a number of factors beyond Microsoft’s control, such as hardware design and device driver optimizations such as interrupt moderation and efficient use of the PCI bus. As the amount of physical memory increases, so does the negative performance impact in comparison to DAC/64-bit devices and LME drivers.

What to Avoid in Drivers

  1. Do not use PVOID64. Using PVOID64 anywhere will return incorrect information because this call does not return valid information on the Intel Architecture platform. Instead, use PHYSICAL_ADDRESS.
  2. Do not call MmGetPhysicalAddress on a locked buffer, discard the high 32 bits, and then program the adapter to DMA into the resultant address. This will certainly result in corrupted memory, lost I/O, and system failure. If this call is made, ensure that all address information returned is used and that the driver correctly operates with that 64-bit address.
  3. Do not use PVOID when manipulating physical addresses. Because PVOID is only 32 bits, address truncation will take place and memory corruption will result.
  4. Do not use ULONG when manipulating physical addresses because this has exactly the same precautions and behavior as PVOID: system failure.
  5. Do not indicate support for scatter/gather in the DEVICE_DESCRIPTION when not true to avoid the buffering provided by HAL (the “mapping registers”).
  6. If the driver cannot support 64-bit addresses, do not call IoMapTransfer without having an AdapterControl function (again, to avoid mapping registers), and do not supply zero as the value for MapRegisterBase. This will fail.

Other functions and calls might cause failures. This list will be expanded as more information becomes available and will be in later revisions of the Microsoft® Windows® Driver Development Kit (DDK), which is available at http://www.microsoft.com/ddk/ and distributed through MSDN®.

Guidelines for NDIS Miniports and SCSI Miniports

Guidelines for NDIS Miniports on PAE Systems

The mechanism for 64-bit address-capable network cards to operate above the 4 GB mark should use scatter/gather DMA;

General Guidelines

Guidelines for 64 bit Address-Capable SCSI Miniports
(Including All Related Adapters for SCSI 2 or later)

Guidelines for Legacy SCSI Miniports

Creating and Testing LME Drivers

  1. Always use the most recent version of the HCT provided on the Windows Hardware Quality Labs Web site.
  2. Test using all available tools from Microsoft such as DevCtl in the HCT and the Driver Verifier functionality.
  3. Test both on systems with greater than 4 GB of physical memory and on systems with less than 4 GB of system memory.
  4. Test legacy drivers to ensure there is no corruption due to incorrect use of calls such as MmGetPhysicalAddress.
  5. Test specifically for PAE systems. This requires a minimum installed RAM of 8 GB.
  6. Use the NOLOWMEM switch in the BOOT.INI file:
  7. Use DevCtl – supplied with the HCT:
  8. Use Driver Verifier – part of the product

Troubleshooting DAC Support and LME Drivers

These steps can help vendors and customers determine if there are any issues relating to the system board, buses, or adapters in supporting DAC and LME.

  1. If the driver fails at initialization, check with the system OEM to determine whether all PCI buses present in the system support DAC.
  2. If a network adapter driver performs a bug check immediately upon a network connection, determine whether all buses support DAC, again by checking with the OEM.
  3. If the PCI buses on the system are all DAC capable, check whether the hardware device is compliant with PCI 2.1 or later.
  4. If the bus supports DAC and the device is PCI 2.1 compliant, check the driver for assumptions being made about physical addresses.

The documentation we provide to our Windows Server 2003, Enterprise Edition and Datacenter Edition, customers concerning PAE memory can be found in Help topic "Enterprise Memory Architecture."

Hardware Requirements for PAE

The system must meet the following minimum requirements:

Determine Whether Network and Storage Adapters Support PAE

  1. Consult the Windows Server 2003 Hardware Compatibility List Web site and search for "Large Memory."
  2. If your hardware is not supported, update hardware or select different hardware on which to enable PAE. If appropriate, contact the hardware vendor.
  3. If you update your hardware, it is recommended that you back up your system before doing so. If necessary, consult user documentation for information on backing up your system.

Enable PAE x86

To enable PAE x86:

  1. Locate the Boot.ini file, which is typically in the root folder (for example, C:/) and remove its Read-Only and Hidden attributes.
  2. Open the Boot.ini file with a text editor, and then add the /PAE parameter to the ARC path, as shown in bold format in the following example:

    multi(0)disk(0)rdisk(0)partition(2)\WINNT="Windows 2000 Datacenter Server" /PAE /basevideo /sos

  3. On the File menu, click Save.
  4. Restore the Read-Only attribute to the Boot.ini file.

Troubleshooting Specific Programs

Problem: The computer will not start after PAE is enabled.
Cause: Your hardware may not support PAE.

Solution: Start the system and run Safe Mode, which disables PAE. Then remove the /PAE parameter from the Boot.ini file. To run Safe Mode:

  1. When you see the message "Please select the operating system to start," press F8.
  2. Use the arrow keys to highlight the appropriate Safe Mode option, and then press ENTER.

Note  NUM LOCK must be off before the arrow keys on the numeric keypad will function.

Problem: After PAE is enabled, the computer runs for a time and then displays a Stop error.
Cause: Your hardware may not support PAE.

Solution: Determine whether your hardware supports PAE by consulting the Windows Server 2003 Hardware Compatibility List Web site and search for “Large Memory.”

If your hardware does not support PAE, or if the problem persists and you continue to believe that PAE may be the cause, disable PAE by removing the /PAE parameter from the Boot.ini file.

WHQL, HCT, and HCL for Large Memory

The HCL has been modified to include the listing category —Large Memory— that will list all components that either support greater than 4 GB [DAC and LME adapters and drivers] or have been tested and known to function correctly in PAE systems.

Systems capable of supporting more than 4 GB RAM support must be submitted to WHQL at the maximum configuration the system supports for both processors and memory, but there will be no separate Large Memory category for these computers because they must pass testing in a PAE-enabled configuration to be listed at all.

Not every device and driver has been, or can be, tested and listed for the LME HCL. At this time, Microsoft does not plan to include any further 32-bit devices and adapters in the Large Memory HCL listings, because of the negative performance impact double buffering has on system performance. Microsoft is encouraging vendors to submit new 64-bit adapters and drivers for Windows Server 2003, Enterprise Edition and Datacenter Edition, systems supporting more than 4 GB of memory, which will be accepted by the WHQL for testing and listing in the LME HCL categories.

Note  There are no Large Memory HCL listings for many adapter types. Examples include Video, IEEE 1394, and USB and others, where, at this time, testing has exposed no problems operating in a PAE environment. In the case of Video, this is not an input device for systems with more than 4 GB of memory. In the case of IEEE 1394 and USB devices, these do not presently function as primary data path devices for this class of system. Other device classes such as Keyboards, Mice, Internal Modems, Audio, Smart Cards, Digital Cameras, IRDA Network, Scanner, Video Conference, Flash RAM, DVD-MPEG, Joystick and TV Tuners that do not presently have Large Memory HCL listings have also been found to present no problems at this time, although testing continues.

LME devices — Large Memory Enabled adapters — are the set of DAC/64-bit adapters and associated LME drivers for Storage and Network I/O. These are current devices that are DAC/64-bit capable for interfaces such as SCSI 2, SCSI 3, SCSI X, and Fibre Channel storage and for 100BaseT, Gigabit, ATM, FDDI, 100 Mb TR, and WSD-SAN for networking. These devices are typically part of the primary data path, which is defined as the data path over which the bulk of network I/O to clients and/or servers traverses for networking, the path over which the bulk of disk I/O traverses, or where the page files reside.

Built on Friday, April 11, 2003