OSRLogo
OSRLogoOSRLogoOSRLogo x Subscribe to The NT Insider
OSRLogo
x

Everything Windows Driver Development

x
x
x
GoToHomePage xLoginx
 
 

    Sat, 21 Oct 2017     115112 members

   Login
   Join


 
 
Contents
  Online Dump Analyzer
OSR Dev Blog
The NT Insider
Downloads
ListServer / Forum
Driver Jobs
  Express Links
  · The NT Insider Digital Edition - May-June 2016 Now Available!
  · Windows 8.1 Update: VS Express Now Supported
  · HCK Client install on Windows N versions
  · There's a WDFSTRING?
  · When CAN You Call WdfIoQueueP...ously

Update: Two New System Calls in NT 4.0 SP2!

 Click Here to Download: Code Associated With This Article Zip Archive, 21KB

With NT 4.0 Service Pack 2, Microsoft quietly introduced two new Win32 calls, with corresponding support in NT 4.0's kernel-mode components (NTOSKRNL.EXE). These two calls, ReadFileScatter and WriteFileGather, provide Win32 programs with the ability to read data from an uncached file into a discontiguous buffer, and conversely, to write data to an uncached file from a discontiguous buffer. However, these new calls are not referenced from any component shipped as part of SP2.

So what are the new calls doing there, and what could have prompted Microsoft to suddenly augment the Win32 API in the second service pack to a major operating system release? The answer is also one that highlights the tight integration between the operating systems and applications groups at Microsoft: the performance of a major Microsoft application, SQL Server, is significantly enhanced by using these new services. An on-line Knowledge Base article called "Performance Enhancements for SQL Server Under Windows NT" documents the functions (That’s an article I’d look at for Win32 updates, wouldn’t you?), which are very useful for applications that perform their own data caching like SQL Server. It is widely known that in the markets that these products occupy, a performance edge held for only a short period of time can be an enormous competitive advantage, and Microsoft has allegedly given themselves just such an advantage.

To level the playing field and further document some useful APIs, we’re presenting these nifty routines for those of you that may have applications that can take advantage of them (wouldn’t it be nice if we could all get our own system calls added to NT?). In addition, we’ll cover the NT API forms of the calls, which will provide insight into the way Win32 connects to NTOSKRNL services.

This article is organized as follows:

v     The Win32 API This section describes the two new Win32 API functions in a manner similar to the Win32 SDK.

v     The NT API This section describes the three new NT kernel-mode API functions for use from a device driver, in a manner similar to the Windows NT DDK.

v      Conclusion Some notes and observations on using the new APIs.

v      Example An example of ReadFileScatter can be found at the OSR web site, http://www.osr.com.

The Win32 API

ReadFileScatter

BOOL ReadFileScatter(
HANDLE
hFile, // handle of file to read
LARGE_INTEGER lpBuffer[], // array describing buffer
DWORD nNumberOfBytesToRead, // number of bytes to read
LPDWORD lpNumberOfBytesRead, // address of number of bytes read
LPOVERLAPPED lpOverlapped // address of structure for data
);

The ReadFileScatter function reads data from a non-cached file into a possibly discontiguous buffer, which is described as an array of page-aligned virtual addresses, with each address specifying the location of one page in the buffer. SP2 supports only the asynchronous form of this call, where an overlapped I/O structure is passed as a parameter. The file must be located on a local drive.

Parameters

Parameter

Description

hFile

Identifies the file to be read. The file handle must have been opened with GENERIC_READ access, as well as with FILE_FLAG_OVERLAPPED and FILE_FLAG_NO_
BUFFERING
flags.

lpBuffer[]

This is a large integer aligned array of virtual addresses that describe the buffer being read into. Each entry in the array represents one page of the buffer and must be a page-aligned address. Enough entries must be present to describe a buffer with pages for the pages indicated by nNumberOfBytesToRead.

nNumberOfBytesToRead

Specifies the number of bytes to be read from the file. This must be page-size granular (i.e. 4096, 8192, etc.).

lpNumberOfBytesRead

In the final implementation, this parameter will be used in a manner identical to the use of the same parameter in ReadFile. In the SP2 implementation, this field must be NULL, or the function will fail.

lpOverlapped

Points to an OVERLAPPED structure. Unlike for ReadFile, in the SP2 implementation of ReadFileScatter, this parameter is required, and consequently the hFile must have been created with FILE_FLAG_OVERLAPPED. See the Win32 SDK or Visual C Help for a description of this data structure.

Return Values

If the function succeeds, the return value is non-zero. If the function fails, the return value is zero. To get extended error information, call GetLastError.

Remarks

This function's API is constructed to mirror ReadFile's. However, in SP2 corners were cut with the implementation. Specifically, in ReadFile, lpOverlapped is an optional parameter and its absence will result in synchronous behavior of the call. In SP2's ReadFileScatter, lpOverlapped is mandatory and the synchronous behavior has not been implemented. To use ReadFileScatter the caller must create an event to pass in lpOverlapped, that will be signaled when the operation completes. At that time, GetOverlappedResult should be called to obtain the return status.

WriteFileGather

WriteFileGather has a prototype identical to ReadFileScatter's, and its SP2 implementation suffers the same limitations.

The NT API

Whereas it is possible for a Win32 program to obtain the addresses of ReadFileScatter and WriteFileGather, the related NT API functions are currently not accessible to device driver writers. Microsoft must release an updated NTOSKRNL.EXE which exports the functions and an NTOSKRNL.LIB that allows for linking to them. See the remarks section at the bottom for a hack that makes them accessible in an unconventional manner.

NtReadFileScatter

NTSTATUS NtReadFileScatter(
IN HANDLE FileHandle,
IN HANDLE Event, /* optional */
IN PIO_APC_ROUTINE UserApcRoutine, /* optional */
IN PVOID UserApcContext, /* optional */
OUT PIO_STATUS_BLOCK UserIosb, /* optional */
IN LARGE_INTEGER BufferDescription[],
IN ULONG BufferLength,
IN PLARGE_INTEGER ByteOffset,
IN PULONG Key /* optional */
);

Parameters

Parameter

Description

FileHandle

Handle to a file that has been opened for GENERIC_READ access and no intermediate buffering. The file must be located on a local drive.

Event

Handle to an event that will be signaled when the operation has completed.

UserApcRoutine

Points to an APC routine that will be called with the function completes. If this parameter is non-null, then Event must be null and vice versa.

UserApcContext

If an APC routine is specified, then this parameter will be passed to the routine when the operation completes.

UserIosb

Pointer to an I/O status structure that will be filled in when the operation completes.

BufferDescription

An array of large integers that describe the buffer being read into. The array must be large integer aligned, and contain virtual addresses that describe the buffer. Each entry is a large integer virtual address that specifies one page of the buffer. Note that the high DWORD of the large integers are ignored by SP2, but are present for future evolution into 64-bit NT.

BufferLength

The number of bytes to be read. This value must be an integral multiple of the system page size.

ByteOffset

Large integer offset in the file at which the read operation will begin.

Key

Passed directly to the file system, which may use it to control the order of I/O operations. No native NT file system currently honors the key parameter.

Return Value

NtReadFileScatter returns the NTSTATUS value that a lower level driver sets in the I/O status block for the request.

Comments

While the SP2 Win32 interface to this function (ReadFileScatter) is not completely implemented, NtReadFileScatter is fully implemented. The function is similar to NtReadFile, but performs the additional job of creating an MDL that describes the buffer specified in BufferDescription. The MDL is prepared using a new support function, MmProbeAndLockSelectedPages, which is described below.

NtWriteFileGather

NtWriteFileGather has the same interface as NtReadFileScatter.

MmProbeAndLockSelectedPages

VOID MmProbeAndLockSelectedPages(
IN OUT PMDL
MemoryDescriptorList,
IN LARGE_INTEGER PageList[],
IN KPROCESSOR_MODE AccessMode,
IN LOCK_OPERATION Operation
);

Probes the pages specified in PageList, making them physically resident and updating MemoryDescriptorList to describe them. This function has been added to support NtReadFileScatter and NtWriteFileGather.

Parameters

 

Parameter

Description

MemoryDescriptorList

Points to an MDL that supplies a virtual address, byte offset, and length. The physical page portion of the MDL is updated when the pages are locked in memory.

PageList

An array of large integer virtual addresses. Each virtual address must be page aligned. SP2's implementation ignores the upper 32-bits of the addresses, which are present to ease forward transition to 64-bit NT.

AccessMode

Specifies the access mode in which to probe the arguments, either KernelMode or UserMode.

Operation

Specifies the type of operation for which the caller wants the access rights probed and the pages locked, one of IoReadAccess, IoWriteAccess, or IoModifyAccess.

 

Comments

This function is a wrapper around MmProbeAndLockPages, and simply builds an MDL that describes each page specified in PageList.

Conclusion

Although SP2 does not ship with a Kernel32 .LIB file for linking to the new functions, ReadFileScatter and WriteFileGather can still be used. To obtain their entry points from a Win32 application, use the function GetProcAddress as shown below:

ReadFileScatter = (void *) GetProcAddress(GetModuleHandle( "Kernel32.dll" ),
"ReadFileScatter" );

A hack that would make the functions accessible from a device driver would be to make Zw-like function stubs in assembly language for each call that consist of system call traps with the appropriate system call numbers.

In reviewing the code for ReadFileScatter and WriteFileGather, it is interesting to note that they are only partially implemented. It appears the NT operating system group fleshed out the NT kernel implementations of the functions, while the Win32 team seems to have rushed to get the new calls into SP2 and sacrificed a full implementation in order to do so.

User Comments
Rate this article and give us feedback. Do you find anything missing? Share your opinion with the community!
Post Your Comment

Post Your Comments.
Print this article.
Email this article.

Writing WDF Drivers I: Core Concepts
LAB

Nashua (Amherst), NH
15-19 May 2017

Writing WDF Drivers II: Advanced Implementation Techniques
LAB

Nashua (Amherst), NH
23-26 May 2017

Kernel Debugging and Crash Analysis
LAB

Dulles (Sterling), VA
26-30 Jun 2017

Windows Internals and Software Driver Development
LAB

Nashua (Amherst), NH
24-28 Jul 2017

 
 
 
 
x
LetUsHelp
 

Need to develop a Windows file system solution?

We've got a kit for that.

Need Windows internals or kernel driver expertise?

Bring us your most challenging project - we can help!

System hangs/crashes?

We've got a special diagnostic team that's standing by.

Visit the OSR Corporate Web site for more information about how OSR can help!

 
bottom nav links