The NT Insider typically discusses complex issues in designing and developing drivers and file systems. We figure these are the issues that people who develop NT systems software care and have questions about. I recently asked our friends in Microsoft’s PSS group what the most frequent question they received was. I figured it would be something about IRQLs or DMA or synchronization on SMP systems. Imagine my surprise when they told me that, by far, the most frequent question they receive was, "How do I write a driver for NT?"

This article is the first in a series in which we’ll answer some of the most fundamental questions about NT drivers. When you need to write one, what software and hardware you need to do driver development, how a driver development environment is set up, how you write a driver, and what some of the basic options are in designing and developing drivers.

Converting WinDos Control Apps

In Windows NT, unlike in Windows 95 and DOS, access to both I/O port space and device memory is restricted to kernel mode as a means of protection. A program that has access to hardware registers can adversely affect the stability of the system, and even crash it at will. Thus, user programs running under NT are not allowed to directly manipulate device registers.

Many programs initially designed for DOS or Windows, including a significant number of industrial control applications, run as simple user applications and perform some sort of device control by issuing "IN" and "OUT" instructions, or by reading and writing registers in memory mapped space. These programs will not run under Windows NT. How can these programs be "ported" to Windows NT?

The classic approach to supporting such programs on Windows NT is to write a standard Windows NT kernel mode device driver. The task of accessing the device’s registers is separated from the application program and moved to kernel mode. A new interface is defined for accessing the device using the Win32 I/O functions (such as ReadFile, WriteFile, and DeviceIoControlFile). Since the functions can be rather cumbersome for many applications programmers to use for device access, a driver interface DLL is often created. This DLL exports a set of abstract functions specific to the application, and when these functions are called they issue the appropriate Win32 I/O function calls to the device driver. The driver then interacts with the device via NT’s Hardware Abstraction Layer (HAL). This is shown in Figure 1.

This approach does have a number of advantages. One is that once the application is converted, it can be easily supported on both Windows NT and Windows 95 by simply changing the DLL used. On NT, the DLL issues I/O functions to the driver. On Win95, the DLL could continue to directly access I/O ports.

Unfortunately, even if you’re already comfortable with writing Windows NT device drivers this project is not necessarily as simple as it might at first seem. Unless the program was written very carefully in the beginning, perhaps with portability in mind, the major challenge in this project is likely to be extricating the device access functions from the other functions the application performs, and creating a clean interface between these two portions of the program. Defining the device driver API to be used can also be quite a challenge. And we haven’t even considered the situation where the application interacts with the device via interrupts.

For many applications there may be a reasonable alternative to writing a custom device driver. This is to use a pre-existing package where the driver work has been done for you. One such package is WinRT from Blue Water Systems (www.bluewatersystems.com ). Blue Water Systems provides you a predefined, low level, interface and an NT kernel mode driver (and a VxD for use in Win95) that implements it. You re-write your program using their interface, and voila! You now have a functional Windows NT control application.

This approach makes a lot of sense if your application is not extremely time critical and you don’t want to take the time to redesign your control application. This is, of course, assuming you do not anticipate the need for ongoing NT system software knowledge. If, for example, your development organization is full of scientists that specialize in fluid dynamics, knowing how to write NT drivers is probably not high on their list of important or interesting things to do. In this case, it might make sense to take a shortcut and use a package such as WinRT.

Figure 1

Believe it or not, we generally recommend that people not even consider writing a device driver unless they have to. Our feeling is that if you can accomplish what you need to do by using one of the existing drivers provided by NT, or by using a package like WinRT, you’ll likely be much better off than trying to write a custom driver on your own.

When Do You Need a Driver?

So, when does it make sense to write a driver? There are a number of reasons that designing and developing a device driver for your device might be the right answer.

First, you’ll typically want to write a driver if your device needs to be simultaneously available to multiple applications on the system. Whacking a few I/O ports is fine for a control app talking to a dedicated device. But if you want your device to receive requests from multiple applications, a driver gives you the opportunity to serialize those requests onto the device and return the results of each operation to the correct caller.

Another reason you’ll need a device driver is if you want your device to integrate into the NT system and even optionally take the place of functionally similar existing NT devices. For example, let’s say you’ve created a new Ethernet card, and you want it to integrate seamlessly into NT and support standard NT networking. To do this, you’re writing a driver. Or, for a very different example, suppose you’ve got a monolithic semiconductor memory device that you want to look like a disk to NT. You write a driver for your MonoMem device, indicate that it’s a disk type device, and the NT file systems will never know the difference between talking to your device and a "real" disk drive.

A third reason you might want to write a driver is to allow you to export a clean, well defined, "standard" type interface to your device. Let’s say you develop a PCI-based colorimeter. You’re just a manufacturer of lab hardware, however. How your customers actually use this device is up to them. You write the device driver for the device, and document the various DeviceIoControl codes that the driver supports. Maybe you even develop a sample driver interface DLL to talk to your device that implements that interface. Either way, developing a driver provides a clearly defined, standard, interface to your device.

A final reason you’ll need to develop a driver is if you want the lowest possible interrupt latency and the highest possible performance for your device. Packages like WinRT are terrific, but the very fact that they are generic solutions precludes maximizing performance for a particular device the way a custom solution can. Since drivers run in kernel mode you have ultimate control over the operating system. You can use this power to make appropriate trade-offs about the performance of your device versus the overall throughput of applications running on the system.

Getting the Stuff You Need

So you’ve decided to write a driver. How do you go about it? What do you need to get started?

Well, the first thing you’ll need is a compiler. Please, make it easy on yourself here. Use Microsoft’s Visual C++. For NT V4.0, you’ll need VC++ V4.1 or later. We often see people asking in the forums if you can use other vendor’s C compilers. Well, yes, you can, but trying to do so is likely to be an exercise in frustration. The Windows NT Device Driver Kit (DDK) and debugger software all assume that you’re using MSVC. Just give in. Use MSVC and avoid being unnecessarily annoyed. You’ll have plenty of opportunity to be annoyed when you’re trying to debug your driver.

But the compiler itself is not enough. The next thing you’ll need is a subscription to the Microsoft Developer’s Network (MSDN). Since MSDN subscriptions come in several different flavors, you’ll need a subscription to the "Professional Edition". But be careful! When you subscribe, you’ll need to specifically request the DDK (Device Driver Kit) CDs. While you’re at it, request all the international versions of the CDs too (they don’t cost any more money, and hey, you never know when you’ll need the Czech version of NT Workstation, right? Plus, if you’ve got a surplus of time and a sick sense of humor you can always come in over the weekend and install the Russian version of NT on your colleague’s workstation. I’m sure he’ll thank you for your effort).

For $500 bucks or so, your MSDN subscription will get you a set of about a zillion CDs. Even measured in terms of shear volume, your subscription is a good value. Each disk has a disk number, title, and date on it. Some of the disks are updated each month. The idea is to replace the disks in your set with the updated ones. For example, this year we were treated to a CD labeled: "Disk 11, Windows NT 4.0 Service Pack 3, June 1997" followed the next month by a CD labeled "Disk 11, Windows NT 4.0 Service Pack 3, July 1997". Makes you wonder what the difference is, doesn’t it? Never ask these questions.

You will find that most of the CDs in an MSDN subscription will be totally unrelated to writing drivers. For example, there’s the ever popular "Glossaries, ODBC Drivers, and Internet Explorer" CD (Are these three things related in some way?). Here at OSR, we are frequently tempted to throw these disks away (no, you can’t recycle them and they don’t make good coasters). But, if you’re smart, you’ll keep everything you ever get from MSDN. Even keep the old, "superceded" disks. You never know when you may need something for some older version of NT. Here’s an idea: Give the old disks to your secretary to file. That should keep him/her busy for quite a while.

While the contents of your MSDN CDs will doubtlessly provide you with hours of amusement, there are three things it provides that pertain specifically to writing drivers: The Microsoft Developer Network Library CDs, the Platform SDK, and the DDK.

The DDK contains the documentation and header files that are needed to write NT drivers. The documentation is provided in on-line format, accessed via InfoViewer. This makes it easily searchable, and it’s even moderately easy to print out hardcopies of large sections of the documentation for casual perusing. Since the DDK is a low-volume product relative to, say, the SDK, printed copies of the documentation are not available from Microsoft.

Most impressive, and useful, about the DDK is that it comes complete with source code examples of more than 50 real Widows NT drivers. The provided samples span (almost) the entire range of the NT driver-writing realm: From SCSI miniport drivers to standard kernel mode drivers and almost everything in between. Only file system examples are absent (and those are available as part of a separate kit). What’s particularly cool about the provided samples is that where the name of one of the sample drivers is the same as a "real" driver provided as part of the regular Windows NT (binary) distribution kit, the sample driver is (almost always) the actual source code for that driver on the NT kit. This allows you see and understand how many things work in the NT I/O subsystem. It also allows you to build and modify one of the provided sample drivers from source, substitute it for the existing NT driver, and watch how it works.

The SDK, or Software Development Kit, contains all the header files and tools normally required to build user mode applications for Windows NT. For driver building, we need only install the SDK tools. The most important tool that will be installed is WinDbg, the debugger from the SDK that supports kernel mode debugging.

Finally, there are the MSDN Library CDs. These CDs contains a wealth of wide-ranging information about software development on all the supported Microsoft platforms. The Library CDs include the Knowledge Base (KB). The Knowledge Base is a collection of articles written to address specific issues (often as the result of a problem or bug report). Some KB articles describe otherwise undocumented features or system behavior. Others describe known problems and work-arounds. The KB is searchable by keyword. It is worth your time to wander through the KB periodically and check what new things pop up.

So, you got the MS VC++ compiler. You have the Windows NT DDK, the Platform SDK, and the MSDN Library that came as part of your MSDN subscription. What next?

Prepare Your Systems -- Plural

Well, it would be nice to have some hardware. To properly develop drivers on Windows NT you need two complete systems capable of running Windows NT. One of these systems will typically be your development system, i.e. the system on which you edit and build your driver. This will also be the system on which you run the debugger. This system will be a stable system, not subject to the random crashes that debugging a new driver tends to cause. The other system will be your test system. This will be the system on which you will be running your driver under test.

I can just hear some readers starting to send me email telling me that you can use Numega Technologies’ SoftIce for Windows NT product and do driver development using a single system. That is certainly true. And, our friends at Numega (who live right down the street from us, actually) certainly have a wonderful product. However, even if you use SoftIce, we strongly recommend using two systems for debugging. I don’t know about you, but I have absolutely no desire to subject my nice stable development system to the vagaries of any new driver that’s under development. Those of you who wish to play disk drive roulette can go ahead and use a single system. But I value the files on my hard disk!

The next thing to do is to install MSVC++, the DDK, and the SDK tools on your development system. Also, to help maximize your chances of success, we recommend that you install MSVC++, the DDK, and the SDK in that order. Plus, it’s easiest if you install them on your system using the account from which you’ll be doing your driver development. By the way, for maximum productivity this system needs to be configured with as much CPU and memory as you can convince your manager to buy for you. CPU speed is always nice, but for happiness with NT, there is no substitute for loading up on memory. Here at OSR, for example, our development workstations are currently dual-processor 200MHZ Pentium-Pro systems, with at least 128MB of memory. Remember, NT is a workstation operating system, not a WinDos knock-off. It needs memory to make it work well. 128MB of memory, at current prices which are dirt cheap, is really very realistic.

We’ll talk more about setting up your development system later. The next thing you’ll need to do is set up a test system. The test system will be used to actually test and debug the driver that you’ll be running. The way kernel debugging works in Windows NT, at least when you’re using the Microsoft supplied tools, is that the debugger (WinDbg) runs on your development system and actually controls and monitors execution on the test system of the driver you’re developing. The two systems are connected by a serial null-modem cable.

The test system can be much more conservatively configured than your development system. In fact, it might even be advantageous to somewhat under configure your test system (since this will force more paging and "help" you find those IRQL_NOT_LESS_OR_EQUAL errors faster). Here at OSR we tend to use test systems with whatever Pentium CPU is available, and with at least 32MB of memory. (See Figure 2 for reference).

The next issue to deal with is precisely what software you’ll need to install on your test system. The good news is that the compiler, DDK, and SDK do not need to be installed on the test system. All you need is NT. But there is actually more than one version of NT that is used during driver development. These two versions are known as the "Free Build" (also sometimes called the "Retail Build") and the "Checked Build".

The Free Build is the normal, ordinary, Windows NT binary distribution kit. It doesn’t say anything unusual on the distribution CD. This version is intended for regular distribution, and is built using compiler optimizations and without much cross-checking or debug-specific code. In contrast, the Checked Build is a version of NT specifically designed for use by device driver writers. It is clearly labeled "Checked Build" on the distribution CD. The Checked Build is built with many fewer compiler optimizations (making the code easier to trace) and it also has the symbol "DBG" defined at compile time. Defining this symbol causes lots of debugging code to be included in the operating system. This debugging code checks many function parameters for "reasonableness", and in general attempts to identify and trap various run-time errors.

Figure 2

The Checked Build is only intended for use on a test system. While it could be used on a regular development system, the inclusion of all the debugging code, and lack of compiler optimizations, makes it run much slower than the Free Build. This is because when you install the Checked Build, you get the checked version of everything. You get a checked NTOS operating system, a checked HAL, plus the checked versions of every single driver and kernel mode DLL in the system.

At OSR, what we usually do is install the Free Build on our test systems, and then replace the individual Free Build components of interest to us with their Checked Build counterparts. So, if for example we’re writing a standard kernel mode driver, we’ll install the Free Build of NT on the test system, and then replace the Free version of the operating system (NTOSKRNL.EXE or NTKRNLMP.EXE) and HAL (HAL.DLL) with the same files from the Checked Build.

As with so many things, finding and replacing these files may not be quite as simple as it sounds. First, it’s important to realize that if you do this, you absolutely must replace the HAL and NTOS image together. You can’t have, for example, the Checked version of the HAL and the Free version of NTOS. They either both have to be Checked or they both have to be Free or the system will not boot. These files all live in the \SystemRoot\System32 directory. To replace the Free versions with the Checked versions, just copy the files you want to replace from the \i386 (Intel platform) directory of the Windows NT (Checked Build) distribution disk supplied with MSDN.

There are still a couple of "tricks" left to discuss, however. One is that you might need to determine which HAL your test system is using. The file in \SystemRoot\System32 is always called HAL.DLL. But the original HAL file that was copied to create this file might be something else. This will be true, for example, when your test system is a multiprocessor system. To find out which HAL is installed on your test system, check contents of the file \SystemRoot\Repair\setup.log. This file will contain a line such as the following:

\WINNT\system32\hal.dll = "halmps.dll","1a01c"

This shows that the HAL being used on the current system is actually HALMPS.DLL. If you have trouble even finding setup.log in the \SystemRoot\Repair\ directory, it’s because it’s a hidden file. Just attrib it –h and you should have no problem!

The remaining trick is that the image of the HAL on the Checked distribution kit might be compressed. This is a good trick to know in general, since which files are and are not compressed on the Windows NT kit disk change from time to time. You can identify a compressed file by a file type that ends with an underscore. Thus, if the HALMPS image is not compressed on the Checked distribution kit, you’ll be able to find it in the \i386\ directory and it will be named HALMPS.DLL. On the other hand, if the file is compressed, it will be in the \i386\ directory and it will be named HALMPS.DL_.

To expand a compressed file, use Microsoft’s Expand utility. The easiest way we’ve found to do this is to copy the desired file to its ultimate location (i.e. copy HALMPS.DL_ from the \i386\ directory on the Checked Build CD to \SystemRoot\system32\HALMPD.DL_). Then simply use the expand command from the command shell, as follows:

This expands the file, and automatically names the resulting expanded file halmps.dll.

More to Come

Sorry, but there’s a limit to the amount of fun we’re allowed to have in a single issue. Next time, we’ll talk in more detail about setting up your development and test systems to enable debugging. We’ll also start to talk about the general procedure for building and debugging drivers on NT. Until then, keep cranking that code!