DeviceIoControl OutBuffer issues

Hi ntdev,

Implementing the invert call model. Basically getting some device status information back in a static buffer of about 80 bytes via side channel communication with a control device. Update frequency varies between 5 to 50 times/sec.

The driver works stable, no issues. We need to use some legacy code for the UI written in Delphi / Object Pascal. Issue is on both FreePascal and Delphi compiler. Depending on how many bytes the OutBuffer has, it might work or not or only a bit.

The error we are getting back from DeviceIoControl is 6 = Invalid_Handle. But the handle from CreateFile is valid. We double checked this. We also check the buffer size in the driver and the user app.

Depending on the OutBuffer size we have cases like:
* 1st call is ok, the 2nd … 4th fails with error code 6 and then all other calls are fine.
* But there are sizes where it doesn’t work at all or always works.

Note: The failure pattern is always static for a given buffer size.

Using the same code in Visual Studio C, it always works, no matter what OutBuffer size we use. So we checked alignment, tried different memory allocation methods in Delphi (including LocalAlloc), no change.

What could be the issue?

>Using the same code in Visual Studio C, it always works, no matter what OutBuffer size we use.

Try to move the interface between the app and the driver in a DLL compiled with Visual Studio C and just let the app load the library and invoke its exported functions.

You might be passing wrong handle to DeviceIoControl. Set a breakpoint in ntdll!DeviceIoControl and see. Delphi could mangle your call parameters.

Assuming your driver doesn’t set the completion status to something that translates to Invalid Hanlde… then Invalid handle (in Win32, at least) ONLY means invalid handle. It doesn’t mean anything else. The size of the buffer doesn’t have any relevance. The request isn’t even reaching the driver.

Sounds to ME like something funky is going on with your Delphi/Pascal (people still use these? sorry…) run time. I like D.T’s suggestion of moving the interface to C language DLL. That should make it easier to debug.

Peter
OSR
@OSRDrivers

DeviceIoControls on Delphi really work, there must be a problem with your
code. If you wish I can have a look.

No matter what others may think of it, Delphi for me is the only reasonable
choice when there’s a requirement of a user interface. The compiler may be
the fastest in the world, doing millions of lines of code in a few seconds.
The language and VCL framework are designed by the guy who also designed C#,
they bought him away for a reason. Yet VCL remains the most logical and
flexible framework imaginable. Anything else is uphill struggle. The
compiler nowadays does Android, MacOs, iOS and Linux too.

//Daniel

Thanks for all your suggestions!

@Peter, we a native code restriction. There aren’t much native UI frameworks for Windows. MFC, QT and Delphi.

@Daniel, yes you are is right. Delphi was fantastic 20 years ago and is still ok these days, even the IDE has not really evolved much.

I’ll take you offer and email you my sample code for review.

Ah, there you go. Too true.

There’s .Net Native… program in C#, get native code out the back end.

OK… that’s a BIT of disingenuous. It’s new and it’s a UWP thing. Never mind. I hear you.

Peter
OSR
@OSRDrivers

xxxxx@gmx.net wrote:

@Peter, we a native code restriction. There aren’t much native UI frameworks for Windows. MFC, QT and Delphi.

WTL never got the respect it deserved. It was (and is) a fabulous
framework. I still use it for my UI projects.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

Daniel made my week!

Of course I was using Delphi 32bit user app on a 64bit machine and as Daniel pointed out the OVERLAPPED had the wrong pointer sizes! How on earth could I miss such a basic thing!

Changing OVERLAPPED to 8 byte pointers fixed the issues, any buffer size works now flawless. It was pure (alignment) luck that it worked with some buffer sizes.

As a side note: It is very likely we move away from Delphi and use FreePascal / Lanzarus IDE. Any Platform!, much nicer Editor, ways more customisation. They just want too much money for their latest Delphi versions.

Peter


From Daniel:
So, a 32-bit Delphi app with a 64-bit driver will not work in any case because the pointer sizes mismatch. I suggest to never use data types that have different sizes on 32 and 64-bit, so always reserve 8 bytes for a pointer.

Pack the structures to avoid possible alignment issues. So:
#pragma pack(1)
typedef struct …
#pragma pack()

and in the Delphi code, use “packed record” instead of just record for the structures so :
OVL_WRAPPER = packed record

xxxxx@gmx.net wrote:

As a side note: It is very likely we move away from Delphi and use FreePascal / Lanzarus IDE. Any Platform!, much nicer Editor, ways more customisation. They just want too much money for their latest Delphi versions.

Ah, memories. I was a HUGE Delphi believer in the 1990s, and was a
serious beta tester of Delphi 2. The IDE was sensible, the IDE and the
compiler were both very fast (as most Borland products were), and
learning their VCL component library taught me how to use C++ more
intelligently. When Anders Hejlsberg left Borland for Microsoft, Delphi
kind of lost its vision.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

But why he got invalid_handle

On 23 Mar 2017 10:29 p.m., “Tim Roberts” wrote:

xxxxx@gmx.net wrote:
> As a side note: It is very likely we move away from Delphi and use
FreePascal / Lanzarus IDE. Any Platform!, much nicer Editor, ways more
customisation. They just want too much money for their latest Delphi
versions.

Ah, memories. I was a HUGE Delphi believer in the 1990s, and was a
serious beta tester of Delphi 2. The IDE was sensible, the IDE and the
compiler were both very fast (as most Borland products were), and
learning their VCL component library taught me how to use C++ more
intelligently. When Anders Hejlsberg left Borland for Microsoft, Delphi
kind of lost its vision.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.


NTDEV is sponsored by OSR

Visit the list online at: http:>

MONTHLY seminars on crash dump analysis, WDF, Windows internals and
software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at <
http://www.osronline.com/page.cfm?name=ListServer&gt;</http:></http:>

>But why he got invalid_handle

The OVERLAPPED structure contains a handle to an event object that the I/O Manager sets to the signal state when the I/O operation has completed.

But I do not understand why the OP needs to define his own OVERLAPPED structure. Isn’t the IDE supposed to provide it ?

(I was confused too)

/shrugs

Peter
OSR
@OSRDrivers

The IDE is supposed to help you write code. That is why it is called an Integrated Development Environment after all.

The IDE itself has no interest in the code that you choose to write; either its quality or correctness

Having said all of that, the comment is quite irrelevant as you have likely gathered.

The purpose of an OVERLAPPED is to track the status of an IO request (an IRP in KM) and one must exist for each IOP that is issued. This is fundamental to the NT kernel design. When calling from UM, either the caller must provide an appropriate structure or the API functions will.

When the caller wants to provide additional context beyond what the base OVERLAPPED the standard method is to define a new struct that contains OVERLAPPED as its first member and additional members afterwards.

Sent from Mailhttps: for Windows 10

From: xxxxx@gmail.commailto:xxxxx
Sent: March 23, 2017 2:39 PM
To: Windows System Software Devs Interest Listmailto:xxxxx
Subject: RE:[ntdev] DeviceIoControl OutBuffer issues

>But why he got invalid_handle

The OVERLAPPED structure contains a handle to an event object that the I/O Manager sets to the signal state when the I/O operation has completed.

But I do not understand why the OP needs to define his own OVERLAPPED structure. Isn’t the IDE supposed to provide it ?


NTDEV is sponsored by OSR

Visit the list online at: http:

MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:

To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:></mailto:xxxxx></mailto:xxxxx></https:>

> The IDE itself has no interest in the code that you choose to write; either its quality or correctness

Of course it is more than an IDE. I don’t know how an SDK is called for the PASCAL language but it is needed.

It’s like in C, you have header files for the system calls. But if you are using the 32 bit Headers on a 64 bit Windows,
you are in troubles as the OVERLAPPED pointers have different sizes (4 byte in 32bit and 8 byte in 64 bit). That was my
mistake, which Daniel pointed out correctly.

On 24/03/17 12:51, xxxxx@gmail.com wrote:

> The IDE itself has no interest in the code that you choose to write; either its quality or correctness
Of course it is more than an IDE. I don’t know how an SDK is called for the PASCAL language but it is needed.


NTDEV is sponsored by OSR

Visit the list online at: http:
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at http:</http:></http:></http:>

For the records - It would have been too easy.

The test console app did work with various buffer length (after I put in Daniels suggestion), but the real user app
failed again. Ok, low level debugging to see whats actually going on.

Ok finally here is the answer to all the confusion:

  1. it may help to use a 64bit user-app on a 64-bit Windows, or a 32-bit user-app on a 32bit Windows.

  2. it may help to do the packed structure alignment

>> Now the important part <<<

  1. the developer **MUST** zero the OVERLAPPED structure otherwise the error might be invalid_handle. It’s an invalid OVERLAPPED.hEvent handle!

Well, read the manual!

All console test apps and real world apps now work without any issues on any buffer length! It doesn’t matter which compiler used.

Again thanks for your help out there!